Update: We no longer recommend using GitHub / GitLab as your QMS software. I wrote down our thoughts in a separate post (read it here) because I didn’t want to destroy the structure of this one. The arguments in this one are still valid, but right now, I’d weigh them differently and come to another conclusion.
After my QMS software comparison which I published almost two years ago, I’ll now finally follow this up with some very concrete advice on how to set up your own QMS using software which is not shady. Like GitHub and GitLab. Everyone likes those. So why not set up your QMS in them?
First things first: I’m a dinosaur and mainly know GitHub (also, do you remember Sourceforge?). So, for the sake of simplicity, let’s only look at GitHub for now. I trust that you’ll be able to perform the mental gymnastics to map these concepts to GitLab. You know, pull requests are called merge requests in GitLab and so on.
Before we get started, let’s have a quick look at the features we need from GitHub to actually set up a QMS. Other consultants generally love to overcomplicate this by saying “but damn, you can’t validate it!” without even knowing what validating means in this context. Hm. But as always, let’s stop ranting and look at the features we need:
- We must be able to review and sign documents in some way.
- We need to keep old versions of documents around.
Wait, that’s it? Yup, turns out, setting up a QMS is simple and you don’t need expensive compliance software. That being said, there are some small problems which we’ll discuss later on. But let’s get into the first point, reviewing and signing stuff.
Reviewing and (Electronically) Signing QMS Documents in an Old-School, Paper-Based QMS
Before we get into our GitHub implementation, let’s first look at how this is done in an old-school paper-based QMS (yes, I’m serious).
You’d print out your documents, say, your SOP for software development, on a sheet of paper. At the bottom of the last page there would be an area for people to sign. It would look something like this:
| Action | Role | Person | Date | Signature |
|---|---|---|---|---|
| Creation | Software Developer | Dr. Oliver Eidel | 2022-05-19 | (signature) |
| Review | Software Development Lead | |||
| Approval | CEO |
The creation section is already filled out because I’m the dude who created the draft.
The review and approval sections aren’t filled out yet. That’s because those should be filled out when the relevant person actually signs it (duh).
Next, this sheet of paper gets transported to the reviewer. I imagine that happening through an autonomous robot-car which drives through the office and transports important documents (like this SOP) in brown envelopes. Remember the series The Americans? No? Never mind.
Okay, who’s the reviewer? That’s a person who works as “Software Development Lead” in the company. Say, that’s me, Dr. Oliver Eidel. I’ll fill in my name, add today’s date, and sign it:
| Action | Role | Person | Date | Signature |
|---|---|---|---|---|
| Creation | Software Developer | Dr. Oliver Eidel | 2022-05-19 | (signature) |
| Review | Software Development Lead | Dr. Oliver Eidel | 2022-05-19 | (signature) |
| Approval | CEO |
Now, is it a problem that the same person reviews it who also created it? Let’s discuss that. Here, lie down on the couch and tell me about your feelings.
No, seriously, it’s probably okay if your company is very small. But if it’s larger, you probably should have reviewers who are different humans than you.
Okay, now, the document continues to travel in the robot vehicle and lands on the desk of the CEO who will also sign it. Maybe that’s also me. So after that step, it would look like this:
| Action | Role | Person | Date | Signature |
|---|---|---|---|---|
| Creation | Software Developer | Dr. Oliver Eidel | 2022-05-19 | (signature) |
| Review | Software Development Lead | Dr. Oliver Eidel | 2022-05-19 | (signature) |
| Approval | CEO | Dr. Oliver Eidel | 2022-05-19 | (important CEO signature) |
And that’s it! The document is approved! Great success. The company hasn’t built any new products or generated any new revenue, but we’ve increased our precious cache of regulatory documents which will surely make everyone more happy. Or at least some auditors.
Okay, so, from this you’ve hopefully gained the understanding how signing works in an old-school world.
How can we implement this in GitHub?
Reviewing and (Electronically) Signing QMS Documents in GitHub / GitLab
The basic idea would be that “documents” are markdown files in a git repository. “Signing” works by having a signing table at the bottom of each markdown document, and the whole “reviewing workflow” thing would be implemented through pull requests in which reviewers are added as, you guessed it, reviewers.
Let’s do this for a document - let’s take our intended-use-template. We’re not actually going to take it, but I just thought I’d make things more concrete by mentioning it.
Before we get started, we need to create a new GitHub repository, of course.
Now you would create a new branch in that repository, and name it intended-use-draft or so. You would always
create new branches for when you want to edit things, because editing things always happens in pull
requests.
Next, copy the markdown content of our template to a new markdown file in your git repository. What sort of folder structure should you create? I don’t know. Come up with your own which makes sense. Maybe we’ll have a video on that in the future.
At the bottom of the file, you’d need to add a signing table like the one we saw above. That’s because our implementation of “signing” documents will include people to write out their names and initials in the actual markdown document.
You’ll now edit that document and make changes to it, just like when filling out any other regulatory document based on our templates.
When you’re done with it, you’ll fill out the “creation” section of the signing table like this:
| Action | Role | Person | Date | Signature |
|---|---|---|---|---|
| Creation | Software Developer | Dr. Oliver Eidel | 2022-05-19 | OE |
| Review | Software Development Lead | |||
| Approval | CEO |
Yup, that looks exactly like the signing table above, except that I’ve written my initials in the signature field instead of signing it with a wet-ink-pen.
Is that a valid signature?
Yes, for EU MDR compliance purposes, that works, because you can prove through your git commit history that those initials were written by a specific person - the person who actually performed the signature, hopefully.
For FDA signing purposes it could get a bit more tricky. The FDA has a few more (and higher) requirements for e-signatures, specified in 21 CFR Part 11, and I’m not sure you could make GitHub fulfil those. You’d have to try it out, I guess. Spoiler and obvious ad here: Our own QMS software, Formwork, creates FDA-compliant signatures.
Okay, now that you’ve created your document draft in GitHub (or rather, git), it’s time to send it out for review. So you save your changes to the file, commit and push, and create a new pull request. You add the reviewer and approver to the pull request (as reviewers in GitHub).
Now, how does the review work?
They’ll review the document and then “sign” it by adding a suggestion to the signing table. So, if I’m the reviewer, I would add a code review comment with a suggestion like so:
```suggestion
| Review | Software Development Lead | Dr. Oliver Eidel | 2022-05-19 | OE |
Oh, and the end of the suggestion section would also need three backticks but that seems to break our minimalistic blogging software here. If you’re wondering what the hell this suggestion thing is, here’s some information on GitHub.
And that’s how reviewing and signing works. The same would now be done by the approver, in this case, the CEO. They would also perform a GitHub “review” and add their initials to the signing table as a code review suggestion.
When everyone has signed, the creator (that’s me!) would merge the pull request. That way, the document is “released”. So, merging to master is releasing something. That also means that your master branch will always contain the most recent released versions of your documents.
Did you understand it? How do you like that? Well, let me go first - there are benefits and drawbacks.
QMS in GitHub / GitLab: Pros and Cons
Before I dive into my usual criticism, let me clear the air and highlight some of the great benefits of using GitHub or GitLab.
It’s good software. The companies behind them are nice companies and not evil -let’s not get into the whole Microsoft debate here, regulatory software companies are much more evil. It’s very cheap or free. It’s well-tested, predictable and understandable - you don’t need to read a manual or attend a weird user training to understand GitHub.
The most obvious and biggest benefit is that git is a near-perfect version history tool and light years ahead of any “normal” regulatory software which offers some sort of change history tracking of documents. That means you can show your auditors the exact change history of each document and which user made which change. That’s really cool! Some auditors don’t even know git (yep, but let’s not go there today..) and you can positively surprise them by showing them something like git blame or so.
Compared to e.g. Google Docs, you wouldn’t need to keep a separate “change history” table at the bottom of each document in which you would describe recent changes in a human-readable way. Nope, you can just use the commit history for that. That’s cool.
It’s easy to customize. We won’t be diving into that today, but e.g. with GitHub actions and the API you can build some cool features into your QMS.
It’s portable, well, mostly - you can simply clone your git repository and move your QMS somewhere else. But this “export” is not entirely complete! Because we’re using pull requests, we’d also need to export those, including the code review comments and suggestions. Those aren’t in git. Ugh. So we can’t do a complete QMS export, unfortunately.
Those were the good parts. Now let’s dive into the bad parts.
Comments and Suggestions
Let’s address the elephant in the room first: What happens if reviewers are unhappy with the draft and want to re-write significant portions of it? This is easy in Google Docs where you can simply toggle the suggestion mode and add suggestions. In GitHub, this is really tricky.
Sure, you can add comments and, technically, you can add suggestions. But it’s really clunky for editing documents. I don’t think any sane person would even attempt to rewrite a markdown document in GitHub’s web interface.
I often recommend clients to create drafts in Google Docs if they’re still unsure about the contents of their document. That way, many people can add suggestions and things remain organized. This would make sense here, too. And then good luck manually copy-pasting those Google Docs to markdown files and fixing the formatting.. ugh.
Markdown Is Tricky, Sometimes
The second-biggest elephant in the room is that everyone on your team must be capable of writing markdown documents and interacting with the GitHub UI. As coders, this may seem like an easy skill to us. But I’ve attempted to teach these skills to a few non-coders in the past, and it’s actually really hard! You have to start from the basics, like “what the hell is markdown” and “what the hell is a git branch”.
Not an enjoyable experience (also not for the “students”). And because they don’t continue to interact with GitHub on a daily basis like us coders do, they tend to forget things and regularly get stuck, requiring time-intensive help from the more technically proficient folks. Ugh.
My rule of thumb is: If your team has at least one person who will be doing regulatory work and is non-technical, I don’t think that GitHub will work for you.
Linking Stuff and Markdown Tables Are Hell
Many regulatory documents end up being lists. If you’d be using Google Drive, you would create a spreadsheet for those. Our software-requirements-list-template is a good example for that. You could just take the table section of it and copy-paste that to a Google Sheet.
When using GitHub as your QMS, things are less easy. You’d have to create actual markdown tables in your markdown files to accommodate that data. And then editing it gets cumbersome. Needless to say, you can’t sort columns or use any other features which a spreadsheet tool like Google Sheets would provide.
A related problem is linking stuff. You’d want to link your software requirements to software tests - in other words, each software requirement would need to be covered by at least one software test.
There’s no great way to do this in markdown documents. You could have a table with software requirements in which reach requirement has a unique ID, and then you’d “link” them by writing down the software test IDs which they test. Ugh. Obviously, as soon as you change certain IDs, your data may become “corrupted” because those text-written IDs don’t automatically correct themselves.
To be fair, Google Sheets doesn’t fare better for this problem. But Formwork does.
Summary: GitHub / GitLab as a QMS
Let’s sum up everything in case you need to get back to this later when choosing your QMS. Read on past the summary to hear my opinion on whether I would build a QMS on GitHub or GitLab.
How to Sign Documents?
Write your initials into the markdown file.
How to Review Documents?
Use pull requests / merge requests.
Benefits of GitHub / GitLab as a QMS
- Generally speaking, good software.
- Manufacturers are non-evil companies.
- Cheap or even free.
- Perfect version history.
- Extendable via API, GitHub Actions.
- Most data can easily be exported (markdown documents in git), though not all (pull requests / merge requests).
Drawbacks of GitHub / GitLab as a QMS
- Adding comments and suggestions to documents is painful. A lot of friction if you’re trying to collaborate on a document, e.g. with team members or consultants.
- Everyone on your team has to be able to write markdown docs and understand GitHub / GitLab and pull requests / merge requests.
- e-signatures are probably not FDA (21 CFR Part 11) compliant. So you can only use this for EU MDR medical devices.
- Table-like documents which would be best suited for spreadsheets have to be handled as markdown tables which is painful.
- Linking stuff between documents / items is “text-based”.
Would I choose it?
So, would I choose GitHub or GitLab as a QMS software?
Intuitively, you might think I would like GitHub / GitLab because there are non-evil companies behind them, you can easily export your data and writing markdown files is doable for technical people.
However, I actually don’t recommend them. Why?
It’s just that, for any given company situation, a git-based QMS seems like a really bad fit. Let’s go through some examples:
Your company is getting started and wants to write documents collaboratively.
If you’re working with a consultant (like, um, us) or just collaborating internally to create documents with multiple people involved, working together on a markdown document in GitHub is nearly impossible.
You need a text editor which has some sort of suggestion mode. I recommend Google Docs for this use case, nothing else comes close (Sharepoint sucks by the way, stay away!).
Your company has more than 3 employees.
This will mean that you likely have at least one person who’s not a coder and can’t handle markdown files an pull requests. There will be lots of friction working with that person. This will only get worse when your company grows and more non-technical people join. I don’t think it’s possible.
Your product is rather complicated and linking entries in markdown tables sounds like a terrible idea.
If your software has, say, more than 50 software requirements and/or tests, handling those in a markdown table sounds like a recipe for disaster. You’d need a QMS software tool which stores this data in a structured way (cough, Formwork). To be fair, Google Sheets would also not handle this very well.
Your company needs FDA compliance.
Yup, already discussed, not much to argue here. GitHub / GitLab “e-signatures” most likely aren’t 21 CFR Part 11 compliant.
Summary
Yup, I generally wouldn’t choose GitHub / GitLab for your QMS. Looking through the scenarios above, there’s really only one scenario which could work: If your company is tiny (1-5 people), comprised only of software developers who can handle markdown, and your product is rather simple. Then it could work.
Am I wrong? Maybe. Feel free to reach out to me, especially if you managed to beat the odds and set up your QMS in GitHub / GitLab :)