GitHub QMS: We No Longer Recommend It
While I wrote up a rather balanced post on the pros and cons of using GitHub / GitLab as your QMS software, I’d like to follow this up with our combined learnings after working with multiple clients who have been using GitHub or GitLab as their QMS software. And boy, the results aren’t pretty! The quick summary is that, right now, I’d advise anyone to not use GitHub or GitLab as their QMS, full stop, no exclusions.
But why? Let’s get into the reasons. I won’t touch upon the pros and cons I mentioned in the other post and mainly focus on the things that commonly go wrong or rather explode spectacularly. I also assume you have a basic understanding of GitHub or GitLab for now - if not, just pretend you do and read on!
But why? Let’s get into the reasons. I won’t touch upon the pros and cons I mentioned in the other post and mainly focus on the things that commonly go wrong or rather explode spectacularly. I also assume you have a basic understanding of GitHub or GitLab for now - if not, just pretend you do and read on!
GitHub QMS Problem #1: Editing Markdown Tables Is Near-Impossible
In every QMS you end up with documents, and in some of those documents, you end up with tables. Those tables are often quite lengthy. It seems to be impossible to prevent this, regardless of what you do. Maybe you use Jira (spoiler: don’t) and you’ve set up fancy tickets for your structured data. Great, but somewhere, deep in a Confluence QMS document, someone still secretly starts creating tables because they think it’s a good idea, and often that’s not true, but there’s no other way. Even if you’re using the best QMS software the planet (it’s ours - I might be biased here), some documents still contain tables. I know this because we run our own ISO 13485 QMS on Formwork and we have documents with tables.
So having tables in documents seems to be a law of nature, and utterly unpreventable regardless of what you do. Tables are like Gollum in Lord of the Rings, no one really likes him but he constantly starts creeping up on you and in the end you need him to throw the damn ring into Mount Doom. So we need tables.
This is where your GitHub QMS, or more generically Markdown documents, will fail. As soon as your tables contain more than, like, five rows or columns, things will get very, very messy. Especially with many columns leading to very wide tables. Or simply line breaks or lists inside tables. Ugh. A huge mess.
Now you may think that surely you can avoid this? Nope. Some of your tables will become wide. Things like your Software Requirements List, SOUP list and traceability matrices, among others. So you’ll have to deal with this.
How can you deal with this? There are some text editors which have some capabilities for editing Markdown tables, but none of them come close to the usability of spreadsheets or other software. So - not really an option.
I’ve seen the following workarounds and all of them are, in their own way, spectacularly ugly:
So having tables in documents seems to be a law of nature, and utterly unpreventable regardless of what you do. Tables are like Gollum in Lord of the Rings, no one really likes him but he constantly starts creeping up on you and in the end you need him to throw the damn ring into Mount Doom. So we need tables.
This is where your GitHub QMS, or more generically Markdown documents, will fail. As soon as your tables contain more than, like, five rows or columns, things will get very, very messy. Especially with many columns leading to very wide tables. Or simply line breaks or lists inside tables. Ugh. A huge mess.
Now you may think that surely you can avoid this? Nope. Some of your tables will become wide. Things like your Software Requirements List, SOUP list and traceability matrices, among others. So you’ll have to deal with this.
How can you deal with this? There are some text editors which have some capabilities for editing Markdown tables, but none of them come close to the usability of spreadsheets or other software. So - not really an option.
I’ve seen the following workarounds and all of them are, in their own way, spectacularly ugly:
- Use spreadsheets (xlsx) for tables, and commit (save) them to git: Ugh! So you start putting table-like data in Microsoft Excel spreadsheets, save those as normal Excel files (xlsx) and commit them to git. You lose many benefits of using git for your QMS, like being able to edit things in your browser and viewing the diff of changes. So ugly.
- Use Google Sheets for tables, save them elsewhere to GDrive: This is similar to the previous solution. Only this time, you edit your spreadsheets in Google Sheets and save them to GDrive. The only elephant in the room is that now your QMS resides in two places: git and GDrive. So it’s probably a two-headed elephant in your room. How do you do versioning in GDrive? How will you know where each document is located? Good luck hunting for documents during your audits - we’ve seen this ourselves and it wasn’t pretty. If you move parts of your QMS to GDrive, why not move all of it to GDrive? GDrive is a more reasonable choice than GitHub / GitLab.
- Use a fancy text editor and smart human being for editing Markdown tables: Typically, there’s at least one software developer in each company who is very capable of using Markdown and git. In fact, it was often this person who made the fatal git QMS decision in the first place. So, once the git QMS and its tables have degenerated into a big mess, they get called in to do the editing. From now on, the entire company relies on them to edit these tables with their fancy text editor because no one else can. They become the bottleneck for all Markdown table editing. What a disaster.
Okay, so this is already bad. But that’s not all of it. While we’re talking about people, let’s talk about those next!
GitHub Problem #2: Non-Technical People “Using” Markdown
I commonly see startups set up a GitHub QMS while they’re still small (3 people) and mostly engineers. That’s understandable because git really is a good version control system.
But there’s a gigantic catastrophe on the horizon: Onboarding the first non-technical person onto this QMS. Like a regulatory affairs employee or an outside consultant.
In the past, I thought this was a solvable problem. Just give them some resources and time to learn Markdown and git, no problem! Or just teach them to use the GitHub / GitHub web UI and they should be fine!
In the meantime, my opinion has changed. I think it’s not possible to achieve this within the constraints of a normal company. Non-technical people aren’t very interested in learning these sort of things - they just want to get their job done and start writing documents! Also, while learning git and Markdown seems easy to learn for us software developers, it’s a daunting task for non-technical people who haven’t had any coding experience whatsoever.
To be clear here, I’m not ranting about non-technical people at all. Instead, I’m pointing out the absurdly high learning curve. It’s like when you drive to a golf resort to play some golf (not that I do that) but the instructor takes you back to an iron forging factory where you have to forge your own golf clubs first. You spend time on this totally unnecessary thing instead of getting the job done.
Now, how do the workarounds look like?
But there’s a gigantic catastrophe on the horizon: Onboarding the first non-technical person onto this QMS. Like a regulatory affairs employee or an outside consultant.
In the past, I thought this was a solvable problem. Just give them some resources and time to learn Markdown and git, no problem! Or just teach them to use the GitHub / GitHub web UI and they should be fine!
In the meantime, my opinion has changed. I think it’s not possible to achieve this within the constraints of a normal company. Non-technical people aren’t very interested in learning these sort of things - they just want to get their job done and start writing documents! Also, while learning git and Markdown seems easy to learn for us software developers, it’s a daunting task for non-technical people who haven’t had any coding experience whatsoever.
To be clear here, I’m not ranting about non-technical people at all. Instead, I’m pointing out the absurdly high learning curve. It’s like when you drive to a golf resort to play some golf (not that I do that) but the instructor takes you back to an iron forging factory where you have to forge your own golf clubs first. You spend time on this totally unnecessary thing instead of getting the job done.
Now, how do the workarounds look like?
- Attempt to teach the non-technical person: Not viable. I haven’t seen this work out. If you try to force this, they’ll either quit or create a mess in your git QMS, or both.
- The prior QMS maintainer has to continue working on it: Quite realistic if you don’t want to migrate away from GitHub / GitLab. Whoever set it up has to continue working on it. That person may also become the “editing assistant” of the new regulatory employee because that person is not able to edit things in git.
- Migrate to another system: Yep, this is the most realistic one. Good luck migrating old document versions and your git history to another system.
GitHub QMS Problem #3: Document Review and Signing
I think the two above points are enough, but there’s a final one I’d like to touch upon. It’s about document review.
How would your typical document review workflow look like? You might opt for using pull requests or merge requests, and specific people have to create approvals for a pull / merge request to pass. So far, so good.
But this really assumes so many things, and for each of this things, you have to be hyper-disciplined: Each merge request should, best case, only make changes to one document, not many. Merge requests should actually get merged soon as otherwise the branches go stale and you end up with a huge bunch of branches with unfinished edits (ugh!). Good luck findings those branches in audits. And finally, this also assumes that you know what sort of edits you want to make and don’t have any need for discussing them with your team beforehand. Because the collaborative editing experience of Markdown in git is non-existent - you can’t make suggestions like in GDocs.
If any of the above conditions are not met, you’ll end up with a jumbled mess of branches and merge requests.
How would your typical document review workflow look like? You might opt for using pull requests or merge requests, and specific people have to create approvals for a pull / merge request to pass. So far, so good.
But this really assumes so many things, and for each of this things, you have to be hyper-disciplined: Each merge request should, best case, only make changes to one document, not many. Merge requests should actually get merged soon as otherwise the branches go stale and you end up with a huge bunch of branches with unfinished edits (ugh!). Good luck findings those branches in audits. And finally, this also assumes that you know what sort of edits you want to make and don’t have any need for discussing them with your team beforehand. Because the collaborative editing experience of Markdown in git is non-existent - you can’t make suggestions like in GDocs.
If any of the above conditions are not met, you’ll end up with a jumbled mess of branches and merge requests.
Conclusion
So here’s the conclusion, it’s rather short: Don’t use GitHub / GitLab as your QMS.
Alternatives
I tried to refrain from mentioning alternatives here as I really wanted to focus on the points at hand. Still, if you’re looking for them, we've created a QMS software list - check it out! Or, if you just want to skip to the best choice (again, we might be biased), check out our software Formwork.
On a different note: Do you need any help with your EU MDR efforts?
We've worked with 100+ companies and helped them certify their devices in weeks, not months. Talk to us now – first calls are free! Check out our services and prices here.
Or, if you don't like talking to humans, check out our Wizard. It's a foolproof, step-by-step video course for getting your compliance done yourself.
And if you're looking for the best QMS software for lean, founder-led companies, check out Formwork. It automates your compliance, and there's even a free version for you to try out!
No image yet.
Setting Up a Quality Management System (QMS) in GitHub / GitLab
Dr. Oliver Eidel
updated 9 months ago
No image yet.
Open Source QMS Software Overview: GitLab As QMS?
Dr. Oliver Eidel
updated 9 months ago
No image yet.
All You Need to Know About QMS Software
Dr. Oliver Eidel
updated 9 months ago
Congratulations! You read this far.
Get notified when we post something new. Sign up for our free newsletter.
No spam, only regulatory rants. Unsubscribe anytime.
0 comments
No comments yet. Be the first one to share your thoughts!
Dr. Oliver Eidel
I’m a medical doctor, software engineer and regulatory dude. I’m also the founder of OpenRegulatory.
Through OpenRegulatory, I’ve helped 100+ companies with their medical device compliance. While it’s also my job that we stay profitable, I try to dedicate a lot of my time towards writing free content like our articles and templates. Maybe that will make consulting unnecessary some day? :)
If you’re still lost and have further questions, reach out any time!
Through OpenRegulatory, I’ve helped 100+ companies with their medical device compliance. While it’s also my job that we stay profitable, I try to dedicate a lot of my time towards writing free content like our articles and templates. Maybe that will make consulting unnecessary some day? :)
If you’re still lost and have further questions, reach out any time!