Can We Integrate Our eQMS With Jira, GitHub, Sharepoint, etc.?

You’re probably looking at eQMS software right now and are wondering which one of them integrates well with your developer tools – e.g. you’re managing your user stories in Jira (ugh) and your code on GitHub. Or even worse, something in Sharepoint. Okay.

Here’s my take: In 100% of cases I’ve seen so far, any sort of integration with dev tools and eQMS software did not make sense. Don’t believe me?

Here’s a question for you: Let’s just assume an eQMS offers to integrate with Jira. What will you use this integration for?

Usually, when I ask this question, there is.. silence. People simply don’t know why the hell they want to integrate these tools with each other. Which makes sense, because there’s no rational reason to do so!

But wait, hold on.. some people do have some ideas. So let’s go through them in order and see whether they make sense.

Jira / GitHub And eQMS Integration For Test Management

The most often mentioned idea is integrating with GitHub for test management. The setup goes as follows: You have some sort of continuous integration (CI) pipeline which e.g. runs every time you push a new commit to a pull request. The CI pipeline runs your test suite which contains unit and integration tests. Cool.

Now you think like “hey cool, can’t we re-use these test results for our compliance documentation?”. Great idea. The answer is no. Here’s why.

The regulatory requirement as per the IEC 62304 for medical device software is that each test has pre-defined steps and an expected result. It also needs to be linked to the software requirements it’s testing.

So if you have these software requirements:

You could have these tests:

Okay, so as you see in the able above, for each of your software tests, you need to define:

• Which software requirement IDs if covers
• Which steps are required to perform it.
• The actual results of those steps.
• And whether the result is a pass (yes/no).

In addition to that, you’d need to record when the test was executed and the configuration it was executed in, e.g. on which version of your software, in which OS / Docker container etc.

So.

Now, tell me again – how will a GitHub integration solve your problem?

Here’s an incomplete list as to why it won’t work:

• Regulatory tests tend to be more “high-level” than actual tests in your code (e.g. unit tests). So using your code tests as your regulatory tests will end up being a huge documentation effort, see next points.
• For each of your code tests, you’re still missing information which is required for regulatory compliance: Which software requirements it covers (IDs!), the actual test steps (human-readable), the result of each of those steps (human-readable), when it was performed, and what constitutes a “pass”.
• Mapping your software tests to software requirements is a tricky problem. What do you do if your software requirements change? Do you manually go through your tests and re-map them to new software requirement IDs?
• How will you store this “meta”-data in your tests? At the very least, you’ll have to start attaching meta-information to each of your tests, a bit like weak types or so. Depending on your language, this will either be a huge or a gigantic undertaking, because you’re messing with your test framework and trying to extend it with functionality it was not meant to have.
• How will you output this sort of data from your test runner? If you’re e.g. using RSpec, a Ruby testing library, it’s not entirely trivial to get a structured output (e.g. JSON) of all of this data to re-use for your regulatory documentation.

Etc., etc. But maybe my main important points are these:

Firstly, the IEC 62304 does not require any of this (please read it and prove me wrong). So why are you doing it?

Secondly, if you document your test cases separately in your eQMS software, i.e. not integrating with your dev tools, this will translate to a huge efficiency gain because you’ll save lots of time. In simple terms, it’s much faster to document 20 high-level human tests than developing an automation pipeline which covers your 1.000 automated tests, not even speaking about all the additional meta-information you have to attach to those 1.000 tests.

So, you choose: Do you want to be done in an afternoon (20 tests) or do you want to make this a multi-month PhD project of developing automated test pipelines for regulatory compliance?

Jira And eQMS Integration For Customer Support

The next “idea” people have is integrating Jira (or any other terrible tool) and their eQMS software for customer support. The thinking goes like this: New customer support tickets are created in Jira. Then, for compliance purposes, with the magic Jira <> eQMS integration, they are copied to the eQMS as a record.

Stop right there.

This does not make sense. There’s absolutely no regulatory or even rational requirements to copy records between tools. The ISO 13485 gives you the freedom to store you records in whatever tool you like, and those can be multiple tools, too!

So if each customer support request is stored in Zendesk, just leave it there! Show it to your auditor when they come and visit your, and they’ll be fine with it. No need to integrate with your eQMS.

Or if your supplier evaluation forms are in Jira (ugh), just leave them there, too! No need to integrate with your eQMS.

Etc.

For anything which can be considered a record, there’s no need to copy between tools. So, here too an integration makes zero sense at all.

Conclusion: eQMS Integrations

I’m out of examples, because I can’t come up with further ones which would even remotely make any sense. I’ve consulted 150+ companies and I’ve never seen an eQMS integration which provided value or increased efficiency.

This, by the way, is why our eQMS software, Formwork, doesn’t have any integrations. We looked very hard for useful ideas here and asked many customers, but we always came out empty-handed.

Other eQMS providers are happy to tout all their integration possibilities, but for me that’s just useless sales talk. What is your use case for integrating your eQMS with other tools? And how do you expect it to save you time?

I think there is none, but I’m happy to be proven wrong. If you have an idea, feel free to comment below!