If I could name one topic which is the most unclear and messy in medical software certifications, it might be this. Evaluating whether a change to your (certified) software as a medical device is a significant one is so subjective, it’s almost ridiculous. It’s highly dependent on your notified body and on your auditor. The worst part is that your auditor can’t provide any consulting services to you (which makes sense), but that leaves you in a bad place because nobody else can tell you whether your planned change is significant.
But before I continue my rant, let’s look at MDCG 2020-3 which is a guidance document on significant changes. It at least gives us some indications on what we could expect. I’ll spare you reading yet another guidance document with poor formatting and titled “DRAFT”. Maybe we could invest some EU money in training people in using Microsoft Word?
Here’s what’s a significant change according to the document. I fixed the 50 typos and added question marks behind unclear phrases.
Changes of the Intended Use
Not significant:
- Limitation of the Intended Use
Significant:
- Extension of the Intended Use
- New user or patient population
- Change of clinical use (anatomical site, access site or deployment methods)
Changes of the Design or Performance Specification
Significant:
- Change which requires further clinical or usability data to support safety and performance
- Do new risks require risk control measures or are existing risks negatively impacted?
- Change of built-in control mechanism, operating principles, source of energy or alarms
Software Changes
Not significant:
- Minor change: Bug fixes (corrections of errors which don’t pose a safety risk, security updates, appearances of the user interface (?), operating efficiencies (?), changes to enhance the user interface without changes in performance (?))
- New languages, layouts
Significant:
- New or major change of operating system or any component
- New or modified architecture or database structure, change of an algorithm
- Required user input replaced by closed loop algorithm
- New diagnostic or therapeutic feature, or new channel of interoperability
- New user interface or presentation of data (medical data presented in a new format or dimensions (?))
Summary of the MDCG 2020-3 Guidance
Most of those points make sense. A change of the intended use always constitutes a significant change, unless you limit it.
The design and performance spec changes make sense, except one: Why should a new risk which requires risk control measures constitute a significant change? This shouldn’t be based on whether a risk control measure was needed. Instead, it should take into account whether the total risk profile of the device became worse.
Some of the software points also make sense. Bug and security fixes aren’t significant changes.
Other points are less clear and sound quite limiting:
- What’s a “major change to a component”?
- Changing the “operating system” sounds like the authors never heard of containerization (e.g. Docker). What would a change to the OS be there? A change in docker version? Or in the host OS? A change in the container? I’d prefer to ignore this. Most modern programming languages run in some sort of VM or interpreted environment which is mostly independent of the host OS anyway (JVM, CPython, V8).
- Change in database structure is significant? So a migration would be a significant change? That would be a real limitation.
- New channel of interoperability is significant? Just having another interface doesn’t really change things a lot in my opinion.
If you haven’t noticed by now: It’s a mess. Ready for more confusion? Based on the experiences I had with auditors in the real world, changes to the software (even major ones) were generally fine as long as the intended use remained the same and the risk analysis didn’t become worse. To me, this sounds like a good and pragmatic approach. It however contradicts the guidance document. And I could imagine there are auditors out there who handle this differently.. welcome to regulatory hell.
In case you missed the link above: I also answered a slightly different question regarding changes to machine learning – based software as a medical device.
