When it came to regulating medical devices, in the past, we had this old thing which was called the MDD. Under the MDD, a lot of software as a medical device was classified as class I, which meant that manufacturers just had to maintain compliant documentation, without being required to go through an audit before entering the market.

Yes, that’s correct: Class I manufacturers would simply draft their regulatory documentation, sign it and declare it “done”, then just stash it in their drawer and bring their product to market. If you think that’s borderline crazy, you’re probably not alone. That being said, theoretically, such a company could get “surprise-audited”, but in reality it didn’t always happen.

With the MDD, a lot of medical software was class I. Accordingly, many software manufacturers followed this “write some documentation and stash it in your drawer” approach. If you think this lead to a lot of incomplete documentation, you’re probably not alone, again - it’s similar to those “optional assignments” given out by highschool teachers, only to realize the next day that students are not really interested in optional assignments.

That lead to some interesting scenarios. I remember even some Radiology AI software being classified as MDD class I.

Were patients harmed? Well, yes, patients are harmed in the hospital every day due to a variety of reasons: Crappy software, tired physicians, the fax machine being broken, and so on. There probably also were some instances where class I devices harmed patients.

At this point, regulators shake their fists and are like “we need to regulate this even more! With a higher classification, they would have been audited, and audits are great at preventing unsafe devices from entering the market!”. We actually have not data to support this claim.

But regulators are not deterred by missing data. Also, the toolbox of regulators only contains one tool, and that’s more regulation. So, in the typical fashion of “if you have a hammer, every problem looks like a nail” they went ahead and applied more regulation to the problem. The MDR was born, and with it many MDD class I devices were now up-classified to class IIa and higher, requiring a prior audit by a notified body.

The World of Regulators

In my perception, many regulators live in a weird bubble - in their worldview, complying with more regulations is totally easy for companies because they either think companies have unlimited regulatory resources and money, or because the regulations are so intuitive that it’s simple to comply with them. Both of these assumptions might be true in a parallel universe, but in the real world, they are not.

This is also mirrored by the various statements we see by the EU on notified body submissions. The conclusion regularly is something like “look! Notified Body capacity is getting better, but there are still thousands of remaining manufacturers who haven’t submitted their documentation yet. They should get their act together!”.

Little do they know that, well, not all of those thousands of manufacturers even plan to submit their documentation. In the real world, it’s not a question of when, but if at all.

So let’s talk about the real world here.

This post is about two real-world case studies of MDD class I devices which were taken off the market due to the MDR.

Case Study 1: A Small Family Business

The first company we’re looking at is a small, family-owned business. It employs around five people, pays good salaries and is profitable - yes, profitable, but not wildly “the CEO is driving around in a Ferrari” profitable.

One of their main products is a medical software which is on the market as MDD class I device. Under the MDR, it would be up-classified to at least class IIa.

They decided to take most parts of it off the market, because going through class IIa certification would have lead to costs which would have made the entire business unprofitable.

Now you might say “well yeah, if this one expense makes them unprofitable, weren’t they a crappy business in the first place?”. Intuitively that might sound right, but realistically, the Notified Body expenses for small businesses are pretty huge. Let’s look at some numbers!

The average salary per person was around 50k€ / year. The CEO did most of the regulatory work for the MDD class I device in the past, and the documentation is surprisingly solid, given that it’s a “write it and put it in drawer” kind of documentation. The company has been profitable for years and makes anywhere between 30k€ and 80k€ in profits per year.

However, the huge problem now is the cost of an audit, both in time and in money. A Notified Body certification can easily cost between 30k€ and 80k€ (yes, prices vary this wildly). More importantly, it would increase the regulatory workload due to the handling of repeated audits, so that the CEO would likely have to hire another employee to handle the regulatory work. We’re looking at increased costs of at least 50k€ (audit) + 30k€ (part-time salary) = 80k€ per year, and those are optimistic estimates. The company would become unprofitable.

So the decision was made to take the medical device off the market. Risking the medical device certification would have risked profitability and therefore the jobs of 5 people.

An interesting solution was found with the competent authority: During the most recent audit, there was a discussion whether parts of the software might actually not be considered a medical device. So the manufacturer went ahead and wrote an intended use for each of it’s sub-features of the software (yes, seriously) and only removed the features which had an intended use which resembled a medical device.

This was a surprisingly pragmatic approach, and I actually appreciate it a lot that the competent authority auditors agreed to it.

The device which remains on the market now continues to generate revenue, albeit with a reduced set of features. Still, the company continues to be doing well.

An interesting side note: Looking at customer complaints, none of the features, including the medical device features, ever lead to any real harms. All in all, this could truly be considered a low-risk device. So removing the medical device features didn’t improve the real-world risk profile at all, but it did negatively impact the usefulness of the entire app. As a user, I would have been quite angry - like, now my software has way less features, due to some weird regulation? Well.. ironically, nobody involved patients when drafting the MDR, so there’s little surprise that they don’t reap any benefits from it.

Case Study 2: An Academic Side Project

Let’s look at the next case study. This one was an academic side project. In short, the situation was that a physician-scientist did some interesting research in predicting a certain disease with certain patient-report parameters, and wanted to bring this to market.

Okay, that was way too abstract, so let me just give you an example for illustrative purposes: Let’s say the researcher found a way to predict explosive diarrhea the next day based on how many hours you slept the night prior. This could be really useful to users, because who wants to be innocently riding on the subway, only to be surprised by explosive diarrhea? Entering your sleep data every day to prevent this occurrence sounds really useful.

The researcher was happy in their current job and wanted to move on to researching other diseases, but thought “hey, this would be cool if patients could actually use this”. That also set them apart from all the other 99% of researchers who just publish stuff in journals which never ends up in patient’s hands. So that was cool!

They then went ahead to partner up with a few people to found a company and draft the MDD class I documentation. It was quite an effort, but doable, and finally, the device was available on the App Stores. Nice!

(By the way, I would have named it “Subway Saver: Tunnel Tummy Time Tracker” - yes, I’m in Thailand again, yes, I ride the subway every day and yes, if you want to build this and use this name, go for it.)

So the app chugged along on the App Stores and helped a small group of people - you know, not everyone is interested in preventing explosive diarrhea. Many people don’t often suffer from this, and some courageous individuals prefer to roll the dice every day, because entering sleep data might seem to cumbersome.

But then the MDR came along. And in this particular case, “explosive diarrhea” was considered a diagnosis, and the app was providing diagnostic information as per MDR rule 11, so it was now a class IIa device.

Mind you, the app was not making any money - it was an academic side project, and people would work on it in their spare time, doing software development, writing regulatory documentation, even doing customer support.

But requiring a Notified Body audit changes these dynamics entirely: It’s one thing to spend some hours every week on a side project, but it’s an entirely different thing to pay up to 80k€ for a certification and spend even more time on regulatory documentation.

So they decided to take it off the market.

Now, regulators might say: “This is a good thing! People shouldn’t be able to bring a medical device to market in their spare time!”. But the question indeed is “who was harmed?” - the app provided a lot of value to a smaller group of patients, and when we looked at the customer complaints, zero users were harmed. So this is another case of taking something off the market which was useful for people and which was causing no harm.

Good? Bad?

Is this good? Bad? I’ll let you decide. Some quick thoughts though:

The only thing we can conclude for sure is that the barrier of entry has gone up dramatically, because the time and costs for bringing medical software to market have increased by a lot.

There might be an interesting comparison to the pharmaceutical industry here: Bringing pharmaceuticals to market has become really expensive, so it’s only done by the huge companies which can shoulder the costs. This has lead to a lot of safety - nowadays, you can pretty much walk into any pharmacy, buy medication and assume that it’s really safe, and that whatever side effects exist, they are already known.

This, however, has come at the cost of diversity and innovation. Only medication which can make a lot of money will be brought to market, because the companies have to recoup their huge regulatory and development costs. So, for many rare diseases, medication isn’t available, even though it technically might exist.

I think this is a trend we’ll also see in medical devices: Looking at our two case studies above, it’ll be increasingly hard for small companies and academics to bring their products to market.

So the barrier of entry has gone up, and that only large (or rich) companies can bring products to market.

Will that improve safety? I truly don’t know. It’s a complex system. We can’t even predict the weather more than a few days into the future, so I’m not sure we can predict this.

We don’t know if audits have any impact on safety. But they do bias manufacturers towards being larger companies, and larger companies are way more risk-averse, so that might indirectly increase safety. On the flip side, when many “niche” devices start becoming unavailable, many patients might go untreated.

Though the funny thing is that, from the regulators point of view, this is perfect safety, because an untreated patient can impossibly be harmed by a medical device.

So maybe the real conclusion here is: We’d need to look at patients who go untreated due to increased regulation. But.. that would mean involving patients in the first place, something which hasn’t happened so far. And then the most likely conclusion would be to reduce regulation, and that’s similarly something which hasn’t happened.