When certifying software as a medical device (or any medical device, really), you need a Clinical Evaluation. If I had to sum it up in one sentence, the Clinical Evaluation basically say: “Our product is safe and has these benefits” or “Our product is very similar to other products which have already been certified, and because those products had to prove that they’re safe, ours is safe, too”.
So that’s what it’s about – showing safety and benefits. Consultants specialized in this field may now shout at me for over-simplifying, but this is how it’s best explained to people who are new to this field.
And this is also good timing for my disclaimer: I’m not very experienced with clinical evaluations. I was somewhat involved in writing a few by providing content and reviewing other people’s work, but I don’t consider myself experienced when writing one from scratch or assessing what sort of content a specific company may need to cover in their clinical evaluation report.
That being said, I’d still like to give you a rough overview about what sort of work lies ahead of you so you can your software certified as a medical device, which is your goal (right?). Based on my experience getting software by startups certified, this information already helped them a lot.
Step 1: Determine What Sort of Clinical Evaluation You Need
Roughly speaking, there are two options:
- Compare your product to other products which are already certified and argue that it’s sufficiently similar (strongly preferred).
- Compare your product to others, determine that there’s nothing like it on the market and do a clinical study to gather data on its benefits and safety (don’t do this).
I have noted my personal preference behind each option. Option 1) will require you to do a literature research and write up a detailed comparison (about as much as a Bachelor’s thesis). Option 2) requires much more – writing study documentation, recruiting patients, analyzing data and lots of other stuff, so just don’t do it. It much more time-consuming and expensive and startups often can’t afford it.
If you have any chance of comparing your product to something which is already certified, I suggest you take it.
Step 2: Research Similar Products
Assuming you chose the first approach (comparing your product to already-certified ones), you need to do some research on which products exist and whether this approach is feasible for you.
One of the best sources for this purpose are the FDA databases, e.g. the premarket notifications. For example, look at this document about HealthPNX, a machine learning – based product by the Israel-based Zebra Medical.
These sort of documents typically include the intended use of the product. That’s perfect for your research because you don’t have to guess what the product does based on incomplete marketing material or the manufacturer’s website. It also makes it easier to subsequently compare those products to your product.
Unfortunately, the FDA databases (obviously) only include documents of devices which were seeking FDA approval. For EU-approved medical devices, it becomes more tricky. For some reason, those sort of documents are not publicly available (why? I don’t know).
For EU-approved (i.e., CE-labelled) devices, you might have to resort to company’s websites and marketing material to discern the product features. If you’re lucky, the manufacturer has published a study based on their device.
When searching for studies, Google Scholar is pretty solid. Scientific publications could also help you back up your claims of the benefits of your device. For example, if you say that your machine learning – based radiology software makes radiologists less prone to errors, maybe it would make sense to cite some papers which demonstrated exactly that in a study. Obviously, their software was different than yours, but it still helps you rationally explain that your product may have certain benefits
Step 3: Compare Equivalent Products to Yours and Write Clinical Evaluation Report
Finally, you summarize your findings by writing your clinical evaluation report. On a very high level, it should contain:
- Description of the medical device (mostly by copy-pasting the intended use, maybe also software requirements or design input)
- Associated risks (mostly by copy-pasting the risk analysis)
- Expected medical benefits of the device
- Current state of the art and equivalent devices (based on your research and competitors so far)
- Comparison with equivalent devices
How do you compare your device to equivalent devices? Basically, you select some criteria, make a table and and fill it out. It could look like this, taken from the Zebra HealthPNX example linked above and simplified:
Technological Characteristics | HealthPNX (Your Product) | cmTriage (Other Product) | Similar? |
---|---|---|---|
User | Radiologist | Radiologist | Yes |
Image Format | DICOM | DICOM | Yes |
Modality | X-Ray | Mammograms | No, but both are radiological medical images |
You get the idea.
There’s a whole guidance document (MEDDEV 2.7/1) about what a clinical evaluation report must contain. You should definitely read it if you plan on writing your own clinical evaluation.
There’s still lots of other important stuff which I haven’t mentioned. But this is unfortunately where most of my knowledge ends. I was always lucky enough to have someone else on the team who write it, or it was outsourced entirely.
While I hope that I could still help you gain a rough understandic of what clinical evaluation is about, I would probably suggest to outsource the writing of your clinical evaluation. It’s also more trustworthy towards your notified body as an external company probably has a more objective view on your product?
On the other hand, you’re paying them to write your clinical evaluation which sounds like a conflict of interest. But wait, you’re also paying your notified body to audit you. Hm. Let’s move on to the next topic.
External Clinical Evaluation: Cost and Procedure
What are the next steps and costs for getting your clinical evaluation done by a third party? First, you’ll search for a company and get a quote. In my experience, those are in the low five figures of Euros (yes, seriously).
After you’ve acquired the required money and decided on a company, you have to send them documentation about the product which they should evaluate. This is a bit of a chicken-egg situation: Technically speaking, you need the clinical evaluation to finish your documentation. But you already need some documentation to get started on the clinical evaluation. Ugh.
At the minimum, you’ll send drafts of your intended use and risk analysis. You’ll also send a list of similar products (see above) and scientific publications which may help. Maybe also design input, software requirements and software tests. And whatever else the company requests.
Then, the company gets to work, does research and writes your report. While they should remain independent, you often sitll have an opportunity to review drafts of their clinical evaluation report and correct them. They may have misunderstood your product or some associated risks. Maybe you’ve also found more papers in the meantime. This process is generally painful as it’s a constant struggle of “they misunderstood stuff and should change the report vs. they should remain independent”.
So, as you might have noticed now, the headline stating “3 easy steps” was somewhat ironic. It’s not really easy, and it resembles a significant part of the total regulatory work you need to do to get your product certified. And that still assumes that you can compare your device to equivalent devices and don’t have to do a clinical study. If that happens, well, then things get even more.. fun.