Question
We are collecting data to write the clinical evaluation report for our software (class IIa). Our software uses an ML algorithm to detect anatomical structures in CT scans with accuracy of a radiologist. We came across the term clinical evidence and we need to prove it in our CER. How do we do that? Do we need clinical data? Do we need to hire a medical doctor and can we write the report yourself?
Short Answer
If you need clinical data depends on your intended purpose (and the opinion of the auditor), the novelty of your device and yes, you can write of that by yourself.
Long Answer
Let’s start a bit boring and take a look into some definitions, the MDR and MDCG documents.
The clinical evidence is defined as “clinical data and clinical evaluation results pertaining to a device of a sufficient amount and quality to allow a qualified assessment of whether the device is safe and achieves the intended clinical benefits, when used as intended by the manufacturer” (MDR, Article 2).
Translated into normal people’s language: the clinical evidence is proof that your device works as intended and is safe to be used. Demonstrate this with (clinical) data from the real world to prove what you claim.
The clinical evaluation is the systematic and planned process to continuously generate, collect, analyze and assess the clinical data pertaining to a device in order to verify the safety and performance, including clinical benefits, of the device when used as intended by the manufacturer.
The result or output of this process is the clinical evaluation report that summarizes data, their assessment and the clinical evidence derived from therefrom.
And how do we demonstrate the clinical evidence?
Let’s take a look into the MDCG 2020-1 Clinical Evaluation for SaMD. The concept is actually quite simple. Three components should be included in the evaluation and derivation of clinical evidence:
- Valid clinical association: Is there a valid clinical association between your SaMD output and your SaMD’s targeted clinical condition?
- Technical Performance: Does your SaMD correctly process input data to generate accurate, reliable, and precise output data?
- Clinical Performance: Does use of your SaMD’s accurate, reliable, and precise output data achieve your intended purpose in your target population in the context of clinical care?
The guidance provides you with much more detailed information that we skip at this point.
When you evaluate the clinical evidence make sure to check the amount and quality of that data.
Typical questions include:
- Does the data support the intended use, indications, target groups, clinical claims and contraindications?
- Have the clinical risks and analytical performance/ clinical performance been investigated?
- Was the statistical approach appropriate to reach a valid conclusion?
Sounds very abstract. Can we have an example?
Sure and let’s take a look at your device. “Our software uses an ML algorithm to detect anatomical structures in CT scans with the accuracy of a radiologist”.
The valid clinical association is established by reviewing the literature. Demonstrate that:
- The normal shape and size of anatomy is well established
- Segmentation techniques on cross-sectional images correlates well with the actual size and shape
The technical performance is the summary of the verification and validation tests. Summarize the tests to confirm that the basic technical performance is achieved. This could include things such as:
- display, modification, window leveling of images, measurements including confirmation of accuracy, sensitivity and reliability
The clinical performance is a bit trickier.
Your device doesn’t produce clinical data per se, but it supports the decision making and diagnosis. The guidance allows you to rely on the usability data and tests. This means you can include the usability assessment, bench testing and preclinical data to demonstrate that your software performs as intended. You could also conduct a retrospective study and use CT scans that already underwent assessment by radiologists to compare them with the assessment results provided by your software. You could think of adding subchapters for the clinical evidence.
And finally you put these three things into the broader context of the clinical evaluation. That means that you create a chapter in the CER and describe the three above mentioned points. According to the MDCG 2020-1 you will subsequently do the risk-benefit assessment.