How to handle risk analysis for regression vs classification models in AI medical devices?

Question

When performing risk management for AI-based medical devices, how should the analysis differ between classification and regression models?
For classification models, risks are often broken down into omission/commission risks due to false negatives (FN) and false positives (FP). However, with regression models, this distinction is less straightforward since there are no FN/FP, but rather continuous errors measured by metrics like RMSE.
Is it appropriate to categorize errors in regression models as "deviating detections", or is there a better way to structure the risk analysis for regression models?

Dr. Oliver Eidel · Accepted Answer

This is a common challenge. With classification models, you have clear categories and can define risks based on false positives and false negatives. For regression models, since outputs are continuous, you can set thresholds to define what counts as a significant error (for example, above a certain RMSE or another clinically relevant measure). By defining these thresholds, you effectively convert the regression outcomes into categories (such as safe/unsafe), making the risk analysis more straightforward.
The key is to clearly document the justification for your chosen thresholds, using sources like PMS data or literature. You should also have a review process in place to ensure these thresholds remain appropriate over time. This approach should satisfy auditors if the logic is transparent and justified.

Answer

It depends on the intended use of the model and whether the difference between classification and regression leads to different risks or risks with different severity levels. If so, it's reasonable to split them up in your risk analysis.

Answer

I would focus less on the terminology and more on ensuring your rationale is logical and well-documented. What's important is that your team agrees with the approach and that you can explain it to auditors. As long as you have a documented justification, the wording (like "deviating detection") is secondary.

Answer

In our process, we also split up risk per model. Not sure if that's what you mean, but it can help clarify the analysis.

Answer

It depends on the intended use of the model and whether the difference between classification and regression leads to different risks or risks with different severity levels. If so, it's reasonable to split them up in your risk analysis.

Answer

I would focus less on the terminology and more on ensuring your rationale is logical and well-documented. What's important is that your team agrees with the approach and that you can explain it to auditors. As long as you have a documented justification, the wording (like "deviating detection") is secondary.

Answer

In our process, we also split up risk per model. Not sure if that's what you mean, but it can help clarify the analysis.

How to handle risk analysis for regression vs classification models in AI medical devices?

8 Answers

Keep reading

Risk Management Plan

ISO 14971 Walkthrough

SOP Information Security Risk Assessment