Want a safe medical device? You'll need hands-on clinicians

Taking a clinical AI product from prototype to CE marked device requires the careful navigation of a complex set of steps. Clinicians add to this process an awareness and deep understanding of clinical pathways and patient outcomes at every stage. 

Whether an idea has come from R&D in industry or an ambitious new startup aiming to disrupt things, it will take time and effort before it can take on the illustrious label of CE marked medical device and start affecting patient outcomes. Whilst on this journey from prototype to medical device there are numerous challenges, pitfalls and traps waiting to trip you unless you have your head screwed on and the right team around you. 

And despite what the evangelists would have you believe, adding AI to the equation inevitably makes things murkier, not clearer. In fact, as the buzz surrounding AI succeeds in both driving exciting new frontiers in technological capabilities and obfuscating value with attention grabbing, hype-laden headlines, you need someone around who is going to ask the right question: How exactly is it going to benefit the patient, and what are the risks?

So it’s now more than ever that we need transparent accountability at the heart of building, testing and certifying these products. However, this clarity now involves ever more previously discrete skill sets working in concert. In this blog post I’ll be focusing on the important contribution that clinicians need to make in this effort, and arguing that they are critically important pieces of the puzzle to building safe medical devices. 

I’ll walk you through a number of the key mandatory deliverables for CE marking that clinicians are well placed to help with, and I’ll be showing how your project and your users can benefit from having early clinical and regulatory involvement.

If you’re serious about making people’s lives better, then regulation and clinical safety cannot be afterthoughts, they need to be a part of your culture and practice from day one. 

Use Specification (Intended Use Statement)

When it comes to the intended use, you’d be forgiven for thinking this an inanely bureaucratic step, but you’d be gravely wrong. Without a clear understanding and intention around exactly what your device is supposed to achieve (and in what population) you would struggle to make any claims (and measure them) around performance. When it comes to medical devices which use AI to improve patient care you’re going to need a crystal clear understanding of who your users are (N.B. your users may not be the patients), what disease process you’re intervening in, what data you need for your models and what situations your product can and can’t perform in. Having a clinician working hand in hand with your regulatory experts is not just nice to have, it’s essential. Every claim you make must be backed up by evidence. Bottom line - if you aren’t crystal clear on what you are building, then the regulators won’t be either.

Benefit-risk analysis

Medical devices aim to deliver benefits to their users whilst minimising risks. Every day clinicians help patients understand the potential risks and benefits surrounding treatment decisions in order to gain informed consent. They must communicate in terms that are both understandable and quantifiable to the patient. In the case of medical devices it is no different - benefits and risks must be defined and measured before a product comes into contact with a patient. Foreseeable misuse, the idea that users may operate the device outside of how it is intended, must also be estimated in its likelihood and potential for harm, and mitigated against through design changes where possible. If these questions sound difficult to answer, that may be because you haven’t seen the findings from a literature review, which should show how well your intended use is currently being delivered, or maybe you haven’t performed usability studies or human factors analyses.

Literature review

The clinical literature review aims to answer what is considered an acceptable level of risk and what benefits need to be achieved. This provides a cumbersome yet golden opportunity to understand the lie of the land that you are navigating, and gives clear goalposts to aim at in terms of performance metrics.

There’s no point in reinventing the wheel if your competitors (or existing treatments) already do the job better than your product, so you really need to know what you’re getting yourself into! You need people who are scientifically literate and able to critique papers, as well as understand what does and does not work about current best practice. Therefore, having a natural understanding of how treatments affect disease outcomes, as well as the healthcare workflows that they are managed in, clinicians are valuable assets here. 

Clinical evaluation plan and report (CEP & CER)

The CEP is the opportunity to explain (in advance) how you will assemble evidence to prove that your device performs as it should. Once that evidence has been collected the CER is the place where it all lives to tell the story that will convince people that your claims are valid. 

The story is told in 3 main parts:

i. Valid clinical association

Question: Is there evidence to suggest potential for your device’s use case, and what are the state-of-the-art benchmarks?

Method used to produce evidence: Clinical Literature Review

ii. Analytical validation

Question: Does your device behave correctly in test conditions?

Method used to produce evidence: Verification testing of software against technical performance requirements

iii. Clinical validation

Question: Does your device behave correctly in the clinical setting?

Method used to produce evidence: Clinical investigation, Post market clinical follow up

These 3 separate steps need to knit together perfectly within the rubric created by your use specification. In other words, you can now see how the intended use statement creates a foundation upon which everything else is built. If there is any misalignment or misunderstanding between these individual but complementary parts the whole thing falls down (known as a ‘non-conformity’ in regulatory speak). 

At this point you may find yourself struggling to differentiate the wood from the trees. It can be easy to get lost in the details of these moving parts, but a strong clinical grounding and focus around patient outcomes can act as your North Star here. 

Risk management plan and report

Product managers, engineers, data scientists and regulatory experts can help convert an idea into a reality. But when it comes to managing risk and uncertainty it helps to have someone at the table who knows what it’s like to take action when things are going south. 

The ground already covered within the benefit risk analysis provides a theoretical backbone to enable you to plan how your device’s risks are to be understood and monitored. Like many other parts of the technical file it helps to think of the risk management plan and report as living documents, which adapt to incorporate any new risks flagged up from the real world evidence generated by your device, thus having a knock on effect on the balancing of potential benefits and risks.

PMCF - planning studies, understanding and measuring clinical outcomes

Does what it says on the tin. Does it?

How do you know that your medical device behaves how you (and your customers) expect it to? Well, you’re going to have to observe it! This is where clinicians can help really drill down into the clinical workflows and ideate around which types of situations will produce the right type of use data. How plausible is it to extract exactly the right amount of data from every customer who uses your device? For your PMCF plan you’ll need to justify what performance data you’re collecting and why, and this has a bearing on what claims you can make, so without someone who deeply understands the clinical use case you will struggle. 

The only route to market is regulatory approval. The only route to regulatory approval is clinical evidence. You’ll need experts in both.

As we’ve seen, the number of core competencies needed to launch, land and look after a medical device are not small, and they only increase as the complexity and/or risk of the device grow. Often companies looking for regulatory support will find pure technical assistance only, and will have to look elsewhere for clinical expertise. In an industry where companies are lauded for speed, market share, scientific novelty and utilisation of buzzwords, there will always be a need for somebody in the room to bring things back to the patient. 

Hardian Health is clinical digital consultancy focussed on leveraging technology into healthcare markets through clinical strategy, scientific validation, regulation, health economics and intellectual property.