Early economic modelling of AI as medical device - answering the ‘why’
In recent months The National Institute of Health Care and Excellence (NICE) have released several key guidances on the use of AI in the UK. The recommendations made in these highlight the current position that many companies with AI technologies as a medical device (AIaMD) find themselves in. That is, a need to demonstrate why health systems need them.
Looking at the three most recent policies published by NICE:
AI CAD software for detecting measuring lung nodules in CT scan images, published in July 2023 concluded that there was insufficient evidence to recommend AI for adoption in routine practice and for the management of patients symptomatic of lung cancer. However, for lung cancer screening the committee concluded there is potential for cost effectiveness but additional clinical and economic data is needed to demonstrate this. Specifically, more evidence is needed on the use of AI-derived CAD when aiding the clinical review of CT scans, how this impacts decision making and turnaround times.
AI technologies to aid contouring for radiotherapy treatment planning published in September 2023 and is the first AI intervention to receive a positive recommendation. The recommendation for use has been given for 3 years whilst further data is collected to fill evidence gaps. Evidence is still needed on clinical outcomes such as radiation dose and adverse events as well as data on resource use. By resource use, we mean information to assess direct medical costs of a health service i.e. all costs incurred by the healthcare system only.
AI derived software to analyse CXRs for suspected lung cancer in primary care referrals, also published in September 2023. The use of AI support CXR review on the lung cancer pathway was not recommended for use, again due to insufficient evidence on the potential benefit. Further information on the technologies performance in clinical practice is needed and the implications for patient outcomes in cases where the AI calls a false positive or negative abnormality.
Interestingly, across all three technologies, regardless of whether the technology is recommended for use or not, the additional data needed broadly fit into two categories: clinical efficacy and economic impact. Both of these aim to assess the why of AI as a medical device. Why will technology benefit patients? Why will a product save money? Why should a payer buy the technology? If they will, why should they pay a specific price? Why will value be provided?
Bang for buck
Through a simple cost consequence analysis, AI to support contouring for radiotherapy treatment planning was demonstrated to be at least cost neutral, however NICE requires further information to assess cost effectiveness. This means that not only is it important to demonstrate why a technology saves money but also why it provides value for money i.e. bang for buck. It is particularly necessary for technologies that have a risk of costing the NHS more money, the additional spending will need to be justified by additional patient and clinical benefit. This rationale underpins standard 18 of NICE’s Evidence Standards Framework which dictates that cost effectiveness evidence should be generated to support technologies that are deemed to be high risk, like most AI devices.
So, how do we go about addressing these whys?
Step 1 - Have a clear and robustly developed value proposition as part of your go to market strategy.
In order to get an idea of where maximum value could be derived, a comprehensive assessment of the healthcare system is needed to determine who the technology would likely impact, what these stakeholders currently have access to, how the technology would change this and what could be the implications for these changes. These questions must be answered through engagement with clinicians, patients, anyone that will come into contact with the technology as well as extracting relevant information from published evidence and policy. This work should be carried out during product development, if not before.
At Hardian, we have conducted multiple stakeholder workshops and interviews for clients, really getting down into the details of all of these questions, helping provide fundamental insight into a product’s true value proposition.
Step 2 - Materialise your value proposition with an economic evaluation, starting with early economic modelling
A value proposition will give a theoretical estimate of the size of a technology's value but an economic evaluation allows a company to put their money where their mouth is.
Economic evaluations such as cost effectiveness rely on the collection of primary data to support value claims of an intervention - but this can present a barrier for digital health technologies. The resource and investment levels required to execute these studies and bring products to market can be substantial and, we find, often underestimated. However, without data it is still possible and important to carry out some degree of economic assessment - early economic modelling (EEM) has been recommended for over 20 years and involves using health economic methods to estimate an intervention's comparative value to alternatives. In EEM model parameters are informed by immature data (Love-Koh, 2020). Using EEM, Hardian helps clients test a set of value outcomes to identify the optimal economic value proposition. So, if you’re not sure what data to collect in order to demonstrate cost effectiveness, think about doing early modelling with us first to better inform your processes.
Not only will early economic modelling provide an initial estimate of a technology’s business case but, through sensitivity and scenario testing, will give insight into the drivers of value. EEM can therefore be used to inform primary data collection, feeding into study design, and consequently reducing the investment risk of clinical studies, catalysing the R&D process. Most of all, it will help digital health developers to start answering why healthcare needs their solution.
Hardian Health is a clinical digital consultancy focused on leveraging technology into healthcare markets through clinical strategy, scientific validation, regulation, health economics and intellectual property.
