Health Foundation chief urges stronger UK AI healthcare evaluation
TL;DR
- Dame Jennifer Dixon, chief executive of the Health Foundation and member of the National Commission for the Regulation of AI in Healthcare, used an MHRA blog to argue UK AI evaluation needs strengthening
- Her core concern: local NHS service evaluations of AI applications are often weak, creating incentives to overclaim results, while post-market surveillance remains underdeveloped
- The intervention comes as the NCRAIH shapes the UK regulatory framework for adaptive AI models in clinical settings
Dame Jennifer Dixon, Health Foundation CEO, has used a Medicines and Healthcare products Regulatory Agency blog to press for a harder edge on how the NHS evaluates AI tools after deployment. Her argument: AI applications in healthcare are not static products but socio-technical activities that require ongoing scrutiny on safety, accuracy, equity, usability and workforce impact — and current UK evaluation infrastructure is not fit for that task.
The structural gap Dixon identifies
Dixon sits on the National Commission for the Regulation of AI in Healthcare, giving the piece weight beyond typical commentary. She distinguishes between formal summative research in the NHS — generally conducted well, if “not always rapidly or cheaply” via NIHR — and local formative service evaluations, which she describes as “of very mixed quality.” That quality gap creates a problem: local implementation evaluations with weak methodology give operators room to overclaim AI benefits, undermining trust and safety.
The MHRA focus is narrower: is the AI safe and accurate, does it remain so as it adapts, and how does it compare to alternatives. Dixon argues this matters, but the wider questions — did it shift care between settings, save time, deliver return on investment, anticipate workforce disruption — fall outside MHRA remit and require better research infrastructure.
Why this matters beyond healthcare
Dixon’s framing has implications beyond the NHS. Her “socio-technical, not bounded product” characterisation of AI aligns with how financial regulators including the ECB and Bank of England are starting to describe AI systems in banking — the same Mythos-era concerns surfaced by Governor Andrew Bailey this week. Dixon is making the same argument inside healthcare: AI is not a product to approve once but a system to monitor continuously, and UK institutions need the evaluation capacity to do that.
Her case for a national system as an AI adoption advantage — coordinated procurement, regulation, data infrastructure and clinical standards — cuts against the narrative that NHS scale is a barrier. In her framing, the NHS’s integrated structure becomes a comparative strength for AI governance, if the evaluation gap is closed.
Looking forward
The NCRAIH’s direction will shape which AI vendors can commercially operate in the NHS over the next 18 months. Expect procurement frameworks to tighten around post-market surveillance requirements, and for the weak-local-evaluation problem to surface in AI for Science Strategy implementation. UK AI healthcare vendors should read this piece as a preview of what the commission will require, not an academic commentary.