TL;DR

The Open Data Institute tested 11 large language models on over 22,000 questions about UK government services. The results: chatbots buried accurate information in excessive output, made unpredictable errors, and almost never refused to answer — even when they should have. Telling them to be more concise reduced their accuracy further.

What Happened

ODI researchers compared LLM responses to answers drawn from official GOV.UK material, judging output on verbosity, accuracy, and refusal rates. The models regularly went beyond authoritative government information, combining material from multiple sources in ways that introduced errors.

Some of those errors were serious. ChatGPT-OSS-20B told users they’d only be eligible for Guardian’s Allowance — a benefit for people caring for a child whose parents have died — if the child themselves had died. Llama 3.1 8B incorrectly advised that a court order was needed to add an ex-partner’s name to a child’s birth certificate, when re-registration is actually sufficient. Qwen3-32B wrongly said the £500 Sure Start Maternity Grant is available in Scotland.

The researchers flagged models’ near-total unwillingness to decline questions as “a dangerous trait” that could lead people to act on misinformation. Some models, including Anthropic’s Claude 4.5 Haiku, were notably more verbose than others.

Why It Matters

This research arrives as the UK government is actively deploying AI chatbots in citizen-facing services. The Government Digital Service plans to add a chatbot to its GOV.UK app in early 2026, Anthropic is building one for job seekers, and the Department for Work and Pensions is experimenting with one for Universal Credit claimants.

ODI director of research Professor Elena Simperl stressed: “If language models are to be used safely in citizen-facing services, we need to understand where the technology can be trusted and where it cannot.”

One practical finding: smaller, cheaper-to-run models delivered comparable results to large closed-source ones like ChatGPT 4.1, suggesting organisations should avoid locking themselves into long-term supplier contracts.

Looking Forward

The ODI released its CitizenQuery-UK dataset of 22,066 questions on Hugging Face for further research. The findings point to a need for keeping AI answers tightly focused on authoritative sources and being transparent with users about uncertainty — particularly as government chatbot deployments accelerate.