TL;DR

• GOV.UK Chat was tested by more than 10,000 people who asked over 26,000 questions across two pilot phases, making it the government’s largest public trial of generative AI
• Accuracy improved from 76% to 90% during the pilots, while all 508 jailbreak attempts were blocked by built-in safeguards
• The assistant will now roll out to all GOV.UK app users, with the full website to follow later in 2026

A major public test reaches its conclusion

The Government Digital Service has completed two pilot phases of GOV.UK Chat, an AI assistant that helps people find answers to questions about tax, benefits, visas, and other government services. The 18-month exercise stands as one of the largest user research programmes GDS has conducted and the most extensive public deployment of generative AI by the UK government to date.

Across both phases, 10,136 users tested a web-based version of the tool, asking 23,838 questions, while a second cohort of 641 users trialled a version embedded in the GOV.UK app, submitting 2,670 queries over four weeks.

What the pilots revealed

User feedback was broadly positive: 73% of app users described the assistant as useful and 64% said they were satisfied with the service. Several participants reported that the tool gave them clear, direct answers with relevant links and step-by-step instructions — sometimes removing the need to phone a government helpline altogether.

On accuracy, expert assessments and automated evaluation tools helped push scores from 76% at the outset to 90% by the end of the trials. The assistant draws only on published GOV.UK guidance, and responses are only counted as accurate when they fully match the original content standards.

Safety testing held up under pressure. All 508 recorded jailbreak attempts were blocked. The system now runs on Amazon Bedrock infrastructure with Anthropic’s Claude models, a setup GDS says can scale to meet future demand.

The team also improved handling of out-of-scope questions. Early on, some ambiguous queries went unanswered. Introducing clarifying follow-up prompts raised the answer rate for in-scope questions to 88%.

Speed remains the outstanding challenge

Response times averaged 10.7 seconds per question — a conscious trade-off between thoroughness and speed. Users tolerated the wait, but satisfaction increased noticeably in tests where faster replies were simulated. GDS is now exploring answer streaming, where partial responses appear incrementally, though this would require reworking some of the system’s safety checks.

What comes next

Sam Dub, lead product manager for GOV.UK AI at GDS, said the evidence was clear: “GOV.UK Chat simplifies interaction with the government, providing clear and helpful information to users, and saving them time.”

The next step is a release to all GOV.UK app users, followed by deployment on the main website later in 2026. GDS is also experimenting with agentic AI that could move beyond answering questions to helping users complete tasks such as form submissions.

For a public sector that has historically moved cautiously on AI adoption, the scale and transparency of these pilots marks a notable shift in approach.