TL;DR

The Department for Education is building a language-model service to inspect apprenticeship vacancies before they go live, flagging possible discrimination and deciding which adverts a person still needs to look at. The department expects it to cut the cost of manual review by 40%. It puts an AI system in the position of judging employers’ compliance with equality law.

What the system would do

The proposed service — named in departmental documents as the “recruit an apprentice AI vacancy quality assurance API” — would run each advert against the Equality Act 2010 for potential breaches. Discrimination screening sits alongside more mundane checks: spelling and grammar, gaps in required information, training that does not match the apprenticeship it is attached to, geographic inconsistencies, and adverts posted more than once.

Only some of that is novel. Duplicate detection and spellchecking are ordinary text processing. Assessing whether wording in a job advert amounts to unlawful discrimination is a different order of task, because it is a legal judgement with case law behind it, not a pattern match.

The question underneath

Employers rarely write overtly discriminatory adverts now. What survives is subtler — requirements that screen out disabled applicants without cause, age-coded language about energy or digital nativeness, availability demands that fall unevenly on carers. Whether a language model catches that reliably, and whether its misses are evenly distributed across employers, are empirical questions the department has not yet published answers to.

The framing also matters. Because the tool decides what escapes human review rather than deciding outcomes on its own, a false negative is not merely a wrong flag: it is an advert published without anyone reading it. That makes the miss rate, not the accuracy rate, the number worth asking about.

The pattern this week

This lands the same week ministers put 70 young people through AI boot camps aimed at apprenticeships in the north west. The state is now using AI both to prepare candidates for apprenticeships and to police the adverts they apply to — the same technology on both sides of the transaction, with the department as its own auditor.

Looking forward

The disclosure to watch is evaluation: what the department measures, whether it publishes error rates by protected characteristic, and who reviews the reviewer once 40% fewer adverts reach human eyes.