Stanford-led study finds AI hiring tools produce systemic racial disparities

TL;DR:

• A Stanford Institute for Human-Centered AI study of 4 million job applications submitted through Pymetrics between 2018 and 2022 found “clear racial disparities” in outcomes, with one in 10 roles showing adverse impact against Black applicants and one in 20 against Asian applicants.
• Jobseekers needed to apply for at least 25 different positions through the platform to be almost certain of one progression recommendation, with 4% of applicants rejected by the same algorithm across 10 separate applications.
• The 156 employers in the dataset — most with annual revenues above $5 billion — were in some cases sharing identical algorithmic models, meaning a candidate rejected once was likely to fail at other employers using the same vendor.

This is the largest published audit of AI hiring algorithms to date and lands in a UK market where game-based assessments from vendors such as Pymetrics, HireVue and Arctic Shores are increasingly used by major employers to triage volumes that have grown sharply since the rise of one-click application platforms.

Context and Background

The study’s most distinctive finding is not bias against individual demographic groups — that is well-documented in earlier audit work, including the University of California / University of Chicago “distinctively Black names” study showing a 2.1 percentage-point drop in employer contact. It is the cross-employer concentration effect: the researchers identified 42 algorithmic models that were “shared across” different employers using the Pymetrics platform.

For UK candidates, that pattern matters because adverse-impact regulation under the Equality Act 2010 operates employer by employer, not vendor by vendor. A candidate filtered out by a shared scoring model across multiple employers has, in effect, been rejected by a single algorithm masquerading as a competitive market. The Information Commissioner’s Office issued guidance on AI in recruitment in 2024 but stopped short of requiring vendor-level transparency or shared-model disclosure. Pymetrics’ owner Harver did not respond to the FT’s request for comment.

Co-author Kathleen Creel of Northeastern University captured the systemic angle directly: “As a single vendor comes to dominate decision-making in a space, their quirks or shortfalls can be present across that entire sector in a way that wasn’t possible before.” Comparable concentration is visible in the UK: a small number of vendors handle screening for most FTSE 100 graduate schemes.

Looking Forward

UK employment lawyers are likely to use the Stanford findings to pressure HR teams to audit the underlying vendor model — not just the employer’s own data — when defending against discrimination claims. The EU AI Act’s high-risk classification of recruitment algorithms takes binding effect for new hiring deployments later this year, and UK businesses serving European candidates will need to assemble the audit trails this paper effectively maps out. Expect renewed pressure on the ICO and the EHRC to clarify whether shared-model effects should be treated as a single discriminatory system rather than a series of independent employer choices.