EY retracts loyalty study after researchers find AI hallucinations

TL;DR:

• EY Canada has withdrawn a published study on loyalty rewards programmes after AI-research group GPTZero found apparent AI hallucinations, fake footnotes and a reference to a McKinsey report that does not exist.
• The study used inconsistent loyalty-market sizing (the $200bn market size figure also appeared as the size of unclaimed loyalty points) and more than half a dozen footnotes that pointed to non-existent or unrelated web pages.
• EY’s Big Four rival Deloitte had to revise a Canadian government report last year for similar issues, and Sullivan & Cromwell apologised to a New York court last month over an AI-hallucinated filing.

EY has retracted a study on loyalty rewards programmes after researchers exposed apparent AI hallucinations, fake footnotes and made-up data — the latest in a sequence of professional services AI lapses that is starting to look less like accident than systemic process failure. The study, “Points of Attack: Uncovering Cyber Threats and Fraud in Loyalty Systems,” was used by EY consultants in Canada to market their cyber security business until research group GPTZero published a takedown late Thursday.

What the researchers found

GPTZero’s Om Ogale, Paul Esau and Alex Cui flagged a series of telling defects. The report used erroneous or inconsistent data — at different points sizing the loyalty scheme market at $200 billion and also putting the number of unclaimed loyalty scheme points at exactly the same number. More than half a dozen footnotes pointed to web pages that did not exist or did not contain the information cited. A referenced McKinsey report had no apparent existence. The report’s central claim — that loyalty schemes are vulnerable to fraud — may well be true, but the supporting apparatus contained the unmistakable fingerprints of unchecked AI generation.

“Publishing a report online is essentially a form of data injection into the pool of knowledge that is the internet,” the GPTZero researchers wrote. “When the report includes fake information (either vibed citations or false claims) it can ‘poison the well’ by misleading future researchers, especially if the report is published by a well-known consulting firm and hosted on a high-traffic website.” That framing matters — a consulting firm’s authority lends its hallucinations downstream credibility long after the original error is found.

A Big Four pattern, not a one-off

EY is not alone. Deloitte had to revise a report for a Canadian provincial government last year after the document was found to contain fake academic citations. Law firm Sullivan & Cromwell apologised to a New York court last month because a filing in a high-profile case repeatedly misquoted the US bankruptcy code and cited cases incorrectly. The pattern is sharp: the most aggressive professional-services adopters of AI are also the ones tripping over its known failure modes.

EY itself claimed in October that its AI-related revenue had grown 30% in the previous year and that 15,000 staff had worked on client projects “ranging from delivering enterprise-wide transformations to AI governance frameworks that help drive the responsible implementation of AI.” The irony is not subtle: a firm selling AI governance frameworks has published external-facing research that would have failed a basic AI-output verification step.

Looking forward

For UK businesses commissioning research, consultancy or legal work from professional services firms, the EY incident is a procurement signal. Statements of work and acceptance criteria for any AI-supported deliverable now need explicit verification clauses — every citation traced, every data point sourced, every claim defensible. The arXiv one-year ban for unchecked AI-generated content, also reported this week, points in the same direction across academic publishing. The combined message: human accountability does not transfer to the tool, and “we used AI” is not a defence — it is the question. EY said it was “reviewing the circumstances that led to this article’s publication” — the more useful corporate output would be a public methodology change.