South Africa pulls AI policy after AI-generated citations exposed

TL;DR:

• South Africa’s draft national AI policy was withdrawn after parliament-stage review uncovered fabricated, apparently AI-generated citations in the document’s source list.
• Communications and Digital Technologies Minister Solly Malatsi acknowledged the failure on 26 April, calling it a compromise of the policy’s integrity that proves “vigilant human oversight over the use of artificial intelligence is critical”.
• Resultsense view: the case is a working cautionary tale for any UK department drafting AI strategy with LLM assistance — particularly given the increasing volume of hallucinated-citation cases logged in legal practice.

The withdrawn document had outlined the establishment of a national AI commission, an ethics board and a regulatory body, alongside tax incentives, grants and subsidies to attract private investment. The country’s stated aim, Reuters reported, was to position itself as Africa’s leading AI hub. Malatsi confirmed via X that AI-generated citations were included without proper verification and gave no timeline for a revised draft.

A broader pattern

Hallucinated citations are not a fringe failure. Damien Charlotin, a lawyer and data scientist, maintains an online database that has logged more than 900 US legal cases in which AI-generated briefs contained fictitious case references — and four prior cases in South Africa, before this latest. Lawyers in multiple jurisdictions have been reprimanded after submitting AI-drafted documents that pointed to non-existent precedents.

The South African case is notable because it landed on a piece of national policy, not litigation. The political failure is therefore more public than typical legal-practice hallucinations: a draft policy moving toward parliament-stage finalisation should have been read carefully by at least one official in a position to verify references.

UK relevance

The UK has been more cautious. The Department for Science, Innovation and Technology has published guidance on AI use in government, the Cabinet Office runs internal tooling controls, and the ICO has issued procurement and data-governance materials. But the underlying risk is the same: as more departments use LLMs for drafting and research, the bar for human verification has to keep rising. Defra’s recently announced agentic-AI service manual project, covered separately, explicitly built editorial review into the workflow — a pattern South Africa’s document lacked.

Looking forward

For UK businesses and public-sector teams using LLMs in document production, the practical lesson is procedural: every citation, statistic and reference produced by a model needs human verification against original sources before publication. The cost of doing so is small; the cost of a withdrawn document, like South Africa’s policy, is high. Expect ICO and Cabinet Office guidance to harden on this through 2026 as more cases surface.