TL;DR: Oxford AI professor Michael Wooldridge warns that the rush to commercialise AI tools without adequate testing makes a major, confidence-destroying incident “very plausible.” He draws a parallel to the 1937 Hindenburg disaster, which ended public interest in airship travel overnight.
Commercial Pressure vs Safety
Speaking ahead of his Royal Society Michael Faraday prize lecture titled “This is not the AI we were promised,” Wooldridge said companies face “unbearable” commercial pressure to release AI products before their capabilities and flaws are properly understood.
He pointed to the ease with which AI chatbot guardrails can be bypassed as evidence that commercial incentives are winning out over careful development. “You’ve got a technology that’s very, very promising, but not as rigorously tested as you would like it to be,” he said.
The scenarios Wooldridge considers plausible include a fatal software update for self-driving cars, an AI-powered hack that grounds global airlines, or a Barings-style corporate collapse triggered by an AI system “doing something stupid.”
The Gap Between Expectation and Reality
Wooldridge’s concern centres on a fundamental mismatch between what researchers expected AI to become and what actually emerged. Many experts anticipated systems that computed sound, complete solutions. Instead, large language models produce answers by predicting the next word based on probability — making them “very, very approximate.”
This creates AI with “jagged capabilities”: impressive at some tasks, unreliable at others. The problem, Wooldridge argues, is that chatbots fail unpredictably, have no awareness of when they are wrong, and are designed to sound confident regardless.
“Companies want to present AIs in a very human-like way, but I think that is a very dangerous path to take,” he said. “We need to understand that these are just glorified spreadsheets.”
Looking Forward
Wooldridge’s warning carries particular weight for UK organisations integrating AI into operations. His advice is not to abandon AI, but to treat it as a tool rather than a trusted colleague. With AI embedded across sectors from finance to healthcare, the question for UK businesses is whether their internal testing and governance frameworks are robust enough to prevent the kind of failure that could shake public confidence in the technology altogether.