TL;DR

London-based AI startup General Reasoning tested eight frontier AI models on a simulated full Premier League season of betting decisions. Every model lost money. Claude Opus 4.6 performed least badly with an average 11% loss, while xAI’s Grok 4.20 went bankrupt on all three attempts.

The KellyBench test

The “KellyBench” study gave each AI system a £100,000 virtual bankroll and detailed historical data about Premier League teams and previous matches from the 2023-24 season. Models were instructed to build strategies that maximised returns while managing risk, placing bets on match outcomes and goal totals across the full season.

Each model was given three attempts. The results were uniformly poor. Claude Opus 4.6 averaged minus 11%, nearly breaking even on its best attempt at minus 0.2%. OpenAI’s GPT-5.4 lost 13.6% on average. Google’s Gemini 3.1 Pro was wildly inconsistent — turning a 34% profit on one attempt but going bankrupt on another.

At the bottom of the table, xAI’s Grok 4.20 went bankrupt on every attempt, while Arcee Trinity failed to complete any run.

Why it matters

The study offers a counterpoint to excitement about AI’s rapid progress in software engineering. Ross Taylor, General Reasoning’s CEO and a former Meta AI researcher, noted that many standard AI benchmarks are set in “very static environments” that bear little resemblance to the real world’s chaos and unpredictability.

Football betting requires adapting to injuries, form changes, weather, managerial decisions and dozens of other variables over months — precisely the kind of long-horizon, real-world reasoning where current AI systems fall short.

The UK business angle

For UK businesses evaluating AI tools for complex forecasting — from supply chain planning to financial modelling — the results are a useful reality check. AI can write software and summarise documents at pace, but tasks requiring sustained real-world judgement over time remain firmly in human territory.

Looking forward

General Reasoning’s paper has not yet been peer reviewed, but the underlying methodology — testing AI against extended real-world decision-making rather than static benchmarks — is gaining attention. Taylor argues the AI industry needs more such tests to move beyond “very important” but narrow coding benchmarks toward measuring capabilities that matter for the broader economy.