Anthropic calls for coordinated way to pause AI

TL;DR:

• Anthropic says frontier developers should build a coordinated, verifiable way to slow or temporarily halt AI development if systems begin self-improving too fast.
• It warned that “full recursive self-improvement” could raise the risk of humans losing control.
• A credible pause, it said, would need agreement among several well-resourced labs, plus rules on what triggers and lifts it.

Anthropic has proposed that the AI industry prepare a collective “brake pedal” for the moment advanced systems start building better versions of themselves. In a statement on Thursday, the company said it would be useful for the world to have the option to slow or pause frontier development so that alignment research and societal structures can keep pace. As a marker of how fast that horizon is approaching, it noted that more than 80% of the code merged into its own codebase in May was written by its Claude model.

Why a unilateral pause does not work

The proposal is notable for what it rejects. Anthropic argued that a single company hitting the brakes alone would achieve little — it would mostly hand leadership to less cautious rivals rather than buy society time. A meaningful slowdown, it said, would require agreement among “multiple well-resourced labs” at the frontier, alongside agreed conditions for triggering and lifting a pause and a body to oversee it. Its research arm plans to study the verification systems such a scheme would need, and to convene policymakers, researchers and rival firms in the coming months.

The timing is pointed. Anthropic recently closed a funding round valuing it at $965bn (roughly £720bn) and filed confidentially for a US IPO — a near-trillion-dollar company arguing for industry-wide restraint. For UK observers, the idea echoes the coordination logic behind the AI Safety Institute and the Bletchley process, where the hard part has always been verification and enforcement, not the principle of caution.

Looking forward

Anthropic’s “pause button” is a governance proposal, not a commitment, and its credibility rests on rivals signing up. It sharpens a week of safety signalling that also saw AI chiefs urge laws against bioweapon design. Whether any coordinated brake can be built — and trusted — is the question UK regulators and international partners will now have to weigh.