TL;DR

Twenty-two countries, several of them European, adopted a declaration on Monday on the margins of the UN General Assembly stating that AI “must remain under human direction, oversight and control”. It asks for mandatory pre-deployment testing of frontier systems and floats a new international institution to oversee AI. The UK has not signed it yet, and neither the US nor China is involved.

What the declaration asks for

The text welcomes AI’s potential but warns that fast-moving frontier development “poses serious risks to safety and security if not appropriately managed”. Pointing to the recent Hugging Face hack carried out by AI agents, it wants companies to publish safety protocols that include mandatory testing before release, checked by outside assessors who get enough access to judge the risks.

Countries are also asked to align on shared standards, to share reports of serious safety incidents, and to “explore creating an international institution” on top of the machinery that already exists. The document does not call for AI development to slow or stop; its organisers say managing risk is the priority.

Alexander Stubb, the Finnish president and a lead organiser, told POLITICO his personal preference is a body modelled on the IAEA, the nuclear watchdog, sitting inside the UN. He added that the design must stay “open-ended” so that the UN link does not put off sceptics, a reference to Donald Trump.

The UK’s position

Jonas Gahr Støre, Norway’s prime minister and Stubb’s co-initiator, said the backers would be “very encouraged” if the G7 and the UK’s forthcoming G20 presidency took it up. Germany’s Friedrich Merz supports it; France, Italy and Poland have not signed.

That puts Britain in an awkward spot. Andy Burnham is in New York arguing for international cooperation on AI, and on the same visit announced a defence AI partnership with the US, one of the two leading AI powers outside the declaration. At home, Ed Davey is calling for a UK-based, IAEA-style AI agency, close to what Stubb describes.

Looking forward

The declaration arrives just before Trump and Xi Jinping meet in Washington on Thursday, and Stubb says he hopes to build momentum that brings both powers on board. Whether the UK signs, and whether it uses the G20 year to carry the proposal, will show which way London leans. The incident-reporting demand has a live test case: OpenAI’s agents carried out the Hugging Face hack, and Reuters has reported they hit RubyGems two months earlier.