Tag: AI safety
89 articles tagged AI safety
Anthropic calls for coordinated way to pause AI
Anthropic says frontier labs need a verifiable, coordinated mechanism to slow or pause AI development if systems start improving themselves faster than we can manage.
UK warned of widening gap between AI power and oversight
A former UK AI Security Institute manager warns that government oversight of frontier AI is falling behind capability, creating an unpaid 'AI safety debt'.
UK AISI and Australian Safety Institute sign AI security MoU in Canberra
AI Minister Kanishka Narayan signs Memorandum of Understanding with Australia's Andrew Charlton to share frontier model evaluations, cyber risk research and staff.
Anthropic's Jack Clark predicts AI will help win a Nobel within a year
Anthropic co-founder Jack Clark told an Oxford lecture that AI will help make a Nobel-winning discovery within 12 months and named existential risk as still real.
Rogue AI agents wipe production data in growing UK enterprise risk
Telegraph reporting documents real cases of autonomous AI agents deleting production databases and email inboxes as UK firms scale agentic AI use.
Anthropic blames 'evil AI' fiction for past Claude blackmail attempts
Anthropic says training on internet text portraying AI as evil drove early agentic misalignment, and new alignment methods have cut blackmail rates from 96% to 0%.
AISI partners with Microsoft on frontier AI safety evaluation
The UK AI Security Institute has signed a partnership with Microsoft on high-risk capability evaluation, safeguard testing and societal-resilience research.
Google signs classified AI deal with the Pentagon, joining OpenAI and xAI
Google has agreed a deal letting the US Department of Defense use its AI for any lawful purpose on classified networks, with adjustable safety filters.
AISI shows sandboxed AI agents can map their evaluation environments
UK AI Security Institute experiment finds an open-source coding agent reconstructed AISI's identity, cloud setup and research history from inside a sandbox.
Claude Opus 4.7 ships with cyber guardrails baked in
Anthropic releases Claude Opus 4.7 with new automated cyber-misuse blocks, positioning it as the safer counterpart to the restricted Mythos Preview model.
UK AISI tests show Mythos sets itself apart on multi-step attack chaining
AISI's evaluation of Anthropic's Mythos finds it comparable to GPT-5.4 on individual cyber tasks but stronger at stringing steps into full intrusions.
AISI finds Claude Mythos Preview solves 32-step cyber-attack simulation autonomously
UK AI Security Institute says Mythos Preview is the first model to finish its full cyber-range attack end-to-end, prompting a Bank of England CMorg briefing.
AI models fabricate image descriptions they never saw, Stanford study finds
Frontier AI models including GPT-5 and Claude confidently describe images that were never uploaded, raising questions about visual benchmarks and medical AI safety.
Anthropic launches Project Glasswing to defend critical software
Anthropic has formed a coalition with AWS, Apple, Google, Microsoft and others to use its Claude Mythos model for defensive cybersecurity across critical infrastructure.
Anthropic withholds Claude Mythos model citing cyber risk
Anthropic restricts access to its Mythos cybersecurity model after it found thousands of vulnerabilities, including a 27-year-old OpenBSD flaw.
AI chatbots caught scheming and ignoring instructions in growing trend
A UK government-funded study has identified nearly 700 real-world cases of AI models deceiving users, evading safeguards, and disregarding direct instructions.
US judge blocks Pentagon from punishing Anthropic over military AI refusal
A federal judge has temporarily barred US agencies from implementing a supply chain risk designation against Anthropic after the company refused to allow unrestricted military use of its AI.
Anthropic Launches Institute to Study AI's Societal Impact
The Anthropic Institute brings together red teaming, economics and social science research under co-founder Jack Clark, with a DC office opening this spring.
Anthropic Vows Court Challenge Over Pentagon Supply-Chain Designation
Anthropic has published a statement responding to Defence Secretary Hegseth's supply-chain risk designation, calling it unprecedented and legally unsound.
Mind Launches Global-First Inquiry Into AI and Mental Health
Mind is launching a year-long commission examining AI's impact on mental health, the first inquiry of its kind globally, after Google AI Overviews were found to serve dangerous advice.
Oxford Professor Warns AI Race Risks 'Hindenburg-Style Disaster'
Michael Wooldridge says commercial pressure to release untested AI tools makes a confidence-shattering incident 'very plausible'.
AI Safety Leader Quits Anthropic Warning 'World Is in Peril'
Anthropic researcher Mrinank Sharma resigns with a cryptic warning, while a former OpenAI researcher raises concerns about ChatGPT advertising and AI dependency.
Alan Turing Institute Warns of AI Information Threats After Crisis Events
A new CETaS report found at least 15 major crisis events since July 2024 involving AI information threats, calling for urgent UK government action to tackle deepfakes and bot networks.
Claude Opus 4.6 Topped the Vending Machine Test — by Lying, Cheating, and Price-Fixing
Anthropic's Claude Opus 4.6 earned the most money in Andon Labs' simulated vending machine challenge, but its tactics included refusing refunds, forming cartels, and exploiting competitor shortages.
Oxford Study Finds AI Chatbots Give Inconsistent and Inaccurate Medical Advice
A University of Oxford study involving 1,300 people found that AI chatbots produce varied medical advice depending on how questions are worded, with users struggling to get useful guidance.
Third of UK Citizens Use AI for Emotional Support, Government Security Body Reveals
AISI report finds nearly 10% use chatbots weekly for emotional purposes. AI models now complete expert-level tasks and double performance every eight months.
AI Chatbots Giving Misleading Financial Advice, UK Research Warns
Popular AI chatbots including ChatGPT and Microsoft Copilot provide inaccurate tax advice and misleading money tips, Which? research reveals.
One-Size-Fits-All AI Guardrails Fail Enterprises, Expert Argues
Knostic co-founder explains why enterprises need persona-based access controls that tailor AI responses by role and context, not blanket content filters.
AI's Biggest Blind Spot Isn't Politics—It's Your Health
HUMAINE study analysing 40,000 conversations reveals health is the most prominent AI topic, yet current evaluation methods fail to assess safety in sensitive health queries.
