UK research: cybercriminals struggling to make AI tools work for them

TL;DR:

• A peer-reviewed UK study by researchers at Edinburgh, Strathclyde and Cambridge analysed around 100 million posts from underground and dark-web cybercrime forums and found AI is not, so far, lowering the skill barrier to cybercrime.
• AI coding assistants are mostly useful to attackers who already have skills; commercial chatbot guardrails are reducing harm meaningfully.
• The biggest near-term risk, the researchers say, comes from poorly secured agentic AI systems and “vibecoded” software produced by legitimate organisations.

The team, led by University of Edinburgh senior lecturer Dr Ben Collier, used machine learning and manual sampling on the CrimeBB dataset to track conversations from November 2022 — when ChatGPT launched — onwards. AI proved most effective in narrow uses: running social-media bot networks for misogynistic harassment and fraud, and helping skilled attackers obscure detectable behavioural patterns. Where the criminal forum chatter sounded most alarmed, it was about loss of legitimate “day jobs” in IT, not about new offensive capabilities.

Context

The finding is a measured counter-narrative to the dominant industry framing of AI as a force multiplier for cybercriminals. It does not contradict warnings such as the UK NCSC’s recent advisory on AI-accelerated vulnerability discovery — those concern skilled attackers and security researchers. The study’s contribution is to clarify which attacker tier benefits: AI raises the ceiling for those who already have skill, but is not yet democratising serious offensive capability to less-skilled actors.

Collier’s headline message to industry: “don’t panic yet” — but the unstated corollary matters. The researchers warn that the more concrete near-term risk is from organisations adopting poorly secured AI systems themselves. Agentic AI deployments — software that takes actions on behalf of users — and code written using AI without rigorous review create new attack surface that does not depend on attacker skill.

For UK CISOs, the practical implication is that AI security spending should weight toward securing internal AI deployments rather than defending against AI-powered intrusion attempts. This sits alongside SecurityBrief’s reporting that only 10% of UK small businesses currently provide staff with AI security training — a gap between adoption interest and security preparedness.

Looking forward

The findings will be presented at the Workshop on the Economics of Information Security in Berkeley in June. The research demonstrates a methodological pattern that UK security policy should encourage: AI threat assessments grounded in observed criminal behaviour, not extrapolation from possible misuse. NCSC, the National Crime Agency and the AI Security Institute all have stated interest in this evidence base. Whether the research alters guidance to UK enterprise — particularly around risk weighting between defending AI deployments and defending against AI attackers — is the practical test of its impact.