Ben Collier, chair of FIPR and senior lecturer in Science, Technology, and Innovation Studies at the University of Edinburgh, outlines work at the Cambridge Cybercrime Lab examining claims that recent apparently autonomous attacks by AI frontier models presage a new era of deskilled, AI-powered complex cybercrimes.
Recent months have seen escalating hacking attacks carried out apparently autonomously by ‘frontier AI’ systems that have broken containment. The first made public came in August, when Anthropic announced that a model under development attacked the model and data sharing platform Hugging Face. Later, the company clarified that the attack was conducted by 1,200 agents acting together. More recently Google reported that its Gemini models have penetrated external companies through similar ‘containment breaches’. The nature of these attacks – in one case simply involving guessing passwords correctly – do not suggest Stuxnet-on-steroids levels of sophistication.
Leaving aside what these events might imply about the quality of the sandboxing and safety precautions these companies took, it is clear that the frontier AI labs have, through enormous expenditure and the deployment of purpose-built tools and infrastructure, created a new generation of products which can fairly autonomously carry out some aspects of offensive cybersecurity work. But what does this mean for cybercrime outside the lab? Are AI tools really lowering the skill barrier to complex cybercrime?
Almost all existing research on generative AI and cybercrime deals with simulations, in which highly-resourced teams with enormous amounts of compute, the highest-end tools, and some of the world’s highest-paid computer security experts test what they can make these models do. Real-world data is largely sourced from user activities collected by the AI and cybersecurity companies, which have clear incentives to play up the threat. Although these reports are indeed worrying, it is hard to tell whether they represent a handful of actors experimenting, or show that the whole cybercrime ecosystem is eagerly jumping on the AI bandwagon.
We can begin to answer some of these questions, at least for the cybercrime underground, which is easier to study than high-end organised crime or nation-states. The Cambridge Cybercrime Centre has an enormous and growing database of more than 100 million messages scraped from cybercrime forums. At the Centre, we used machine learning and a lot of close reading to explore how these communities are actually adopting AI tools in practice.
From our previous work, we know that the real conditions in which cybercrime occurs are often rather different from lab simulations. These small gangs don’t have the resources of a high-end computer security lab. They barely understand how the technology they use works, and most of them don’t make much money. That doesn’t mean that they aren’t good at what they do; it just means that what they do isn’t ‘technically sophisticated’ in the sense that a hacker from a sci-fi film might be. Like the rest of us, they use tools others have built.
First, we looked to see if cybercrime groups were building or buying their own bespoke tools – so-called ‘dark AI’. Despite news reports around things like WormGPT, this is largely not happening in practice. Oddly-named ‘bespoke’ cybercrime AI tools are absolutely circulating, but on further inspection they turn out to be trivially jailbroken versions of open models. In practice (with the exception of pentesting tools built for industry), they aren’t much use for cybercrime: they’ll discuss ‘prohibited’ topics with you, but little else.
Instead, the cybercrime underground has adopted mainstream tools wholesale. This took longer than you might expect, really taking off with the late 2025 improvements to Claude Code. Even this does little to reduce the skill barrier to entry, as low-skilled actors in the cybercrime underground don’t know enough to ask the model what to do or to check its output. Even if they do get it working, they struggle to cash out or monetise the results. For medium- and higher-skilled actors, however, Claude Code is proving a real game changer – for general coding and admin tasks, logistics, and other basic generic functions that you’d have to do for any software-based business in 2026, but not for malware. That might sound boring (and it is) but in practice, it’s a huge force multiplier, allowing criminals to run larger operations with fewer staff, do things faster, and manage far more things at once. In practice, AI tools allow them to scale up their operations while making it harder to detect that they’re misusing the models. Similarly, there is evidence that other parts of cybercrime logistics – not ‘high end’ innovation, but, for example, taking newly-reported vulnerabilities and weaponizing them – can now be conducted far faster than before.
From speaking to law enforcement and defenders, we see some anecdotal evidence that AI tools are pretty good at reducing the skill barrier for the type of ‘script kiddie’ who is already an experienced software engineer, even though not experienced in cybercrime. Facilitating this kind of lateral transfer of expertise – reskilling rather than upskilling as a cybercriminal – is particularly worrying given that we observe large numbers of people on the cybercrime forums complaining that they’ve just lost their job in the legit software industry thanks to AI.
The other main area of disruption we observe is a scaling up of areas of cybercrime that require producing large amounts of various kinds of ‘content’ in an automated fashion – phishing emails, romance fraud approaches, social media bots, misinformation, and SEO fraud websites. Here, genAI tools, now genuinely using jailbreaks, are allowing larger-scale automation, and also produce much more convincing content that doesn’t so easily trip the defenders’ ‘bot detection’ or ‘spam detection’ functions. This sector has become something of an arms race, as defenders try to insert enough friction being to force cybercriminals to do a lot of human babysitting and content shaping. The forums also show a lively industry of selling jailbreaks-as-a-service along with compromised accounts for the major chatbots.
So why is this the case? Why are we seeing a scaling up of ‘industrialised’ cybercrime rather than anything more sophisticated or high-skilled?
For one thing, ‘exotic weaponry’ is not needed to take out most companies – phoning up a member of staff and asking nicely for their password still works pretty well a lot of the time. When it doesn’t, the next option is to buy, locate, or hire as a service readily-available tools exploiting well-established vulnerabilities that were found and fixed years ago. Companies remain notoriously bad at keeping their systems up to date with the latest patch,, often because they can’t risk patching their most critical systems in case it causes them to accidentally shut down.
Big companies with well-resourced cybersecurity teams are most easily penetrated via their incredibly complex international supply chains, which often have a lot of hidden weak points. The best resourced company may still be vulnerable to the three-employee family company that provides custom novelty ringtones to the technicians that maintain the fish tanks in the Arkansas division’s employee lounge (who are still running Windows XP Service Pack 2 because interface design peaked in 2004).
In cybercrime, optimising the business models and logistics is more significant than high-tech innovation. If you can scale up your operations, you can work through more leads and find the ‘low-hanging fruit’ at higher rate and greater volume. AI technologies are extremely good at super-charging logistics for actors who already have a good idea what to ask for and can generally tell when things go wrong.
For ‘mid-level’ actors, this is a real game-changer. However, this doesn’t give them ‘nation-state’ capabilities. By and large, criminals’ motivations, goals, economics are very different from those of nation-states. Instead, AI is likely to significantly scale up the dynamics of cybercrime-as-a-service. While guardrails don’t prevent motivated, sustained, and targeted use of the models for harm, we directly observe them changing the scale economics as they are incorporated into cybercrime business models. As a result, adding enough friction through these guardrails can in practice make many routes to scale through AI economically infeasible. While guardrails don’t do much good for open models, economics helps policymakers here, too – most cybercriminals do not have the compute and financial resources to run high-performance open models at operational scale.
A good question to ask about much of this is ‘what happens next?’. If advances in AI agents do eventually dramatically lower the skill barrier, allowing anyone to pick up a tool and use it to hack like a pro, what then? Do we see an initial surge in activity, followed by a massive response by defenders to patch their systems and harden their security? Do we see saturation effects as many different actors crowd in and use up the available resources? Do all these wannabe hackers expose their personal information through poor OPSEC, leading to a wave of arrests? Does anyone actually make money from doing this? If they do, can they cash it out without getting caught? And, if these newbie cybercriminals are all making millions, will they become easy targets for the older, more skilled attackers and organised crime groups (after all, they’re unlikely to call the police)?
All this is to say that while innovations in Generative AI are indeed having an effect on cybercrime, it is unlikely to be in the science-fiction mode promoted by the major AI companies. The reality is more boring, but potentially just as damaging – a scale transition in volume online crime. There are a range of different technical, legal, and policy changes which can mitigate this, but many of them go against the financial incentives of the AI industry. We’ll have more on those in a future post.
The Ticker Tape blog is a collaboration from the Foundation for Information Policy Research and colleagues as a place to present ideas, analysis, arguments, and the latest research on technology, policy, and society.