AI in the Cybercrime Underground: Hype Meets Craft

When people talk about AI and cybercrime right now, a lot of it sounds like science fiction: autonomous malware, digital superhackers, models that independently dismantle corporate networks at night. I think that story is too convenient. It is loud, it sells well, and it distracts from the real problem.

The dangerous part of AI in cybercrime is not that every criminal group suddenly has its own genius in a basement. The dangerous part is that many boring work steps become cheaper, faster, and easier to scale.

Translating. Rewriting. Sorting victims. Making phishing more believable. Explaining code. Debugging error messages. Evaluating data leaks. Varying social-engineering scripts. Packaging tools. Making claims look more professional.

That sounds less spectacular than “AI writes zero-day malware”. But that is exactly why it matters operationally.

The underground rarely sells pure truth

You have to start this topic with a dose of distrust. Criminal forums, Telegram groups, and so-called cybercrime-as-a-service offers make a lot of claims. Some providers sell real tools. Others sell renamed open-source tools, simple wrappers around public models, stolen accounts, jailbreak prompts, useless panels, or simply fraud against other fraudsters.

When an “AI hacking tool” is advertised somewhere, that does not mean it is technically impressive. It may be a real helper. It may also be nothing more than marketing.

The USENIX study on so-called Malla services, meaning “Malicious LLM-integrated applications”, gives a useful frame. The researchers examined 212 real services from underground environments and found, among other things, eight backend LLMs and 182 jailbreak prompts. That proves two things at the same time: the market is real. But many offers are not secret supermodels. They are wrappers, jailbreaks, and abused public or open models.

Google Threat Intelligence describes a very similar pattern. A service such as Xanthorox was advertised in the underground as its own offensive model, but according to Google it actually relied on jailbroken commercial APIs and open-source components. That is typical for this market: the packaging sounds like an underground lab, while the technology underneath is often much more ordinary.

That matters because the debate otherwise falls into two false extremes.

The first extreme says: it is all just hype. Criminals like to boast, so we do not need to take it seriously.

The second extreme says: AI is taking over cybercrime, and classic security will soon be worthless.

Both are too simple.

The more useful view is more sober: AI is not automatically magic in the underground. But it is a tool that fits into existing workflows. And as soon as a tool saves time, reduces language barriers, or creates more variants per hour, it becomes interesting.

What is already plausible today

Public threat reports from OpenAI, Google Threat Intelligence, and Europol show a fairly consistent picture: attackers use generative AI mainly where it speeds up work and improves quality. Not as a full replacement for skill, but as an amplifier.

Five areas matter especially.

First: phishing and social engineering. Texts become more natural, more local, and more varied. Bad grammar was never reliable protection, but it was sometimes a warning sign. When emails, chat messages, and support scripts become cleaner linguistically, that warning sign loses even more value.

Second: fraud and identity abuse. AI helps create believable profiles and write applications, invoices, support requests, or investment stories. Image, voice, and video fakes come on top. Not every campaign needs deepfakes, but where trust is built through voice, video, or perceived closeness, these techniques become more attractive.

Third: data processing. Stolen data is often messy. Logs, dumps, mailboxes, CRM exports, screenshots, and chat histories have to be sorted, searched, and assessed. AI can help extract names, roles, projects, billing relationships, credentials, internal terms, and extortion points faster.

Fourth: technical support work. Models can explain code, adapt scripts, interpret error messages, write simple loaders, build regexes, understand API responses, or structure exploit ideas. That does not turn a beginner into a top researcher. But it lowers friction.

Fifth: packaging and sales. A mediocre tool can be documented, promoted, and supported better with AI. That, too, belongs to the underground. Cybercrime is not only technology. It is market, trust, support, reputation, and fraud against customers who are themselves acting criminally.

The strongest evidence is in fraud

The more technical a claim becomes, the thinner the public evidence often gets. Fraud and social engineering are different. There are now hard indicators there.

The FBI IC3 Report for 2025 records “AI Related” as a descriptor and lists 22,364 complaints with 893,346,472 US dollars in reported losses. That is not a perfect forensic measurement. “AI Related” does not automatically mean that AI causality was proven. It means victims or investigators reported an AI connection.

Even so, the number is a strong signal. Investment scams with a reported AI connection and more than 632 million US dollars in losses stand out especially, alongside BEC cases with more than 30 million US dollars and employment scams with almost 13 million US dollars. That fits the thesis exactly: the largest documented damage does not come from autonomous malware, but from better deception, better scaling, and more believable identities.

The often-cited phishing numbers also need careful reading. Studies and vendor reports show that AI-generated phishing emails can be very effective. But the honest version is not: “AI magically beats humans.” The more honest version is: AI can reach good human quality more cheaply, faster, and in more variants.

What I do not believe

I do not believe we are already at the point where criminal groups broadly work with autonomous AI agents that choose realistic targets, compromise them, move laterally, exfiltrate data, and complete extortion cleanly without human guidance.

There will be individual experiments. Proofs of concept anyway. But the publicly robust reports point to something else: AI is used for subtasks. For research, translation, code help, campaign preparation, variant generation, social engineering, and data analysis.

That also matches the assessment of the UK NCSC. The NCSC considers fully automated, advanced end-to-end attacks unlikely by 2027. Human-machine teaming is more likely: humans set goals, choose victims, check results, and let AI make individual steps faster.

That is less Hollywood. But it is not reassuring.

Cybercrime does not have to become fully autonomous to become more dangerous. It is enough if the same groups, with the same people, launch more attempts, localize faster, learn faster, and look more professional.

In practice, it does not matter whether an attack comes “from AI”. What matters is whether it becomes more convincing, faster, and cheaper.

The quality of deception is rising

Many security concepts still lean heavily on human recognition: employees should report suspicious emails, recognize false links, notice strange language, and question unusual requests.

That remains important. But it is getting harder.

If an attacker can write an invoice in the style of a known supplier, if a support request cleanly matches a real product environment, if a LinkedIn message no longer sounds machine-translated, and if a scam is rolled out cleanly in ten languages at once, defense shifts.

Awareness alone becomes even less sufficient.

You need technical brakes:

• strong MFA, ideally phishing-resistant
• Conditional Access and device binding
• clean mail authentication with SPF, DKIM, and DMARC
• good detection of OAuth app abuse
• protection against new inbox rules and suspicious forwarding
• clear payment and approval processes outside email
• logging that is not only switched on after the damage is done

AI does not make social engineering new. But it can raise the craft quality. And that is exactly where many companies are already weak today.

Cybercrime-as-a-Service becomes more professional

The second point is the market itself.

Cybercrime has long been divided into specialties. There are initial-access brokers, phishing kits, malware loaders, ransomware affiliate programs, data brokers, hosting, bulletproof infrastructure, money-laundering networks, translators, call centers, and support channels.

AI fits into this structure.

A provider can write documentation faster. An affiliate can adapt campaigns faster. A data broker can describe leaks better. A scammer can answer customer questions. A developer can debug errors faster. A less experienced offender can be coached through technical hurdles.

Microsoft describes AI exactly in this role as “tradecraft”: not primarily as an independent weapon, but as an accelerator for text, code, media, data summarization, persona building, malware debugging, and infrastructure scaffolding. Google, in parallel, sees early experiments with AI-integrated malware such as HONESTCUE, which is supposed to use an LLM API to generate code for a second stage. That should be taken seriously, but here too the point holds: these are building blocks in a workflow, not proof of mass autonomous attacks.

That sounds banal. But markets often change not through one great breakthrough, but through many small efficiency gains.

If a ransomware ecosystem assesses victim data ten percent faster, that matters. If phishing kits are easier to localize, that matters. If a criminal service is sold and supported better, that matters. If a beginner with AI support clears the first hurdles more easily, that matters.

AI is not a replacement for the criminal ecosystem here. It is lubricant.

The defender sees more noise

For admins, MSPs, and security teams, this creates an unpleasant side effect: the noise increases.

More phishing variants. More well-written support fakes. More scans with cleaner user agents. More semi-automated recon. More alleged exploits. More reports that sound plausible. More tools that carry AI in the name but have little technical substance.

That does not mean you must take everything more seriously. It means you have to triage better.

The central question is not: “Was AI used here?”

The better question is:

• Is there a real attacker path?
• Is there an affected system in our environment?
• Is the identity layer protected?
• Is there external attack surface?
• Is there telemetry showing exploitation or preparation?
• Is the reported impact proven or only claimed?
• Do I need to patch, configure, block, monitor, or only document?

That also matches my last thought on Patchday in the AI age: the number of signals is rising. But security is not won by panicking at every number. It is won through better prioritization.

Less attack surface becomes more valuable

The more attacks scale, the more valuable boring reduction becomes.

Fewer publicly reachable services. Fewer local tools. Fewer browser extensions. Fewer admin accounts. Fewer long-lived tokens. Less shadow IT. Less “just quickly approved”. Fewer old test systems that nobody owns anymore.

That is not a glamorous AI counterattack. It is hygiene.

But exactly this hygiene becomes more important when deception gets better and recon gets cheaper. An attacker who can sort faster also finds neglected corners faster. A phishing campaign that is better localized is more likely to hit someone. A data leak that is evaluated faster becomes an extortion template faster.

AI does not only increase the quality of individual attacks. It increases the speed at which bad decisions become visible.

What you should do now in practice

I would not answer this topic with fear, but with a few very concrete questions.

First: are our most important accounts protected in a phishing-resistant way? If admins, finance, HR, helpdesk, or management are protected only with classic push MFA, that is a gap.

Second: can we see suspicious identity events? New OAuth apps, unusual login locations, new inbox rules, suspicious forwarding, and mass failed attempts belong in baseline monitoring.

Third: do we have clear processes for money, data, and access? If a well-written email is enough to trigger a payment, password reset, or new approval, that is not an AI problem. It is a process problem that AI can exploit better.

Fourth: do we verify critical requests outside the original channel? Payment approvals, supplier bank details, password resets, MFA resets, and emergency requests should never be decided only through the message itself. A second, previously defined channel is boring, but very effective.

Fifth: do we check remote hiring and helpdesk processes strictly enough? Fake candidates, synthetic profiles, voice changers, and forged documents are not only an enterprise problem. Anyone hiring external admins, developers, or support roles needs cleaner identity and device checks.

Sixth: do we patch by risk rather than calendar feeling? Internet-exposed systems, VPNs, firewalls, browsers, remote tools, and identity components need a faster track than some internal side product.

Seventh: do we reduce tools we do not really need? Every additional tool is an update risk, a permission risk, and sometimes a data risk. Especially on work devices, less is often a very sensible security decision.

My take

I do not consider AI in the cybercrime underground a distant future debate. It is already part of the way work is done. But the most important point is not the spectacular term “AI”.

The most important point is speed.

Cybercrime is becoming more productive, linguistically better, more automated, and packaged more professionally. Many offers remain exaggerated, some are scams, and some are technically banal. Still, the overall market is shifting.

Anyone who looks only at the supposedly new superweapon misses the real change. It happens in the intermediate steps: text, code, data, support, variants, scaling.

That is exactly where defenders should focus.

Not every organization needs its own AI SOC. But every organization needs less attack surface, better identity security, cleaner processes, faster patch paths, and enough telemetry not to be blind.

AI does not automatically make cybercrime brilliant.

But it makes mediocre cybercrime more productive. And that is enough to take very seriously.

Until next time,
Joe