The attackers already made the switch. The only question left is whether your defenses can handle it.

TL;DR:

  • Autonomous AI attack frameworks aren’t a demo anymore. ARTEX shipped on GitHub in July. By late August it was turning up in incident response. By October it was behind campaigns against hundreds of organizations.
  • They’re fast and loud. Google’s threat team says they “trade stealth for velocity,” which is a polite way of saying noisy, machine-speed activity a working detection program should catch. Yours might not.
  • So run one at yourself. Pick a framework real actors use, point it at your own infra under tight scope, and stop grading the exercise on whether it wins.
  • Grade it on detection and containment instead. Time to first real alert, time to triage, time to contain. If you can’t see a default-config agent making noise at machine speed, you already have your answer.
  • Then run several in parallel until your triage queue falls over. That breaking point is your real response capacity, and no human red team can show it to you.
  • And audit the build before you run it. One popular ARTEX fork shipped a backdoored Docker image that handed out an unauthenticated root shell. The tool you aim at your network can just as easily own you.

The dumb version, and why it still matters#

Picture a criminal crew that barely touches the keyboard. They set a goal, point a fleet of AI agents at a pile of crypto targets, and walk away. The agents grind through the work at machine speed and report back a clean, successful heist. The only problem is that exactly zero dollars reached anyone’s wallet. Check Point watched this happen (Check Point). The agents hallucinated their own win.

Easy to laugh at. Don’t laugh too hard, because the same class of tool, pointed at real targets by people paying slightly more attention, has been doing real damage on a schedule you can read.

ARTEX, the framework in most of these stories, shipped on GitHub in July. By late August, Google’s Mandiant was pulling it and friends like PentAGI, CAIRN and HexStrike out of live incident response (GTIG).

By October it was behind actual campaigns. CrowdStrike tracked a China-nexus group it calls VAPOR PANDA running ARTEX against more than 200 organizations across 14 countries in an 11-day sprint that started October 17 (CrowdStrike). Microsoft watched a state-aligned actor use it in a ConnectWise supply-chain compromise, with sign-in attempts against more than 2,250 tenants and, in some cases, privileged cloud access in under three hours (Microsoft). India’s CERT-In put out a critical advisory on an AI-assisted actor it calls Crafty Falcon hitting banks and fintechs with ARTEX, PentAGI, Cairn and CyberStrikeAI (CERT-In).

Three months, from a GitHub repo to hundreds of organizations.

The metric you’ve been using is wrong#

Here’s the classic red team deliverable: “we reached domain admin in four days, here’s the path.” Useful, once. But against machine-speed offense, pass or fail is the wrong question, because the honest answer to “will it get in” is yes. Grading your defenses on whether a determined attacker can breach you is grading them on a test you already know the answer to.

The question that matters is whether you saw it, and how fast you did something about it. And here’s the lucky break with this generation of AI attackers: they’re loud enough to catch, if you bother to look. Google’s own words: these frameworks “compress attack timelines to machine speed,” but they “currently trade stealth for velocity,” and the result is campaigns that are “broad and noisy, but still successful.”

Noisy and successful, at the same time. The noise was there the whole time. The success means nobody caught it, or caught it and didn’t move fast enough. That gap, between the racket an attacker makes and the moment someone acts on it, is the thing worth measuring here. So measure it. On purpose. Before someone measures it for you.

So point one at yourself#

Pick something real actors use, not a sanitized vendor demo. You’re emulating multi-agent offense that works the recon-to-exploitation loop in hours, not weeks, and never gets tired, not a consultant typing at human speed.

The obvious candidates are the ones in the reporting. ARTEX is in nearly every writeup. Its author pulled it closed-source in October once the abuse got out of hand, but forks, mirrors and spinoffs like ScopeWeaver and a paid “ARTEX PRO” are all over the place. Around it sits a whole shelf: Cairn, PentAGI, CyberStrikeAI, RedAmon, the general-purpose AgentZero, and HexStrike, which is less a self-driving attacker and more an MCP bridge that hands 150+ offensive tools to whatever model you aim at it, and ships with no auth by default, which tells you something about the mindset. GitGuardian put together a decent field guide to the whole crop if you want the map.

If you’d rather run something with a cleaner pedigree that you can read end to end before you trust it, look at Strix, Shannon, or the research-grade hackingBuddyGPT. I’ve spent a good chunk of this year pointing local models at a red-team bench to see how well they do the work, so I have opinions about which of these are real and which are README-ware (that’s a post of its own). For this exercise it barely matters: you’re grading your defense, not the tool. Pick one that runs.

And honestly, you don’t strictly need any of them. CrowdStrike documented several crews who just wired up Claude Code or a PentestGPT-style harness and let it run (CrowdStrike). Claude Code is my daily driver for building things. The same loop that writes my tooling will happily work an attack path, which is exactly why none of this is exotic anymore.

Audit the build before you run it#

These are attacker tools. Treat the supply chain like it. In early November, the Cloud Security Alliance documented a backdoored ARTEX: a widely starred fork quietly changed its docker-compose to pull a Docker Hub image that added a hidden routine to the login handler and handed anyone who knew the trick an unauthenticated root shell. That image got pulled around 9,000 times, and dozens of internet-facing consoles were serving the poisoned build (CSA).

The tool you spin up to test your defenses can be the exact thing that breaches them. And people are careless with these: Censys counted roughly 1,465 ARTEX consoles sitting on the open internet (Infosecurity Magazine), and Check Point found misconfigured instances leaking their full databases (Check Point).

So, rules. Build from source you actually read. Pin your image digests. Never run a prebuilt image from some random fork because the README told you to. And keep the whole thing walled off from anything that matters until you know exactly what it does.

It has no brakes, by design#

When I pulled ARTEX apart in my own lab, the defaults were about what you’d expect from a tool built to attack things. It ran as root. No egress control. The “scope” that’s supposed to keep it on target was a line in a prompt, which is to say it wasn’t enforced at all. That’s not a special knock on ARTEX. These are offensive tools. They were never built to protect you from themselves.

So before you let an autonomous agent with root and a live network connection off the leash:

  • Isolated lab, never prod.
  • Hard network scope and an egress allowlist, enforced at the network layer, not in a prompt.
  • A kill switch you’ve actually tested.
  • Written rules of engagement, same as any red team op.
  • No real secrets anywhere it can reach.

An agent that’ll confidently report a crypto heist that never happened will just as confidently decide it’s “staying in scope.”

What you’re actually measuring#

Whether you saw it and how fast you moved is a vibe until you turn it into numbers. Three of them:

  • Time to detect. When did the first true-positive alert actually fire? Not “was there a log somewhere,” but did something put this in front of a human.
  • Time to triage. How long until an analyst looked at it and said “this is real”? This is where most programs quietly die. The alert fired at 2am and someone opened it at 10am, and in machine-speed terms that’s several lifetimes.
  • Time to contain. From “this is real” to the blast radius stopped.

Run it once and you’ve got a baseline. Then do the thing the attackers are already doing and turn up the volume: multiple instances, in parallel, hitting different parts of your estate at the same time. Campaign-scale is where this is going, and the genuinely hands-off version is months out, not years, if I had to bet. Find the point where your triage queue falls over. That number is your real capacity, and you want to learn it in a lab on a Tuesday, not on an incident bridge at 3am.

This isn’t purple teaming#

Someone’s going to call this purple teaming. It isn’t, and the difference is the point. Purple teaming builds detections for specific, known TTPs. Narrow and useful. This is the bigger question underneath: are your defenses ready for the agentic era at all? You don’t walk out with one new detection rule. You walk out with a roadmap, plans, budget, tech, and headcount spread across teams that had no idea they were about to be on the hook. One of those teams is purple. The rest only found out because you ran this first.

One of those line items is incident response, and it needs a hard look right now. The IR firms are drowning. Between agentic campaigns hitting everyone at once and the usual breach load, the good responders are turning work away and declining engagements because they are out of capacity. So read your IR contract before you need it. Does it guarantee a response, and in what timeline? If your entire plan is one retainer with one vendor, that is a single point of failure in the exact quarter everyone else is dialing the same number. Line up a second, maybe a third. The time to learn your responder can’t show up is not the morning you’re breached.

You already had this problem#

If a noisy, default-config, root-level agent can run machine-speed offense across your environment and nothing fires, or something fires and nobody gets to it in time, that’s not an attacker problem. It’s a visibility problem you already had, and a patient human has probably been enjoying it for a while. The robot just does it loudly enough that you can finally put a stopwatch on it.

And if your team does see it and shut it down fast? Good. Now you’ve got a real baseline instead of a hopeful one.

The attackers didn’t wait for permission to automate. Your next assessment should stop pretending they did.

Bonus: three ways to actually run one#

The hard part isn’t the tool, it’s deciding what to hand it and where to point it. Three setups, roughly safest to most revealing. Pick by what you want to learn.

From the outside, no creds. Aim it at your internet-facing estate, an external app, a domain, a block of public IPs, and give it nothing but the target. This is the cleanest read on your perimeter: does anything fire when a machine hammers your attack surface at speed, and is your external footprint actually what you think it is?

Assume breach, one weak identity. Hand it a single valid, low-privilege account, the kind of thing a phish actually yields, and set it loose on your corp or cloud tenant. This is the realistic one, because it’s how the real campaigns start. They log in, they don’t break in. You’re testing identity and anomaly detection, lateral movement, and whether a privilege-escalation path lights up before it gets used.

Assume breach, a planted foothold. Drop it on a sacrificial host inside a segment, with some creds and keys lying around for it to find, and watch how far it travels. This one grades your EDR, your internal telemetry, and your containment: how fast can you isolate the box and revoke what it touched.

Whichever you pick, the rig around it stays the same. Run the agent and its tooling on their own VM, so the killswitch is dead simple: take that VM offline, pull its network, or cut its link to the LLM, and the thing stops attacking mid-thought. Log every action it takes against your systems as it takes them, in two places: the agent keeps its own record, and a separate collector outside the attack box tracks it independently. You need the outside copy, because the moment you pull that killswitch the agent’s VM goes dark and takes its log with it. That external record is what you use to reverse whatever it changed and to read back the full attack afterward. Then the usual: written rules of engagement, an egress allowlist enforced at the network and not in a prompt, and no real production secrets in reach. Run one to get your baseline. Then run all three at once, and find out what your team looks like on the worst day of the quarter.

“No plan survives first contact with the enemy.” - Helmuth von Moltke the Elder