The Tragic Alley of the LLM
The attacker and the defender download the same weights. One of them is already using them to get past the other's web application firewall; the other, as of this month, cannot use them in the place where it would matter. Two earlier articles on this blog worked through the evidence for that (what the measurements say, what can be done anyway). This one is about why the asymmetry exists, where it holds, where it does not, and why the way out is the same in every case.
The short version: the asymmetry is not about who has the model. It is about the time budget of the decision. An attacker uses the model offline, needs one success out of many attempts, pays nothing for a wrong guess, and gets to iterate against a defence he can observe. A defender must be right on every input, inline or nearly so, with a false-positive cost paid by legitimate users, on a distribution the attacker chooses. Generative models are very good at the first job and, on every measurement I have found, not yet good at the second.
The shape of the asymmetry
Put the two sides in a picture and the asymmetry stops looking like a property of the technology. It is a property of the row.
In the top row both sides work offline. The model looks for a bug in a codebase, or writes a rule from a cluster of attacks, or rewrites a payload into a thousand variants, and it does the same work at the same speed for whoever is running it. Whatever advantage remains is not about the model: it is about how many findings each side needs and how fast each side can act on one.
In the bottom row only the defender is under the clock. The attacker still generates offline, at seconds per variant, and only sends the variants that got through in his own testing. The defender has to answer each of those in the request path, in the 275 microseconds that a production WAF pipeline actually has, with a wrong answer costing a legitimate user, on a distribution the attacker chose. That is the alley: the model that made the attack cheap is exactly the model that cannot fit the defender's budget.
Where it is symmetric
Zero-day discovery is the case people reach for first, and it is the one where the asymmetry is smallest. By the time an attacker has a model reading a codebase for a bug, a defender can have the same model reading the same codebase for the same bug, and patch generation is exploit generation with the sign flipped. A June 2026 study put Claude Opus 4.6, GPT-5.4 and three self-hosted open models against four WordPress plugins with known vulnerabilities; the best found 63% of them, the best open model 48%, in 32 GPU-hours (arXiv 2606.21397). Read as an offensive tool that is a strong result, because an attacker needs one of the 63%; read as a defensive audit it is a weak one, because a defender needs the other 37% too. But the tool is the same tool, at the same speed, for both.
What separates the two sides in this row is not the model but two things the model does not touch: how many findings each side needs, and what each side has to do with a finding. The attacker needs one and can act on it immediately. The defender needs all of them and then has to ship a fix through a release pipeline, and the window between the first finding and the last deployed patch is the window the attacker lives in. That window existed before language models and is not widened by them; if anything, cheaper discovery on both sides shortens it. The same holds for malware analysis in a sandbox, for red-teaming your own detector, and for building training data: the defender runs the model in the attacker's mode, offline, with many attempts and no per-request cost, and the model is worth exactly as much to him as to the other side.
So the symmetric row is not where the irony lives. It is where the defender should try to be.
Where it is not
Every domain in the bottom row has the same three ingredients: a decision the defender cannot defer, a cost for getting it wrong that lands on a legitimate user, and an attacker who tests offline before sending. The numbers below are the ones I could find that measure the gap directly.
| Domain | What the attacker's model does, offline | What the defender's model has to do, inline | Measured gap |
|---|---|---|---|
| Payload inspection (SQLi, XSS, command injection) | Generates obfuscated variants and keeps the ones that pass | Classify every request within a microsecond budget | GPT-4o XSS variants bypassed ModSecurity at 80%, AWS WAF near 100% (GenXSS); detectors of every family fell from ~98% to 60% on LLM-generated SQLi (IEEE Access, Jan 2026); the cheapest transformer costs 14.7 ms against a 275 µs pipeline (WAMM, Cloudflare) |
| Bot and agent traffic | Browses like a person; solver services clear challenges | Tell an agent from a human before serving the page | Two LLM agents produced "nearly indistinguishable event traces"; the one in a real browser profile passed reCAPTCHA v3, the other did not; solvers beat challenge CAPTCHAs at $0.10–5 per 1,000 (Broken Gates, Jul 2026) |
| Prompt-injection guards | Crafts injections with the same model the guard is built from | Screen every input to the application's model | A guard at 0% false positives on its benchmark ran at 41% on live traffic; the LLM judge behind it took 2–4 s per call (Prismor, Sept 2026) |
| Alert triage | Generates volume: variants, retries, noise | Decide which alert is real, at analyst speed | LLMs classified about a third of attack-related IDS alerts correctly zero-shot, 75–83.5% with examples, and the authors rejected open models as too weak (Springer, Aug 2026) |
Two rows have no recent number I could stand behind and are listed on reasoning alone. Phishing: the model writes fluent, personalised lures at zero marginal cost, and the defender's filter must not quarantine real correspondence; "detect model-written text" is not a reliable basis for that filter. Liveness and identity checks: synthetic voice, face and document generation against a verifier that must decide in seconds with a low rejection rate for real customers. Both have the three ingredients; neither has a measurement in the 2025–2026 record that I found.
What the table does not show is as important as what it does. In none of these rows did the attacker's model do anything the defender's model cannot do; in every row the defender was the one who had to answer now, on every input, and be right. The model is not on one side. The clock is.
The way out of the alley
If the asymmetry is the clock, the remedy is to move the decision off the clock, and that is a design choice, not a research result. A defender cannot make the model faster than the budget, and the record says the model is not the best inline classifier anyway: on every head-to-head I found, a small fine-tuned encoder beat a prompted decoder on both accuracy and cost (an 8B classifier matched DistilBERT at 42× the cost on SQLi and XSS; a fine-tuned DeBERTa reached 95–99% F1 at 12.4 ms on log sequences where a local LLaMA-3-8B needed 156 ms). But a defender can restructure the problem so that most of the judgement happens in the top row.
That is what the two earlier articles amount to. The first established the budget and the ranking: rules and a fast classifier on every request, an encoder on the escalated tail, and the generative model only in the offline loop that writes rules, generates red-team payloads and filters training data. The second proposed three places the local model can sit once it is off the request path, all of them deciding seconds later and acting on an identity rather than on a request: post-processing the escalated payload in context, learning per-endpoint profiles from legitimate traffic, and reading session flows for the abuse that lives in the sequence rather than in any payload. In each one the model does the attacker's kind of work (many attempts, offline, no per-request cost) on the defender's own data, which is the one advantage the attacker does not have.
The irony, then, is real but narrower than it looks. The attacker got the model first because his job was already shaped like what the model is good at. The defender's job was not, and until it is reshaped the model stays on the wrong side of the wire. Reshaping it is the work; the model has been ready for a while.