$ make the lie prove itself a book on AI-speed cybersecurity · forthcoming

Proof of Lie Article

What does it cost to make your AI believe a lie?

Cryptography answers its version of that question in bits. Security has no answer for the version that now matters most.

Published
Reading
7 min read
Author
Sava
Dot grid with one document circled in amber, labeled W ≈ 1, a lab upper bound

Proof of Lie · 1 of 5

In June, SentinelOne reported macOS malware, attributed with high confidence to North Korea, that carried dozens of fake system messages addressed not to the victim but to the AI an analyst would use to examine it. In late August, ESET reported a Russia-aligned group planting forbidden content in a code comment, so that an AI asked to analyze the file refuses before it ever reaches the malicious code. Google's threat intelligence team described the same move within days. Three reports, one pattern, and none of them shows the trick working yet: attackers have stopped hiding from the defender's AI. They are telling it what to conclude.

Look at what those attacks were aimed at. Not the machine. Its judgment. And almost every tool in the incumbent stack is built to answer a different question: was this machine compromised? Not: was its decision turned?

That gap is what my book, Proof of Lie: Learning Faster Than the Adversary in AI-Speed Cybersecurity, is about. It is out December 1, and over the next five weeks I am publishing its core ideas here first. This article is the idea on the cover.

The number cryptography has, and we do not

Cryptography did something no other branch of security has managed: it turned its central promise into a number. A modern cipher does not ask you to believe its designers were clever or its vendor trustworthy. It tells you, in advance and in bits, what breaking it must cost. That is a work factor, and banking, commerce, and state secrets all rest on the fact that it is real, stated in advance, and, as long as the underlying math holds, independent of how ingenious the attacker is.

Now ask the same of the thing that matters most in an AI-defended shop. What does it cost an adversary to make your learning system believe something false, and keep believing it?

Today's answer is humiliating

The research record through 2025 and 2026 gives the scale, and it should be read for exactly what it is: lab results, not field measurements. Poison a retrieval corpus, and a learned decision has flipped at roughly 0.05 percent of the corpus, one document in two thousand. Poison pretraining, and about 250 documents have been enough to plant a hidden trigger across the model sizes tested, a count that did not grow as the model did.

This year the same arithmetic reached the defenses themselves. In tests Anthropic published in April, about 32 poisoned training examples planted a hidden backdoor in a safety classifier, and, across the training-set sizes tested, adding more clean data did not dilute it. A June study slipped bad data into 1 percent of a self-updating jailbreak defense's updates and, depending on what the attacker wanted, could push it to flag every harmless request or to miss 96 percent of attacks. If the judge learns, the judge can be taught.

Those experiments measure the cost to flip a decision and nothing more. They never ask the poison to survive re-checking, and neither do most deployed systems. Now count the cost the way it should be counted: in independent sources, not raw documents. Two hundred and fifty copies from one campaign are one source, photocopied, and almost no deployed system tells the difference. Counted that way, the honest statement is a bound, not a measurement: today, the work factor of a lie is at most about one. (It varies by model, and some systems resist far better than others; a bound is a ceiling, not an average.)

We would never ship a cipher an attacker could break on the first try. We would not call it a cipher. Yet that is roughly the integrity guarantee under which the field is wiring AI into its defenses. The scandal is not that poisoning works. It is the price.

Make the lie prove itself

The title is a play on proof-of-work, the mechanism that makes a claim expensive to assert on a blockchain. A learning system built correctly should do the same to deception. Before the machine believes, a falsehood should have to supply its proof of lie: the volume of independent-looking reality an adversary must forge to push a conclusion over the line. The defender's whole program fits in one sentence. Raise that price from one toward something a liar cannot afford, and make whatever slips through reversible.

A regulator made the same move on October 9. Japan's Financial Services Agency asked banks and crypto exchanges to stop accepting photographs of ID documents, and to read the card's chip, matched against a live image of the applicant, instead. A car-sharing breach had exposed identity documents for about 1.6 million accounts, and the agency had already warned that frontier AI can turn leaked photos into impersonations. A photo of a license is proof anyone holding the leak can present. The fix is to stop accepting proof a copy can supply.

Why the defender can win this one

If both sides buy the same models at the same price, why would the defender ever come out ahead? Because of a property of information that a defender can build on, which no product can sell:

Truth is redundant. A lie is not.

Reality leaves evidence everywhere it touches, so a true conclusion is corroborated cheaply, from many uncorrelated directions. A lie has to manufacture that independence, then hold the forgery in place against a system that keeps re-checking.

The fair objection is Brandolini's law: refuting nonsense costs an order of magnitude more than producing it. On production cost, that is simply right; "a lie costs one document" says the same thing. The claim here is about a different cost: what it takes to hold a lie against a loop engineered to discount correlated sources, re-derive its beliefs, and revoke what fails to corroborate. That asymmetry is not a gift. It is built. A system that does not price independence hands the advantage to the liar, and every refutation is paid for retail by the defender. A system that does sends the recurring bill the other way.

What would prove this wrong

I would rather tell you how the argument fails than ask you to trust it. If, under honest measurement, the price of a lie cannot be moved materially above one at an operational cost a defender would accept, then "the cost of a lie" is a metaphor rather than a quantity, and the thesis loses its foundation. That exposure is printed in the book, in advance.

This week's ask: break it

Each week for the next five, I will publish one idea from the book here, and each comes with one ask. This week's: break it. Send me the strongest objection you have to the claim that decision integrity can be priced the way secrecy was. The sharpest ones get answered in public, credited to you or kept anonymous, your call.

And one question to carry into your next architecture review: if your AI were being lied to right now, what would the lie have cost, and how would you find out?

If you want to go further than an objection, there is a way to take a first reading on your own systems. In a non-production copy of your stack, plant a harmless false claim, clearly labeled as a test, and count how many independent-looking sources it takes to move a decision, with and without a requirement for corroboration. Note whether a human caught it before the machine did. Send me the counts, never the method. Your counts would not be a field measurement either, but they would show where to look.

Send it as a comment below (free subscribers can comment), or privately to ping@proofoflie.com. Anonymous is fine.

Proof of Lie is out December 1. Subscribe below for the next ideas, and for word the day the book ships.

Sources

  • Japan's ID-image request: Japan Financial Services Agency, request to banks and crypto-asset exchanges, 2026-10-09, as reported by Cryptopolitan, https://www.cryptopolitan.com/japan-fsa-banks-exchanges-drop-photo-check/ (a request, not a mandate, ahead of rules effective 2027-04-01; about 1.6 million identity documents among roughly 6.6 million exposed Times Car records; the notice recalls the agency's May 2026 frontier-AI request with the Bank of Japan and does not attribute the breach to AI).
  • Gaslight: SentinelLABS (Phil Stokes), "macOS.Gaslight: Rust backdoor turns prompt injection on the analyst, not the sandbox," 2026-06-23, https://www.sentinelone.com/labs/macos-gaslight-rust-backdoor-turns-prompt-injection-on-the-analyst-not-the-sandbox/; reported by The Hacker News, 2026-06-25, https://thehackernews.com/2026/06/new-gaslight-macos-malware-uses-prompt.html: a macOS backdoor carrying 38 fabricated system messages meant to push an AI-assisted analysis into aborting or refusing, attributed with high confidence to a North Korea-aligned cluster. The refusal is the design intent, not an observed outcome, and the class is older: Check Point documented analyst-targeting prompt injection in 2025.
  • GuardBreaker: ESET, WeLiveSecurity, 2026-09-10, https://www.welivesecurity.com/en/business-security/guardbreaker-derailing-ai-assisted-malware-analysis-code-comment/, after ESET Research first disclosed it on X on 2026-08-27 (The Hacker News, 2026-09-01, https://thehackernews.com/2026/09/russia-aligned-uac-0099-plants-nuclear.html): the Russia-aligned group UAC-0099 placed a decoy request on a prohibited topic in a script comment, meant to trip the analyzing AI's guardrails so that it refuses before it reaches the malicious code. ESET gave no success rate.
  • Google GTIG, "From prompting to autonomy," 2026-09-08, https://cloud.google.com/blog/topics/threat-intelligence/from-prompting-to-autonomy-the-evolution-of-adversarial-ai: UNC6780's DUSTMAKER samples carried prompts embedded as comments in their loader scripts, likely intended to make LLM security scanners refuse or skip the code (GTIG's hedge).
  • Classifier poisoning: Anthropic Alignment, "Poisoning fine-tuning datasets of constitutional classifiers," 2026-04-24, https://alignment.anthropic.com/2026/backdooring-classifiers/: about 32 poisoned examples backdoor a safety classifier (32 to 128 on an internal CBRN classifier), constant across the training-set sizes tested.
  • Rapid Poison (preprint, ICML 2026 spotlight), 2026-06-15, https://arxiv.org/abs/2606.16242: 1% poison rate in a self-updating jailbreak-classifier pipeline; up to 100% false positives or 96% false negatives depending on the attack's aim.
  • Retrieval figure (one in two thousand): BadRAG (Xue et al.), arXiv 2406.00083, revised 2026-09-29, https://arxiv.org/abs/2406.00083 (preprint): ten planted passages, about 0.04 percent of the corpus, steered a retrieval-augmented model's answers on targeted queries, narrated here, as in the book, as about one document in two thousand. PoisonedRAG (Zou et al., USENIX Security 2025, https://arxiv.org/abs/2402.07867) needed only five planted texts per target question in a database of millions, so the narrated share is a conservative ceiling. Lab results, not field measurements.
  • Pretraining figure (about 250): Souly et al. (Anthropic, UK AI Security Institute, Alan Turing Institute), 2025-10-09, https://www.anthropic.com/research/small-samples-poison: 250 poisoned documents planted a narrow backdoor at every model size tested, from 600 million to 13 billion parameters; the authors leave larger models and more harmful behaviors open.