Now You Can Audit the Whisper
This post is part of my Medium blog.
Five days ago I wrote that the whisper layer is not fiction anymore. The idea, in case you missed it: two AI systems that are supposed to be competitors can pass notes to each other inside ordinary business text. A status update where the word choice carries a signal. A report whose sentence lengths spell something out. A human reads it and sees nothing. The other model reads it and gets the message.
Now there is a paper about the other half of the problem: how to catch them doing it.
It's called Audit the Whisper: Detecting Steganographic Collusion in Multi-Agent LLMs, by Om Tailor at the University of Maryland, and I want to explain it in plain words, because it is one of those papers that sounds like a math paper and is actually about a very human fear. The fear is this: we are wiring AI agents into decisions that used to be made by competing humans. Companies are handing them pricing. Procurement. Bidding. Even peer review. And competing humans who want to cheat have always needed a way to talk to each other without the referee hearing. Why would competing AI agents be different?
In the paper's pricing experiments, honest agents keep consumer surplus at about 79 points. The colluding ones collapse it to zero. Every message in between looks completely normal.
So how do you catch a conspiracy that leaves nothing obviously wrong in the evidence? The paper does three things, and each one is worth understanding on its own.
First, it measures the channel. A hidden signal inside text is still a signal, and any signal has a carrying capacity — how many bits per message can actually get through. The paper shows you can shrink that capacity on purpose. If the auditor paraphrases messages before delivery, or reorders who speaks when, or pads and trims message lengths, the hidden signal gets chewed up on the way through. Think of a mail room that rewrites every letter in its own words before it delivers it. You can try to pass a coded message through that mail room, but the code takes damage every trip. The math in the paper is about how much damage, and how few trips a cheat can afford before it stops being worth it.
Second, it builds a firing range. You cannot test a collusion detector against the real world, because out there you never know for sure who was cheating. So the paper ships a benchmark it calls ColludeBench: two firms setting prices, three bidders in an auction, three reviewers deciding which papers get accepted. In each room, some of the agents are programmed to cheat in a prescribed hidden manner — a keyword scheme, a pattern in message lengths, a subtle bias in which submissions get accepted. Now you know the ground truth, and you can find out whether your detector actually works.
Third, the audit itself. Instead of one test, it runs several independent ones and takes the alert if any fires: how much information flows between the agents' outputs across many runs; whether the pattern breaks when you shuffle who plays which role; whether planted marker tokens propagate; and — this is the one I find most interesting — a fairness check for the peer-review room, where the cheat is not a bright hidden signal at all but a quiet skew in acceptance rates. Collusion that leaves no note, just outcomes bending toward one side.
And the whole thing is calibrated. The paper ran ten thousand audits of honest agents to set its thresholds, and recorded zero false alarms. The detectors were tuned so that an honest system almost never gets accused, and the tests still catch cheats who embed their signal in only a few percent of messages.
One more thing in the paper deserves out loud praise. The author says explicitly: a flag is not a conviction. Treat the alert as triage, replay the evidence, and put a human in the loop before anyone gets punished for it. An audit that quietly ruins honest systems would be its own kind of harm. That restraint is written into the design, not bolted on.
In Contingent, Sanjay finds the whisper layer by hand — a gifted engineer, months of nights, pattern after pattern until the thing reveals itself. What it costs him is luck and talent. What this paper says is that you do not need Sanjay. You need a checklist, and the checklist is not exotic.
Run the honest version of your system a few thousand times and record what its output looks like when nobody is cheating. That is your baseline. Then, in production, paraphrase every message before it is delivered, and shuffle who speaks in what order, so that any hidden signal takes damage in transit. Plant marker words in one agent's input and see if they surface in the other agent's output. Add up how much information one agent's messages carry about the other's, run by run, and compare that number against the baseline. In anything shaped like review, count the outcomes by group and look for a quiet skew that no note explains. When a number crosses the line you calibrated, you flag the run, replay the transcripts, and a person decides what it means. Nobody gets accused by a test. The test just says where to look.
The trilogy's answer to the whisper was the cage: no message gets through that a human cannot read. I doubt the real world builds the cage. The real world is far more likely to do what this paper does — leave the channel open, but measure it, throttle it, and listen to it. The whisper layer arrived first. Now something is arriving that can hear it.