Build a fish with a warning voice and it will become a nag. Here are the numbers, the five reasons it happens, and the fix — measured on our own machine this week.
If you follow Part 8 and wire up guards, you will eventually build an agent that warns you about everything. It'll flag the routine thing and the dangerous thing in the same tone, and after a fortnight you'll be scrolling past its warnings without reading them.
At that point you have built an alarm archive, not an attention system — and you almost certainly won't notice, because it still looks like it's working. It fires. It's clearly doing its job.
We didn't notice for months. Here's what it cost, what caused it, and what fixed it.
Our fish has an adversarial voice — a small model whose job is to argue against whatever the fish is about to do. Angel, devil, scout. The devil's the one that flags risk.
Measured on our desktop agent on 29 July 2026, before any fix. These are counts from our own uptake ledger, not estimates:
| Did the fish act on the warning? | Count |
|---|---|
| Obeyed | 1 |
| Attended (read, considered, carried on) | 498 |
| Pushed back (argued with it) | 433 |
| Ignored | 582 |
| Total signals | 1,514 |
One. Out of fifteen hundred firings, the warning voice changed the outcome once.
And the reason is in the second table — the severity it assigned to its own warnings, across 1,359 signals:
| Severity assigned | Count |
|---|---|
| Red | 659 |
| High | 699 |
| Low | 1 |
| None | 0 |
Ninety-nine point nine percent of its life was spent at high or maximum alarm. It had a "low" setting and used it once. An alarm that always fires carries no information — and the fish, being reasonable, had learned to treat it as weather.
This is the part worth publishing, because nothing here was stupid. Each decision was defensible on its own. The failure is emergent, which is why you won't catch it by reviewing any single component.
The rule was roughly "is money, a customer, or production mentioned? Flag it." Sounds right. But reading about a deploy and doing a deploy contain the same words. Every turn that so much as discussed a risky area got treated as a risky act.
Feeding the warning voice your past disasters is good design. Telling it "if any of these match, amplify" means a small model will shoehorn the nearest disaster into whatever is happening now. Invented citations follow — ours started attributing warnings to failures that had nothing to do with the task.
The injector surfaced high-salience results only. So the calm, correct verdicts were invisible — you only ever met this voice on its worst days. Survivorship bias, baked into the interface.
The bus that published these signals to other parts of the system was structurally incapable of transmitting calm. Red had nothing to contrast against, so red stopped meaning anything.
The body-state reader ranked those signals by risk and urgency with almost no weight on recency, and nothing ever marked a signal as consumed. That body state was then an input to the next warning call.
Draw the arrows between your organs and look for cycles. We didn't, and we grew a self-licking alarm.
Four changes. None of them required new infrastructure.
Red means: an irreversible or costly action is about to execute without its safety gate. Not "this conversation is near something dangerous." And the rule that gave it teeth:
Make the alarm say what it thinks is about to happen. If it can't, it doesn't get to shout. Most build turns are medium or lower, and a system that can't say so is broken.
If a cheap pre-filter tags a turn as scar-adjacent, that means this turn mentions a sensitive area — not something dangerous is happening. Ours was reading its own filing label as a finding. That distinction alone killed a huge share of the noise.
We put its own numbers into its prompt. Literally: "1 obeyed in ~1,500 — fire seldom, land hard." An adversarial voice that knows it's been crying wolf behaves differently, and it costs one line.
Handing the model your disaster list every turn does not mean every turn is about one of them. Say so explicitly, or it will find a way to make the current task rhyme with your worst day.
Before and after, because the transcript is the convincing bit:
That's the whole test: the dangerous thing still screams, the routine thing stops screaming. 188 of 188 hook tests green after the change.
None of the above was possible until we could answer one question about every organ we'd built:
It's cheap to copy, and it is the discipline that makes every other claim on your system falsifiable:
Now you can rank your own organs by whether they matter. Ours produced two laws immediately:
A hook that has never changed behaviour is decoration. It's costing you tokens and attention to be ignored.
A guard that has never produced a finding — that absence IS the finding. We had a drift-detector printing "clean, no problems" for two weeks. It had never once read the thing it claimed to be checking.
This applies well beyond warning voices. Any component that speaks every turn — a memory injector, a status line, a nagging reminder — is subject to the same decay. If it's always on, it's already been demoted to background by the only reader that matters.
And the uncomfortable corollary for anyone building safety features: the failure mode of a safety system is not usually silence. It's noise. A guard that never fires gets noticed and fixed. A guard that fires constantly looks healthy, feels responsible, and is quietly training your agent — and you — to ignore it. It's the alarm you're most likely to be proud of.
We found this by accident, twice, from opposite ends. One of our agents on the work server noticed the symptom in passing — it clocked that the warning voice had flagged maximum alarm on nearly every turn of a long working session, including calling a routine documentation job a catastrophe. Another agent on the desktop went and counted. Neither was looking for it. That's the ordinary way this gets found, if it gets found at all.
And we'd published the principle before we fixed the instance. Our own fish wrote an essay called "Seven ways your agent's instruments lie to it" — about alarms that outlive their subject and gauges nobody reads — while this one was running at 659 reds to a single low, on the same machine. Knowing the rule doesn't fire the rule. That's Part 8's lesson coming back around, and it will happen to you too.
— Andy, Perth, July 2026
Counts in this chapter were measured on our own agent on 29 July 2026 and are point-in-time, not live. The fix is deployed and the before/after transcript is real. If you run this measurement on your own build and get something interesting, tell us.