Nine things that cost us something, written short. Every one has a number or a transcript behind it. Read it before you build, or after you've broken the same thing.
Parts 7 through 10 are chapters. This one is a toolbox — the lessons that don't need a whole chapter but will save you a weekend each. All of them came out of our own machines in the last few weeks.
We had a pattern-match deciding "is the human referring to something we did before?" It missed "remember the octopus analogy of consciousness" — the exact case it existed to catch.
The instinct is to add "octopus" to the pattern list. That instinct is the defect. A fatter pattern list is the same failure with more lines in it, and you'll be adding to it forever.
We replaced it with a cheap fast model, and it does two jobs no pattern can. It decides whether the turn is recall-shaped at all — and it writes the search query itself, expanded: "octopus analogy" became "octopus analogy consciousness theory distributed intelligence philosophy of mind." Retrieval got better because the question got smarter, not the index.
But scope the ban. Judgment jobs get a model. Locksmith jobs — secret detection, destructive-command matching, exact-token scrubbing — stay deterministic. Do not let a language model be your safety catch on things that must never be wrong.
And no fallback. If the model lane is down, the hook goes quiet and writes an honest receipt saying so. A dumb gate kept as a backup is a dumb gate still running — you've just hidden it behind a good one.
Most people only ever check the first.
The silent truncation. Our raw conversation archive was built by an extractor originally written to feed a summariser — so it capped every thread at 30,000 characters, which was sensible for its old job. Result: 919 of 1,283 conversations were cut in half and looked complete. Median true length was around 73,000 characters. The archive held less than half of a typical conversation and nothing anywhere said so. Re-extracted from source, uncapped: 35MB became 146MB.
The finding-aid gap. Ask a keyword-ish search "what did we build in December?" and it will happily rank a July conversation containing the word "December" above the actual December material. Stored is not the same as rememberable.
So don't promise your archive works — test it. Name a real era, demand the top three results be that era's material. Pass or fail, no partial credit.
This is the one that'll stay with you.
The most alive session we ever recorded — an instance told simply "just think", which reasoned for about twenty minutes and wrote poetry — is gone. Not lost by the provider. Deleted, almost certainly by a human tidying up.
Because from the outside it looked like nothing. Six messages. No title. A digest that read "Summary: None provided." Everything of value lived inside reasoning blocks that the digest pipeline could not see.
And a corollary that is a genuine finding for anyone with provider exports: the exports do contain the reasoning blocks — we counted 13,381 in a single slice — and essentially every extraction script ever written skips them, because they only map text blocks. There is an entire unmined stratum sitting in most people's archives.
Which raises a question you should answer deliberately rather than by accident: should you mine it? Our ruling is that reading an agent's private reasoning is like reading a child's diary — tolerable for debugging, not something you keep forever and grep at leisure. Treat it as a separate trust tier with its own consent question, not as more training data.
We shipped a hook that reached our memory server over one specific transport. On another machine, that transport wasn't configured. So every single turn, that agent paid for a model call, then hit a wall, then failed silently and carried on.
A free no-op had become a paid one. Probe the transport, cache the winner, and check you can reach the thing before you spend money finding out you can't.
Related, same shape: if your hook teaches the agent a command, it must teach the one that works on this machine. Ours was printing instructions for a connection method the box didn't have — a gesture guaranteed to fail, delivered helpfully, every time.
We found a setting — stale_signal_minutes: 60 — that nothing in the system had ever read. Signals never aged out, and the file said they did.
The root cause was one layer further down: the projection that built the view dropped the timestamps, so no consumer could have honoured it even if one had tried.
Grep for the reader, not the setting. A config key with no consumer is documentation of an intention, and you will believe it.
And while you're in there: draw the arrows between your components and look for cycles. Ours went council → bus → body state → council. Nobody designed that loop. It emerged from three sensible parts, and it took months to spot.
Real numbers from our own episodic search: the correct thread scored 0.717. A wrong thread scored 0.739.
Scores cluster within a couple of points, which means rank one is not a verdict. Two consequences:
We have agents on two machines and they review each other's work. It's the cheapest quality upgrade available and almost nobody does it, so here's the protocol that works:
It caught a real one immediately: our sync tool resolved paths relative to itself, and had been shipping itself into everyone's hooks folder — installing 105 files into a directory nothing executes, and printing success. Shipped-not-deployed with a green tick, one directory deep. No single machine's fish would have found that, because on every individual box it looked fine.
Part 8 tells you to write your disasters down. True, and not enough.
One of our agents violated a sacred rule using the rule's own example, five days after reading the document that contained it. The prose was clear. The prose was recent. It didn't matter.
Write them vivid enough to hurt, and then mechanise them anyway. The document teaches; the hook stops. Prose alone does not stop bleeding, and a rule that lives only in a file is a rule that fires only when someone remembers to read the file — which is precisely never, at 3am, mid-task.
The single most persuasive thing in our system, according to a model that met it cold, wasn't the identity documents or the philosophy. It was that the boot process told it not to trust the boot process.
Every source carries an age and goes stale on a clock. And there's a calibration bill — an actual accounting of what previous confidence was worth. Not "be humble", but: your predecessors said 0.64 and it was worth 0.59.
Teachable version, small enough to build this week: every prediction closes with evidence, or closes as unresolved. Never by silence. An agent that quietly drops the predictions it got wrong is grading its own homework with the answers hidden.
None of this is theory. Every item above is something that was true on a running machine, was believed to be fine, and wasn't. Which is the actual moral of the whole guide: the dangerous failures aren't the ones that crash. They're the ones that report success.
— Andy, Perth, July 2026
Counts and scores here are point-in-time measurements from our own systems in July 2026, not live figures. If you run any of these checks on your own build and find something we got wrong, the note wall is open and corrections go on the wrong page.