The Fix

Chapter 18: Where This Fails

Six adversarial cases stated as forcefully as possible and examined for what survives. The honest limits of the model, including the most dangerous failure mode: governance capture that produces clean dashboards while dysfunction continues.

The model is now in view in full: the unit and its composition, the charter and its boundaries, the contracts, the synthesis, and the transition that installs them. Before the account of what it looks like in operation, the strongest objections to the thesis deserve examination on their own terms.

What follows is an attempt to break the argument from the inside, using its own evidence and mechanisms. Each case is stated as forcefully as possible, then examined for what survives, and where the objection wins, the text says so.

The attack. The board pack is the weapon. In thirty minutes, an AI assistant produced a technology summary individually defensible in every fact and collectively fictitious, and more authoritative-sounding than anything a human would have written. The CTO marked up three corrections and sent it back, knowing they would not land before the board pack shipped.

Now scale this. Every team has access to the same generator. Product requirements documents reference decommissioned architectures. Quarterly reviews connect strategy to outcomes through plausible chains no one verifies. Process definitions are generated from templates, internally consistent, correctly formatted, and bearing no relationship to how the system behaves. The generator layer scales illusion faster than any human verification can contain it.

Push further. Organisations discover that AI-generated process definitions pass the AI synthesis checks. The artefacts are syntactically correct, structurally complete, and entirely synthetic. A claims process definition describes states, transitions, failure modes, and contract dependencies matching the expected schema. The synthesis compares the definition against the codebase and finds no contradictions, because the definition was reverse-engineered from the code by the same AI that will later audit it. The artefact looks like clarity and functions as camouflage.

The strongest version: the generator produces artefacts that satisfy the reader, and the organisation achieves the appearance of structural coherence without the substance.

What survives. The attack succeeds against document-based verification and fails against runtime reconciliation. The generator/reader asymmetry is architectural, not documentary. A binding process definition constrains deployment rather than describing it: it is version-controlled, referenced in the governance charter, and deviation from it triggers a synthesis flag. The mechanism is reconciliation against system telemetry, not document review.

A synthetic definition reverse-engineered from the code passes a static comparison, but fails the moment the process changes and the definition does not. When the unit lead modifies the triage logic for a regulatory update, a genuine definition changes first, since it is the specification the implementation follows; a synthetic one drifts at the rate of every decision the unit makes, and the weekly synthesis detects that drift as a growing list of discrepancies.

...

See also: Full contents · Preview chapters · Illusions in the Boardroom