Two controls on the AI that builds for me

--- title: "Two controls I put on the AI that builds for me" kicker: "AI in Finance — Tested, Not Hyped" date: 2026-06-20 ---

AI in Finance — Tested, Not Hyped

I let an AI system build things for me most days. The reason I can sleep on that isn't the model — models are confident whether they're right or wrong. It's that I wrapped the thing in two controls any finance team would recognise on sight: it audits itself, and nothing it changes about itself goes live without a second set of eyes and my approval. Segregation of duties, applied to code. Neither control is clever. That's the point.

One: the builder doesn't get to approve its own work

In a finance function this is the oldest rule in the book. The person who raises a payment is not the person who releases it. The one who books the entry is not the one who signs the reconciliation. We call it four-eyes — a change needs a second, independent reviewer before it's real — and segregation of duties — the maker and the checker are never the same person. It exists because a single actor, however competent, is a single point of failure and a single point of self-deception.

I run the AI the same way. A change gets proposed by one pass, audited by a separate, independent pass whose only job is to find what's wrong with it, and then it waits for me to approve before it goes anywhere. The maker and the checker are different actors, and the human sign-off is non-negotiable — most of all when the change is one the system wants to make to itself. An AI that is pleased with its own work is still just one set of eyes. Speed is not a reason to collapse the controls; it's the reason you need them. The model will happily generate in a minute something it would take me an afternoon to unpick — so the checker and the gate are what keep that speed from becoming a liability.

Two: assume most failures are quiet

The second control is that the system checks itself and reports — its own health, and whether what it claims is still true of what's actually there. The finance word for it is reconciliation: the report says the balance is X; does the ledger agree? Automations drift. A record says a thing was done; the thing quietly stopped being done three weeks ago. A self-auditing system is the one that notices the gap and tells you, instead of letting the drift compound in silence.

But here is the trap, and I walked straight into it this week. A green light only proves the check ran — not that the thing it checks actually works. I had a status that had been reporting "healthy" for days. It was telling the truth about the wrong question: it confirmed the job had started, not that the job had done anything. The capability underneath had been dead the whole time, behind a reassuring tick. That is the most dangerous kind of failure — not the loud one, the quiet one wearing a clean report.

So I've learned to design for noise. Silence is only safe if every broken state is built to make a sound. A monitor that can only ever say "fine" isn't a monitor; it's a comfort blanket. I now test the hands-off claim against reality — does the thing actually happen, end to end? — rather than trusting that "no error" means "all good." Put plainly: you have to audit the auditor.

None of this is exotic

That's the part worth saying out loud. The conversation about autonomous AI agents fixates on capability — and the question that actually decides whether you can trust one isn't "how capable is the model" but "what controls is it running inside." The answer I keep arriving at is the unglamorous one a controller already knows: independent review, reconciliation, and a healthy distrust of a report that looks too clean.

The AI is the new junior who works fast and never gets tired. Which is exactly why it gets the four-eyes treatment and the self-audit — not in spite of how good it is, but because of it.