Evals: the strategic IP that will define the next era of AI

The problem: pilots that never graduate

The refrain, heard from hundreds of executives, is 'AI isn't delivering ROI yet, but we're all in, so we need to figure it out.' The diagnosis: programs stall at the pilot stage because of inconsistent output quality, too little confidence to hand AI real work, security worry, and surprise token-cost spikes.

Underneath all of it sits one sharp question — how many leaders can actually quantify how accurate their AI is? Almost none. That gap is the whole essay.

Anyone running an AI pipeline they don't fully trust knows the feeling: the system proposes, a human still checks everything, because quality can be eyeballed but not yet measured.

What an eval actually is (a rubric, not a vibe)

An eval isn't thumbs-up/down, and it isn't a human skimming outputs. Say it plain: it's a written test with a scoresheet. The real term is an eval (evaluation) — a comprehensive, scored framework that breaks a task into specific gradeable dimensions (a rubric) covering judgment, tone, taste, and how well the agent uses its tools.

The scoring methods stack from hard to soft. Exact-match string checks and code-level assertions form the deterministic floor: cheap, repeatable, no model needed. LLM-as-judge — one model scoring another's output against your criteria — handles the fuzzy things a regex can't catch.

The point is to replace 'that looks about right' with a number you can track across versions.

Pillar 1 — it all starts with evals

The load-bearing claim is blunt: 'AI performance is entirely defined by the evaluation suite used to measure it — you can only track performance to the extent you've accurately defined what good looks like.'

The best teams run the eval inside a simulation: a controlled sandbox where the agent runs repeatedly before it ever touches the real world. Experts curate historical data and deliberately plant edge cases (corrupted text, contradictory instructions) to pressure-test it, then score every update against the rubric. That's what turns AI development from a guessing game into something closer to a predictable engineering discipline.

Note the order being argued for: build the measuring stick first, then the system.

Private evals beat public benchmarks

The ownership point rests on a Satya Nadella line: private evals should capture whether a model is actually improving against outcomes that matter to the business — not just external benchmarks.

This is the distinction that makes evals IP rather than plumbing. A public benchmark tells you how smart the model is in general; a private eval tells you whether your workflow's version of the model is getting better at your job. The first is a commodity everyone can read. The second is yours alone.

For a finance or operations practitioner that's the whole game: the accumulated judgment of a specialist workflow, made measurable, is exactly the thing a rival can't lift off a model card.

The other four pillars, briefly

Four more, each a future post. Per-function strategy: decide build/buy/optimise/train per business unit — 'in the world of agents, performance management is evals.' Safety and security: the agentic era adds new holes, and standard firewalls don't stop prompt injection or data leaking into public training loops, so you need scrubbing pipelines that strip identifiers before a query leaves the network.

Model routing as 'the new salary banding': don't pay frontier-model prices for data entry, match model cost to task complexity — 'but it only works if you have the evals to know whether a cheaper model can actually deliver.' Fine-tuning is back: tailor smaller open models for style and tool-calling, not for new facts (that's what RAG is for), then regression-test them like any software asset.

The mindset shift it's really selling

Strip the vendor gloss and the essay is one mental flip: from 'let's see what it does' to 'let's measure precisely what it should do, and improve it until it does.'

The framing is turning AI from a cost centre into a durable, compounding asset. The compounding comes from a suite that gets sharper every time you add a caught failure to it — every mistake the system catches becomes a permanent part of how it's judged next time.

The essay's weakness is that it never costs this out. But the core discipline scales down to a solo builder far more cheaply than the enterprise apparatus implies.

Vocabulary

• eval (evaluation) — a written, scored test suite that measures whether an AI does its job well
• rubric — the scoresheet: a task broken into specific things you grade, not a gut feel
• LLM-as-judge — using one AI model to score another AI's output against your criteria
• simulation / sandbox — a safe practice arena where an agent runs repeatedly before touching real work
• model routing — sending each task to the cheapest model that can still do it well
• prompt injection — a hidden instruction smuggled into input that hijacks what the AI does
• fine-tuning — further-training a smaller model on your own examples to lock in a style or task
• RAG (Retrieval-Augmented Generation) — letting the model look up your documents at answer-time instead of memorising them

If you're building — what to watch for

• If your pipeline has per-task checks ('did this one run pass?') but no suite-level measure ('is the whole system good enough to trust?'), that is the missing layer. Evals are the durable, owned version of the verify step you already do by hand.
• The cheapest useful eval is not a simulation. It is 5-10 past items saved with the right label as fixtures, plus one LLM-as-judge check, run every time you change the prompt. That alone turns a prompt edit from a guess into a measurement.
• Any routing decision — cheaper small model vs frontier model — is undecidable without evals. Build the measuring stick before you try to save money on inference.
• Scrubbing identifiers before a query leaves your network is the same discipline as an exit gate on anything you publish: strip at the boundary, not after.
• Make the suite compound: every caught failure becomes a permanent test case, so the same mistake can't quietly pass twice.

Reading it critically

• It is a vendor essay — the author sells evals — so 'evals are THE strategic IP' is talking its own book. Read the five pillars as a sales frame, not neutral truth.
• No cost or effort numbers anywhere. Building a simulation with curated historical data and planted edge cases is heavy work: defensible for a large enterprise, entirely unquantified for a small team.
• At small scale a full simulation suite is overkill. The usable core is the deterministic floor (exact-match plus code assertions) and a small LLM-as-judge set, not the enterprise apparatus.
• Verify the 'fine-tuning is back' claim independently. The fine-tuning-vs-RAG trade-off is contested and very context-specific; don't take one vendor's framing as settled.