Migrating from Claude to DeepSeek

It's a margin move, not a tech upgrade

Lindy priced its product from day one on a bet: that AI models would keep getting cheaper fast enough to move users to lower-cost engines without making the product worse. This migration is them collecting on that bet — they shifted most managed-agent traffic to DeepSeek v4 Flash and cut the cost of running the model (the inference bill — what it costs to actually run a trained model for each answer) by around 90% on the migrated routes.

Read it the way a finance person would: the model is one of the biggest lines in the cost of goods, and this is supplier-switching to protect the gross margin, not a feature release. The line that sticks is "you do not need God to write your emails". Most jobs don't need the most expensive intelligence, so paying frontier prices for all of them is waste.

In software terms it's a dependency swap: replace a costly supplier with a cheaper one that does the same job. The whole post is about how you do that swap without the product silently getting worse.

Why 'it's just a dropdown change' is the trap

The naive migration takes five minutes: pick a cheaper model, run a few prompts, declare victory. That, the author says, is exactly how you ship a worse assistant. A single prompt tells you almost nothing, because a daily-use agent has to survive thousands of tiny moments: write in the user's voice, decide when not to send, stay crisp, recover when a thread is messy.

Their proof is a real failure. They tried Kimi K2.5, which did well on offline evals (replaying saved real tasks and scoring the model before any user sees it), then rolled it out to a small slice of live usage where it failed almost immediately. One user said it felt like their assistant "had brain surgery overnight."

That gap is the whole lesson: a model can score fine on your saved tests and still feel wrong in the wild, so the test set is necessary but never sufficient. The blunt verdict in the post — "offline evals are necessary, they are not enough" — is the bit worth keeping.

How they actually chose, and why the provider mattered

They built a stack of offline evals first. If you run an agent in production you need a way to replay real tasks and compare models without turning users into test subjects. Then they ran several candidates through it: GLM, Kimi, DeepSeek v4 Flash.

The non-obvious finding: the same model scored differently depending on which company served it, because some providers quietly ship a shrunk-down (quantized) version or have a weaker serving stack. So "we tested DeepSeek" isn't precise. You tested a model, a provider, a serving stack, and your own prompts as one system.

Once DeepSeek won, they didn't stop. They ran a GEPA (prompt-optimization loop) — a loop that auto-rewrites prompts to push the eval score up instead of hand-editing in the dark — until the new setup's scores matched the old one. The provider choice is part of the test, not a footnote: pin the exact provider, not just the model name.

The rollout that earned the right to swap

They started with internal users, because employees complain fast and with useful detail. If the assistant feels dumber, you hear it before the graph has enough data to even show it.

Then a small percentage of real users, watching two things. First, online evals — scoring the model on live traffic — because production invents weird inputs faster than you can write tests for them. Second, retention, the slow and honest signal: a subtly worse product rarely gets a support ticket, people just quietly use it less, so you need weeks of data before you trust it. As long as both held, they ramped to 100%.

The author's framing is the keeper: the interesting work was not moving traffic from the old model to the new one, which he calls plumbing. It was earning the right to move it at all. A software engineer would recognise this as a canary release with a regression suite and a real service-level check. The discipline isn't new; it's just applied to a model swap instead of a code deploy.

What it means for the labs

The closing argument is about whether this turns model labs into commodities. The honest answer: partly, and that's not the same as doomed. He uses airlines — brutal, capital-heavy, hard to tell apart, yet Delta is still worth tens of billions — to make the point that commoditization (when rival products become interchangeable and compete mainly on price) can still support huge companies when the market is huge.

The pressure is on margins, not relevance. The tier just below the frontier keeps getting cheaper, and every time it gets good enough for another class of work, application companies move that work down the cost curve (the falling price of a given capability over time). The frontier still commands a premium for the hard slice.

The keeper across the whole piece: the model is rented and swappable. The eval suite that lets you swap safely is the thing you own.

Vocabulary

• inference — running a trained model to get an answer; the per-use cost of an AI feature
• offline eval — replaying saved real tasks to score a model before any user sees it
• online eval — scoring the model on live, real traffic after a limited rollout
• GEPA (prompt-optimization loop) — a loop that auto-improves prompts against an eval score instead of hand-editing
• cost curve — the falling price of a capability over time; you move down it as cheaper models get good enough
• commoditization — when rival products become interchangeable, so they compete mainly on price

If you're building — what to watch for

• Treat the eval suite as the asset, not the model. If you can't replay real tasks and score a candidate against them, you can't switch suppliers safely, and you're locked in by your own lack of testing rather than by the vendor.
• Pin the provider, not just the model name. The same model can score differently depending on who serves it, because some hosts ship a quantized version or run a weaker serving stack.
• Passing offline evals is not permission to ship. Route a small slice of live traffic first and watch a slow signal like retention — a subtly worse product rarely gets a complaint, people just use it less.
• Blend the tiers instead of picking one model: send the bulk of the work to a cheap tier and keep the expensive one for the slice that genuinely needs it.
• Cost is not the only axis on a model swap. Where the cheaper provider runs, and what data you'd be sending it, is a governance decision that can veto the saving outright.

Reading it critically

• It's the vendor's own blog and it ends as a hiring pitch. The ~90% is self-reported, on migrated routes only rather than total spend, with no absolute figures and no quality-delta numbers. Treat the headline as a claim to verify, not a result.
• DeepSeek v4 Flash is a Chinese-developed model served on Atlas Cloud. Routing data there is a data-governance decision, and the post never mentions data residency — a non-starter for regulated or client data.
• Survivorship: you only read about the migration that worked. The Kimi failure hints at how many candidates quietly lost. The method is sound but the success rate isn't shown.
• For a solo builder the playbook is over-built. Retention and online evals across user slices assume thousands of users; with a handful, the honest test collapses to your own judgement — so import the offline-eval and golden-set slice, not the whole apparatus.