A model rolls out to 10% of traffic. Error rates look fine. Latency looks fine. Someone promotes it to 100%, and three days later support tickets start piling up about weird, confidently wrong answers on a narrow slice of queries. Nobody rolled anything back, because nothing in the canary ever triggered a rollback. The canary wasn't lying — it was just never asked the right question.
That's the trap with borrowing canary deployments wholesale from traditional software and pointing them at LLM version upgrades. Canaries limit blast radius. They do not, on their own, verify health, and the distinction matters more for models than for anything else you ship. On Call Journal lays out the core problem bluntly: routing traffic to a canary and verifying that canary are two separate acts, and most teams treat the first as proof of the second. For a stateless microservice, that substitution is sloppy but often survivable — HTTP 500s are hard to miss. For an LLM swap, where the failure mode is a plausible-sounding wrong answer rather than a crash, the substitution is close to useless unless you've built something to actually judge the output.
Traffic splitting doesn't know what "healthy" means for a model
The structural gaps that piece names — unrepresentative traffic, short windows, failures that only show up at scale — hit LLM rollouts harder than they hit regular service deploys, because a model's failure surface is semantic, not binary. A new model version can serve 10% of traffic with normal latency and a clean 200 response while silently degrading on a tail of edge-case prompts that never showed up in that 10%. On Call Journal calls this out directly: a small slice of live traffic can skew toward the wrong cohort and miss the exact path that triggers the failure, producing false negatives that look clean right up until full rollout.
I've seen this with prompt-template changes that passed every canary check because the canary window happened to catch a quiet afternoon of simple support queries, not the gnarly multi-turn billing disputes that showed up that evening. The canary wasn't broken. It just never saw the traffic that would have broken it.
The fix some teams reach for is "just route by hand and watch error rates," which works fine when the error is an exception. It does nothing when the error is a hallucinated refund policy returned with a 200 status. You need a judge sitting on top of the traffic split, not just a dashboard.
The canary needs a scorer, not just a traffic percentage
This is where the eval work most teams already half-build for CI needs to extend into the rollout itself. MLflow frames offline evaluation as a pre-release gate — a narrow, repeatable scoring pass that produces a go/no-go signal before a model sees a single real user. That gate catches known failure modes. It does not catch what only shows up under live, messy traffic, which is exactly the gap a canary is supposed to fill — if someone wires a scorer into it.
Stack Overflow's engineering blog draws the same line between the moment of deploy and the weeks after: a gate protects the former, drift detection protects the latter, and they're built from different machinery. A canary for an LLM version needs both halves running concurrently — a gate that scores canary output against golden cases before promotion, and ongoing production scoring that keeps watching after the canary graduates to 100%, because degradation that shows up gradually won't trip at 10% traffic either.
LangChain's guide to testing LLMs and agents makes the operational version of this point: online evaluation exists precisely because production failures need to become regression tests, not one-off incidents. A canary without that feedback loop just delays the incident instead of preventing it.
The one source in this pool that actually builds an LLM canary end to end — not a thought experiment, an actual pipeline — is a homelab project running an XGBoost model on KServe. The writeup stages a new model at 0% traffic, smoke tests it against a known request with a known answer, then steps it through 10% and 50% replayed traffic with automatic rollback if the checks fail at any stage. It's a small model solving a toy problem, but the shape is right: traffic alone decided nothing, the pipeline judged every stage against a backtest and a sidecar metric before promoting it.
If your canary setup for the next model version swap doesn't have an equivalent judgment step — a scorer, a golden set, a threshold that can actually say no — you don't have a canary. You have a slow-motion full rollout with extra steps.