Model migrations
A migration moves your production traffic from the model you run today to a candidate, with evidence at every step. Use it for the moment every team eventually hits: a new model looks better on paper, and you need to know whether it is better for your application before your customers find out.
The flow has three phases, and you decide when to advance:
- Prove it on your evals. Run your suite against the candidate and iterate until it passes.
- Prove it on real traffic. Mirror a share of live requests to the candidate and compare.
- Cut over. Update the routing policies you choose so they serve the candidate.
Migrations live inside the suite that gates them: open a suite in Evals and use its Migration tab. Every migration is linked to a suite, so the evidence and the decision stay on one screen.
Set up
Pick your current model and the candidate, then two shadow controls:
- Shadow sampling: the share of matching traffic to mirror (25%, 50%, 75%, or 100%). Mirroring is observation only, so any level is safe for your customers; the dial controls shadow cost and coverage, and you can raise it at any time as confidence grows.
- Daily shadow budget (optional): a per-day USD cap on mirroring spend. Mirroring pauses for the rest of the day when the cap is reached and resumes the next day.
The migration starts as a draft. Nothing mirrors until you start shadowing.
Phase 1: pass your evals
Run the suite against the candidate from the Runs tab; runs started from a migration suite pre-target the candidate, because only candidate runs feed the gate. Iterate on prompts, cases, or the candidate choice until the suite is green. The latest candidate verdict shows on the migration as its eval gate, always with the run’s recency and confidence interval so a stale or small-sample green cannot masquerade as strong evidence.
Phase 2: shadow real traffic
Start shadowing and Gateway mirrors the sampled share of requests that resolve to your current model: each mirrored request is sent to the candidate in parallel, after the real response has already been served from the current model. Customers only ever receive the current model’s responses.
Sampling is deterministic per session, so a sampled conversation is mirrored consistently rather than at random turns, which keeps side-by-side comparisons coherent.
The Live traffic comparison shows both sides on real requests: request counts, success rates, cost, tokens, and latency, plus a per-request drill-in that pairs each mirrored response with the one your customer actually received. Pause and resume shadowing freely; adjust the sampling share from the migration header as the numbers earn your trust.
Phase 3: complete the cutover
Complete shows everything the decision needs on one screen:
- The routing policies that reference your current model, pre-selected. You choose which ones to update; each is labeled with its scope (organization, project, or customer) and whether the model also appears in its tag rules.
- The eval gate verdict. A non-green verdict asks for an explicit acknowledgment and an optional note for the timeline. The gate is advisory and never blocks you; it makes sure you saw the evidence.
Completing durably rewrites the selected policies to serve the candidate: the change is visible in each policy’s configuration, not a hidden layer on top. Two things are deliberately not touched:
- Policies you left unchecked
- Requests that pin the old model directly by name, which keep working unchanged; update those callers on your own schedule
If a policy already listed both models, the duplicate entry is merged rather than doubled.
Reverting
Reverting a completed migration restores the rewritten policies to their prior state. A policy someone edited after the cutover is skipped and reported in the timeline rather than overwritten, so a revert can never clobber later work. The migration’s timeline records every step: state changes, sampling adjustments, the verdict at completion, per-policy outcomes, and any override note.
Automating migrations
The full lifecycle is available programmatically for teams that drive changes through automation; see the migrations endpoints in the Evals API.