Intelligent routing

The Intelligent strategy scores each prompt’s complexity and routes easy prompts to cheaper models and hard ones to capable models in your policy. Use it for mixed workloads where one model is overpriced for some requests and underpowered for others.

How it works

1. Gateway embeds the prompt and scores its complexity from 0 (simple) to 1 (complex)
2. The axis reshapes the score: Cost squares it (toward cheaper models), Performance leaves it linear, and Capability takes its square root (toward capable models)
3. Gateway picks the policy model whose catalog quality score is closest to the reshaped score, then picks its vendor

Despite the dashboard describing Performance as minimizing time to first token, it doesn’t measure latency; it sits evenly between Cost and Capability. If scoring fails or takes over 500 ms, the policy’s most capable model serves the request.

Use case

On the Capability axis, a Use case (General, Coding, Knowledge, Creative, Math, Reasoning, or Finance) ranks models on that use case’s benchmarks instead of general quality. Gateway then treats models within 0.02 quality of the best fit as equivalent and picks the cheapest, reporting the swap in routing metadata as price_swap with both models and the quality and price differences.

Configure an Intelligent policy

Eligible models: catalog models with capability metadata, and priced self-hosted models.

Next steps