Routing policies

Routing policies control which AI provider and model handles each request. Instead of hardcoding a single provider, policies automatically select the best option based on your optimization goals, whether that’s minimizing cost, maximizing uptime, or balancing both with ML-powered routing.

For details on how a policy gets applied to a specific request and how to use default_routing, see Using routing policies.

Strategy comparison

The type and axis values above are the dashboard’s policy vocabulary. The Management API and Projects API express the same policies with uppercase strategy values: fallback maps to PRIORITY, intelligent to INTELLIGENT, and the intelligence axis to CAPABILITY. Strategies marked “Dashboard only” can’t be created over the API.

Examples

Request using a routing policy

The project’s policy picks the provider and model. No model field is needed.

1
from merge_gateway import MergeGateway
2
3
client = MergeGateway(api_key="YOUR_API_KEY")
4
5
# This project has a Cost Optimized routing policy. The ID is on the
6
# project's Overview tab in the dashboard.
7
response = client.responses.create(
8
    input=[
9
        {"type": "message", "role": "user", "content": "Summarize this quarter's earnings call."},
10
    ],
11
    project_id="4f8b2c6e-9d1a-4e7a-b3f5-2c8d0a6e1b47",
12
)
13
14
print(response.output[0].content[0].text)

With no project_id, the same request falls back to the org default routing policy.

Policy definition

Policies are created in the dashboard. These JSON bodies describe the policy configuration itself. They are never sent in a POST /responses request.

1
{
2
  "name": "Production - Cost Optimized",
3
  "default_strategy": {
4
    "type": "intelligent",
5
    "axis": "cost",
6
    "providers": [
7
      { "provider": "openai", "model": "gpt-5.4-mini" },
8
      { "provider": "anthropic", "model": "claude-sonnet-5" },
9
      { "provider": "openai", "model": "gpt-5.2" }
10
    ]
11
  }
12
}

1
{
2
  "name": "HA Failover",
3
  "default_strategy": {
4
    "type": "fallback",
5
    "providers": [
6
      { "provider": "openai", "model": "gpt-5.2", "priority": 1 },
7
      { "provider": "anthropic", "model": "claude-sonnet-5", "priority": 2 }
8
    ]
9
  }
10
}

Build Your Own Router is configured entirely in the dashboard, not via JSON. See Build Your Own Router for the setup.

Choosing the right strategy

How Gateway picks a vendor

A strategy chooses the model. Choosing which vendor executes it is a separate step, and it runs the same way whether a policy resolved or you named a model directly.

Gateway starts from every active route for that model, then filters:

1. Routes the request excludes, through a vendor or vendors pin
2. Routes your organization excludes, through vendor allow and ignore lists, region rules, and zero data retention
3. Routes that cannot serve the request’s required capabilities
4. Routes that cannot honor the reasoning.effort level the request asked for, but only when at least one surviving route can, so a request never fails on this filter alone

Among the survivors, the cheapest route wins, measured on combined input and output price per million tokens, with catalog priority breaking an exact tie. A vendor pin and your organization’s vendor routing preference both override the whole thing, and BYOK preference still steers traffic to the vendors you hold keys for.

Two consequences worth planning around:

• Price correctness in the catalog is load-bearing for routing and not only for billing, so a route whose price drops becomes the default for that model.
• A model hosted by several vendors will not necessarily run on its author’s own vendor. deepseek/deepseek-v4 served by a cheaper reseller is the expected outcome rather than a misconfiguration.

Pin the vendor when the serving route has to be a specific one, and read x-merge-vendor on the response, or routing.vendor_used with include_routing_metadata, to see which one served you.

Fallback candidates are ordered the same way, so a failover hop goes to the cheapest remaining capable route rather than back up the list.

Intelligent routing applies the same idea one level up, at the model. Candidates within a small quality band of the closest fit are treated as equivalent and the cheapest of them wins, with the band widening as the strategy leans toward cost: widest on Cost Optimized, narrower on Balanced, narrowest on Quality First. When that swap happens, routing metadata records it and the routing_reason says so.

Tag-based routing

You can attach tags to requests (user tier, region, environment, and so on) and use them to route to different policies. Rules are evaluated in priority order: the first matching rule applies, and unmatched requests fall through to the default strategy.

Conditions support AND/OR logic and operators like eq, gt, in, contains, starts_with, and exists. Configure tag-based routing through the dashboard.

FAQs