Routing policies

Routing policies control which AI provider and model handles each request. Instead of hardcoding a single provider, policies automatically select the best option based on your optimization goals — whether that’s minimizing cost, maximizing uptime, or balancing both with ML-powered routing.

Strategy comparison

How routing policies are applied

Routing policies are attached to projects or set as the org default. They are not passed per-request. When Gateway receives a request, it resolves the routing policy in this order:

1. Project-scoped policy — if the request includes a project (via the project_id body field) and that project has a routing policy, Gateway uses it.
2. Org default policy — if no project is specified, or the project has no routing policy, Gateway falls back to the org-level default policy.
3. No policy — direct model calls — if neither exists, Gateway routes directly to the model specified in the request.

Switching strategies per request

You cannot pass a routing strategy in the request body. To route different requests through different strategies:

1. Create one project per strategy — e.g., one with Cost Optimized, another with Quality First.
2. Pass the appropriate project_id field in the request body per request. All projects share the same API key.

This is useful when different features or user tiers need different cost/quality trade-offs.

Examples

Request using a routing policy

The project’s policy picks the provider and model. No model field is needed.

1
from merge_gateway import MergeGateway
2
3
client = MergeGateway(api_key="YOUR_API_KEY")
4
5
# The "production" project has a Cost Optimized routing policy.
6
response = client.responses.create(
7
    input=[
8
        {"type": "message", "role": "user", "content": "Summarize this quarter's earnings call."},
9
    ],
10
    project_id="production",
11
)
12
13
print(response.output[0].content[0].text)

With no project_id, the same request falls back to the org default routing policy.

Policy definition

Policies are created in the dashboard (or the policy management API). These JSON bodies describe the policy itself — they are never sent in a POST /responses request.

1
{
2
  "name": "Production — Cost Optimized",
3
  "default_strategy": {
4
    "type": "intelligent",
5
    "axis": "cost",
6
    "providers": [
7
      { "provider": "openai", "model": "gpt-5-mini" },
8
      { "provider": "anthropic", "model": "claude-sonnet-4-20250514" },
9
      { "provider": "openai", "model": "gpt-5.2" }
10
    ]
11
  }
12
}

1
{
2
  "name": "HA Failover",
3
  "default_strategy": {
4
    "type": "fallback",
5
    "providers": [
6
      { "provider": "openai", "model": "gpt-5.2", "priority": 1 },
7
      { "provider": "anthropic", "model": "claude-sonnet-4-20250514", "priority": 2 }
8
    ]
9
  }
10
}

Choosing the right strategy

Tag-based routing

You can attach tags to requests — like user tier, region, or environment — and use them to route to different policies. Rules are evaluated in priority order: the first matching rule applies, and unmatched requests fall through to the default strategy.

Conditions support AND/OR logic and operators like eq, gt, in, contains, starts_with, and exists. Configure tag-based routing through the dashboard or the API.

FAQs