Routing policies

Routing policies control which AI provider and model handles each request. Instead of hardcoding a single provider, policies automatically select the best option based on your optimization goals, whether that’s minimizing cost, maximizing uptime, or balancing both with ML-powered routing.

For details on how a policy gets applied to a specific request and how to use default_routing, see Using routing policies.

Strategy comparison

Examples

Request using a routing policy

The project’s policy picks the provider and model. No model field is needed.

1
from merge_gateway import MergeGateway
2
3
client = MergeGateway(api_key="YOUR_API_KEY")
4
5
# The "production" project has a Cost Optimized routing policy.
6
response = client.responses.create(
7
    input=[
8
        {"type": "message", "role": "user", "content": "Summarize this quarter's earnings call."},
9
    ],
10
    project_id="production",
11
)
12
13
print(response.output[0].content[0].text)

With no project_id, the same request falls back to the org default routing policy.

Policy definition

Policies are created in the dashboard. These JSON bodies describe the policy configuration itself. They are never sent in a POST /responses request.

1
{
2
  "name": "Production - Cost Optimized",
3
  "default_strategy": {
4
    "type": "intelligent",
5
    "axis": "cost",
6
    "providers": [
7
      { "provider": "openai", "model": "gpt-5-mini" },
8
      { "provider": "anthropic", "model": "claude-sonnet-4-20250514" },
9
      { "provider": "openai", "model": "gpt-5.2" }
10
    ]
11
  }
12
}

1
{
2
  "name": "HA Failover",
3
  "default_strategy": {
4
    "type": "fallback",
5
    "providers": [
6
      { "provider": "openai", "model": "gpt-5.2", "priority": 1 },
7
      { "provider": "anthropic", "model": "claude-sonnet-4-20250514", "priority": 2 }
8
    ]
9
  }
10
}

Choosing the right strategy

Tag-based routing

You can attach tags to requests (user tier, region, environment, and so on) and use them to route to different policies. Rules are evaluated in priority order: the first matching rule applies, and unmatched requests fall through to the default strategy.

Conditions support AND/OR logic and operators like eq, gt, in, contains, starts_with, and exists. Configure tag-based routing through the dashboard.

FAQs