Using routing policies

A routing policy isn’t sent in the request body. Gateway resolves it for you on every call based on what you attach to the request and what’s configured on your org. This page covers how that resolution works and what you can do from the request side.

How routing policies are applied

Routing policies are attached to an API key, a project, or a customer, or set as the org default. When Gateway receives a request, it resolves the policy in this order:

1. A policy named on the request. An explicit routing_policy_id, a model alias, or a routing policy id sent as the model value.
2. A policy bound to the API key. A key can carry its own policy, which beats everything below it. See Bind a policy to a key.
3. The customer’s default policy, if you resell Gateway capacity and the request names a customer. See Embedded Routing.
4. The project’s policy. A request resolves to a project when it’s authenticated with a project API key, or carries the project_id body field or X-Project-Id header.
5. The org default policy, when nothing above resolved.
6. No policy, direct model calls. Gateway routes directly to the model named in the request.

A key binding that cannot be used, because the policy was deleted, deactivated, or is scoped to a different project or customer, is skipped rather than treated as an error, and resolution continues down the list. POST /v1/embeddings ignores key bindings entirely.

Switching strategies per request

You cannot pass a routing strategy in the request body, but you can name a policy. Create one policy per strategy (for example, one Cost Optimized and one Quality First) and select between them per request:

1. Send the policy’s id as routing_policy_id, or as the model value, or send its @alias/<slug> as the model.
2. Or bind a policy to the API key each workload uses, so no per-request field is needed.

Either way one organization key reaches any policy, which is what you want when different features or user tiers need different cost and quality trade-offs. Setting both routing_policy_id and a policy id in model is rejected with 400 conflicting_routing_policy rather than resolved by guesswork, and a policy scoped to a project or customer that the request does not resolve to returns 400 routing_policy_requires_project or 400 routing_policy_requires_customer.

Sending a priority list with the request

When a call already knows its own order of preference, send it as priority_order instead of creating a policy first. It is priority routing, the same PRIORITY strategy a stored policy uses, expressed inline: Gateway tries each entry in turn and returns the first success. It bypasses every stored policy (the organization default, a customer’s default, a key binding) the way a direct model call does, while each entry still runs under your controls: the key’s allowed models, vendor and region limits, ZDR, and vendor access all apply to every entry, and a model none of them permit is skipped.

curl -X POST https://api-gateway.merge.dev/v1/responses \
  -H "Authorization: Bearer <your_api_key>" \
  -H "Content-Type: application/json" \
  -d '{
    "input": [{ "type": "message", "role": "user", "content": "Summarize this ticket." }],
    "priority_order": [
      "openai/gpt-5.6-sol",
      { "model": "anthropic/claude-sonnet-5", "vendors": ["bedrock"] },
      "zai/glm-5.3"
    ],
    "include_routing_metadata": true
  }'

Each entry is a model id (provider/model, or a bare name that resolves the same way model does) or an object with model, an optional priority (lower tries first, as in the routing policies API; list order otherwise), and an optional vendors list, which pins that entry to those execution hosts in order of preference. Up to 10 entries.

The list becomes a policy

The first request that sends a given list creates a routing policy from it, under the customer or project the request is scoped to, or the organization otherwise. The policy’s id is derived from the list’s content and scope, so the same list is always the same policy: every request that sends it is attributed to that id (routing.policy_id, the x-merge-routing-policy-id header, and per-policy usage), and you can name it later with routing_policy_id instead of resending the list. Creation happens after the response, never on the request path. The policy is named from its content, for example Priority: gpt-5.6-sol → claude-sonnet-5 (bedrock) → glm-5.3, is never a default, and is hidden from the policy list endpoints unless you pass include_inline=true.

How it fails over is the same as a stored PRIORITY policy: an entry moves to the next on a provider fault (a 5xx, a 429, a timeout, a billing 402), not on a request error (a 400, 401 or 403 stops the chain), and an entry tries each vendor that serves its model before the next entry is tried unless it is pinned. On a stream, failover happens before the first byte; a fault after the stream has opened is not retried. When every entry fails, the last entry’s error is returned.

model may be omitted, or set to the first entry, which is what the OpenAI and Anthropic SDKs need since they require a model; any other value is rejected with 400 priority_order_model_conflict on the SDK-compatible surfaces (422 on /v1/responses). priority_order cannot be combined with routing_policy_id, a policy alias, or the top-level vendor / vendors pins: pin per entry instead.

Using default_routing

Set the request’s model field to the sentinel value "default_routing" to explicitly request policy-based routing without naming a model.

The sentinel value is case-insensitive and whitespace is stripped, so " Default_Routing " resolves the same as "default_routing".

default_routing is useful when you have a client that always sends a model field, for example the OpenAI SDK or a templated request payload, but you want every request to honor the project or org default policy. Set the value once at the client level and Gateway will pick the model.

from merge_gateway import MergeGateway
client = MergeGateway(api_key="YOUR_API_KEY")
response = client.responses.create(
    model="default_routing",
    input=[
        {"type": "message", "role": "user", "content": "Draft a response to this support ticket."},
    ],
    project_id="4f8b2c6e-9d1a-4e7a-b3f5-2c8d0a6e1b47",
)
print(response.output[0].content[0].text)

FAQs