Using routing policies

A routing policy isn’t sent in the request body. Gateway resolves it for you on every call based on what you attach to the request and what’s configured on your org. This page covers how that resolution works and what you can do from the request side.

How routing policies are applied

Routing policies are attached to an API key, a project, or a customer, or set as the org default. When Gateway receives a request, it resolves the policy in this order:

1. A policy named on the request. An explicit routing_policy_id, a model alias, or a routing policy id sent as the model value.
2. A policy bound to the API key. A key can carry its own policy, which beats everything below it. See Bind a policy to a key.
3. The customer’s default policy, if you resell Gateway capacity and the request names a customer. See Embedded Routing.
4. The project’s policy. A request resolves to a project when it’s authenticated with a project API key, or carries the project_id body field or X-Project-Id header.
5. The org default policy, when nothing above resolved.
6. No policy, direct model calls. Gateway routes directly to the model named in the request.

A key binding that cannot be used, because the policy was deleted, deactivated, or is scoped to a different project or customer, is skipped rather than treated as an error, and resolution continues down the list. POST /v1/embeddings ignores key bindings entirely.

Switching strategies per request

You cannot pass a routing strategy in the request body, but you can name a policy. Create one policy per strategy (for example, one Cost Optimized and one Quality First) and select between them per request:

1. Send the policy’s id as routing_policy_id, or as the model value, or send its @alias/<slug> as the model.
2. Or bind a policy to the API key each workload uses, so no per-request field is needed.

Either way one organization key reaches any policy, which is what you want when different features or user tiers need different cost and quality trade-offs. Setting both routing_policy_id and a policy id in model is rejected with 400 conflicting_routing_policy rather than resolved by guesswork, and a policy scoped to a project or customer that the request does not resolve to returns 400 routing_policy_requires_project or 400 routing_policy_requires_customer.

Using default_routing

Set the request’s model field to the sentinel value "default_routing" to explicitly request policy-based routing without naming a model.

The sentinel value is case-insensitive and whitespace is stripped, so " Default_Routing " resolves the same as "default_routing".

default_routing is useful when you have a client that always sends a model field, for example the OpenAI SDK or a templated request payload, but you want every request to honor the project or org default policy. Set the value once at the client level and Gateway will pick the model.

1
from merge_gateway import MergeGateway
2
3
client = MergeGateway(api_key="YOUR_API_KEY")
4
5
response = client.responses.create(
6
    model="default_routing",
7
    input=[
8
        {"type": "message", "role": "user", "content": "Draft a response to this support ticket."},
9
    ],
10
    project_id="4f8b2c6e-9d1a-4e7a-b3f5-2c8d0a6e1b47",
11
)
12
13
print(response.output[0].content[0].text)

FAQs