How it works

Every request passes through the same pipeline. Knowing the order explains most behavior you’ll see: why a request was rejected before reaching a provider, and which routing policy applied.

The request path

1. Authenticate. The API key identifies your organization. Management keys (mgmt_) can’t call model endpoints.
2. Attribute. The request resolves to a project (from a project API key, the project_id field, or the X-Project-Id header) and, for resellers, a customer.
3. Check spend. Organization credit, the project’s budget, and the key’s spend limit are all enforced now; an exhausted one returns 402 before any provider is called.
4. Filter. Blocklist, geo-location, and zero data retention rules reject the request or remove non-compliant vendors.
5. Route. A routing policy is resolved (precedence below), then Gateway picks the vendor and model from the policy’s candidates by capability and current health.
6. Protect. Context compression trims oversized histories; DLP and prompt injection scanning can block with 422.
7. Call, with failover. Provider throttles and outages trigger failover to the next candidate; client errors don’t.
8. Meter. The response is normalized to one shape with the served model, vendor, and usage including per-call cost, and spend is recorded at every scope.

Every rejection along the way has a stable code; see Errors.

Which routing policy applies

Highest precedence first:

1. An explicit routing_policy_id on the request (Embedded Routing, with customer)
2. The customer’s default policy
3. The project’s policy
4. The organization’s default policy
5. No policy: the request’s model is used directly

Naming an exact provider/model bypasses the policy; omitting model or sending default_routing hands the choice to it. See Using routing policies.

Why the order matters

Spend gates run before provider calls, so a blocked request costs nothing. Filters run before routing, so a policy can’t pick a vendor your compliance settings exclude. And attribution runs first, which is why a project API key alone gets you the project’s policy and budget with no per-request configuration.

Next steps