Service tiers

A service tier picks how a request is processed. flex trades latency for a lower price and is a good fit for background jobs, evals, and batch work that tolerates variable speed. priority trades price for faster, more consistent processing on latency-sensitive traffic. standard is the default and is used when you send nothing.

Gateway treats a tier as a vendor-route capability, the same way it treats caching. The same canonical model can offer flex on one vendor and only standard on another, so the tier follows the route, not the model name alone.

How service tiers work

A route supports a tier only when that route is priced for it.

Support is defined by pricing presence: Gateway advertises a tier on a route only when that route has a price for it. That keeps billing honest, at the cost of requiring each route to be explicitly priced for every tier it offers.

Which routes price which tiers

Of the 438 vendor routes in the catalog, 396 are standard only. The rest break down as follows.

Nine routes price all three tiers, so a gpt-5.4 or gpt-5.6 request on the openai vendor can go either direction from standard. Two details worth knowing:

• Gemini flex is served by both the direct google vendor and the ZDR-compliant vertexai vendor, so a flex request reaches Gemini flex on whichever route your policy or ZDR setting selects.
• priority is an OpenAI-only tier today. A priority request that resolves to any other vendor fails closed, so pin the route when you need it.

Counts are as of the 2026-09-02 catalog and grow as routes are priced, so read the tiers a route offers from GET /models rather than hardcoding the list. Each vendor route reports a service_tiers list and per-tier pricing:

1
{
2
  "model": "openai/gpt-5.4",
3
  "vendors": {
4
    "openai": {
5
      "service_tiers": ["standard", "flex"],
6
      "pricing": {
7
        "input_per_million": 2.5,
8
        "output_per_million": 15,
9
        "flex": { "input_per_million": 1.25, "output_per_million": 7.5 }
10
      }
11
    }
12
  }
13
}

A tier missing from service_tiers isn’t offered on that route; request it and you get the fail-closed 400.

Requesting a tier

Send service_tier on the request. Optionally set service_tier_fallback to true so a throttled flex request retries once at standard instead of returning an error.

$
curl https://api-gateway.merge.dev/v1/responses \
>
  -H "Authorization: Bearer YOUR_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "openai/gpt-5.4",
>
    "vendor": "openai",
>
    "service_tier": "flex",
>
    "service_tier_fallback": true,
>
    "input": [
>
      {
>
        "type": "message",
>
        "role": "user",
>
        "content": [
>
          { "type": "input_text", "text": "Summarize this transcript: ..." }
>
        ]
>
      }
>
    ]
>
  }'

service_tier accepts standard, flex, and priority, plus the spellings the provider SDKs use for the same three tiers, so a request written against another vendor’s API parses unchanged:

Any other value is rejected with 422. scale and PROVISIONED_THROUGHPUT stay rejected because no route prices them.

Fallback on throttling

flex is best-effort at the provider, so a busy provider can throttle it (429 / 503). What happens next depends on service_tier_fallback:

• false (default): the throttle surfaces to you as the provider’s 429 / 503, and nothing is billed. Retry later, or resend at standard.
• true: Gateway retries the request once on the same route at standard, billed at the standard rate. The service_tier in the response shows the tier that actually served, so a fallback shows standard even though you asked for flex.

Set service_tier_fallback: true when you would rather get a standard-priced answer than an error under load, and leave it off when a flex request must either run at the flex price or not at all.

Reading the served tier

Every response reports the tier that actually served the request, which is also the tier you were billed at. On a fallback this differs from what you requested.

Each surface reports it in its own SDK’s dialect, so read the field your surface uses:

On a stream the tier lands on the terminal frame: the final chunk on the chat surfaces, and inside response.completed on the Responses surfaces. Interim chunks carry service_tier: null.

1
{
2
  "model": "openai/gpt-5.4",
3
  "vendor": "openai",
4
  "service_tier": "flex",
5
  "usage": {
6
    "input_tokens": 812,
7
    "output_tokens": 180,
8
    "total_tokens": 992
9
  }
10
}

Bill off service_tier, not the tier you sent: it reflects the rate charged, including the standard you land on after a throttle fallback. Track it across your traffic to measure flex savings.

Best practices

• Use flex for latency-tolerant work. Batch scoring, evals, offline enrichment, and background agents are ideal; interactive, user-facing calls usually are not.
• Reserve priority for traffic where the premium is worth it. At roughly double the standard rate, it pays off on user-facing paths with a latency budget, not on bulk traffic.
• Add service_tier_fallback when an answer beats an error. For flex traffic that must still complete under load, fallback trades the discount for reliability on the throttled subset.
• Pin model and vendor when the tier matters. A routing policy can land on a route that does not price the tier, which fails closed. Pin the route, or configure the policy with routes that all support the tier.
• Discover support at runtime. Read service_tiers from GET /models instead of hardcoding which models offer flex, since the catalog changes.

Common errors

Next steps