Service tiers

A service tier sets how the vendor processes a request. flex costs about half of standard for slower, best-effort processing, suiting evals, enrichment, and background jobs. priority costs more for faster, more consistent processing on latency-sensitive traffic. ultrafast is OpenAI’s fastest tier, at six times standard on every rate. standard is the default.

Tiers follow the vendor route and fail closed: if the serving route isn’t priced for the requested tier, Gateway returns a 400 before calling the vendor, so you’re never served or billed at a tier you didn’t request.

Request a tier

Send service_tier on the request. Add service_tier_fallback: true to retry at standard when the vendor throttles the tier.

curl https://api-gateway.merge.dev/v1/responses \
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "service_tier": "flex",
    "service_tier_fallback": true,
    "input": [
      {"type": "message", "role": "user", "content": "Summarize this support transcript: ..."}
    ]
  }'

If you pin several vendors with vendors, Gateway keeps only those that price the tier. With a routing policy or alias, make sure every route it can pick prices the tier, or the request can fail.

Check which routes price a tier

Each route in GET /v1/models lists its tiers in service_tiers and their rates under pricing. Read them at runtime instead of hard-coding a model list.

{
  "model": "openai/gpt-5.4",
  "vendors": {
    "openai": {
      "service_tiers": ["standard", "flex", "priority", "batch"],
      "pricing": {
        "input_per_million": 2.5,
        "output_per_million": 15,
        "flex": {"input_per_million": 1.25, "output_per_million": 7.5},
        "priority": {"input_per_million": 5, "output_per_million": 30}
      }
    }
  }
}

Flex is priced on OpenAI and on Gemini through both google and vertexai, and requiring zero data retention limits Gemini flex to vertexai. Priority is priced only on OpenAI routes, at 1.67x to 2.5x standard depending on the model. Ultrafast is priced only on direct OpenAI routes, today openai/gpt-6-astra and openai/gpt-6.1-sol; its cached-input, cache-write, and long-context rates are six times standard as well. OpenAI serves ultrafast only on its Responses API, so Gateway sends ultrafast requests there from every surface, Chat Completions included. batch prices batch jobs and isn’t a service_tier value.

Fallback on throttling

A busy vendor can throttle flex, priority, or ultrafast with a 429 or 503. With service_tier_fallback off (default), you get that status back with the message 'flex' tier capacity unavailable for this request, and nothing is billed. With it on, Gateway retries once on the same route at standard and bills the standard rate, which suits jobs where an answer beats the discount. On a stream, it retries only if the throttle arrives before the first token.

Reference

Every response reports the tier that served the request, which is the tier you’re billed at (standard after a fallback). On a stream it arrives on the final frame; earlier chunks carry null.

service_tier also accepts vendor spellings: default, auto, unspecified, and standard_only mean standard, and fast (OpenAI’s newer name) means priority.

Next steps