Rate limits

Gateway imposes no request-per-minute or token-per-minute limits by default, on any credential type. The only rate limits Gateway enforces are ones you set yourself on an individual API key. Otherwise your throughput is governed by the providers’ own capacity, and your protection against runaway usage comes from budgets and spend limits. This page covers the limits that do exist and how to handle them.

Limits Gateway enforces

Provider rate limits

When an upstream provider throttles a request, what happens depends on your routing:

• With a routing policy, a provider 429 counts as a provider failure and triggers failover to the next candidate, so most throttles never reach you.
• Without a policy (direct model calls), the 429 passes through with type: "rate_limit_error" and source: "provider". No Retry-After header is forwarded; back off and retry.
• On the flex service tier, set service_tier_fallback: true to retry once at standard when flex capacity is throttled. See Service tiers.

Handling a 429

Retry with exponential backoff, and honor Retry-After when present. A 429 from a per-key limit carries it; a provider 429 does not:

1
import time, random
2
import requests
3
4
def create_response(payload, max_attempts=5):
5
    for attempt in range(max_attempts):
6
        resp = requests.post(
7
            "https://api-gateway.merge.dev/v1/responses",
8
            headers={"Authorization": "Bearer YOUR_API_KEY"},
9
            json=payload,
10
        )
11
        if resp.status_code != 429:
12
            return resp
13
        retry_after = float(resp.headers.get("Retry-After", 2 ** attempt))
14
        time.sleep(retry_after + random.uniform(0, 1))
15
    return resp

The stronger fix is structural: a Priority routing policy turns provider throttles into automatic failover instead of client-side retries.

Rate-limit headers

Responses carry X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset. When the key has no rate limit configured (the default), all three read 0. They become meaningful only for keys created with rate_limit_rpm or rate_limit_tpm. Retry-After appears only on 429s from a per-key limit.

What Gateway does not limit

There are no default request-per-minute or token-per-minute limits on any key, and none on Merge-managed provider credentials. There is no cap on request body size beyond the inline-media limits, no max_tokens ceiling beyond the model’s own, and no concurrency cap. If you need a hard ceiling on what a workload can consume, use a project budget or per-key spend limit; those are enforced before any provider is called.

Next steps