Rate limits

Paid organizations get no request or token rate limits by default: vendor capacity bounds throughput, and budgets and spend limits bound runaway usage.

Limits Gateway enforces

Every key with rate_limit_rpm counts against one organization-wide counter in fixed 60-second windows, and each key compares that count with its own limit. Keys without a limit don’t add to it. The /v1/keys Management API and the dashboard can’t set a rate limit.

The per-key 429 body has no code:

{ "error": { "message": "Rate limit exceeded. Please retry later.", "type": "rate_limit_error" } }

Vendor rate limits

Under a routing policy, a vendor 429 fails over to the next route, so most throttles never reach you. On a direct model call it comes back with type: rate_limit_error, source: provider, and no Retry-After. On a flex, priority, or ultrafast request, service_tier_fallback: true retries once at standard on a vendor 429 or 503 (on a stream, only before the first token).

Handling a 429

Retry with exponential backoff, honoring Retry-After when present. Stop on free_tier_daily_limit instead, since its Retry-After can be up to a day.

import os
import random
import time
import requests
def create_response(payload, max_attempts=5):
    for attempt in range(max_attempts):
        resp = requests.post(
            "https://api-gateway.merge.dev/v1/responses",
            headers={"Authorization": f"Bearer {os.environ['MERGE_GATEWAY_API_KEY']}"},
            json=payload,
        )
        if resp.status_code != 429:
            return resp
        if resp.json().get("error", {}).get("code") == "free_tier_daily_limit":
            return resp
        wait = min(float(resp.headers.get("Retry-After", 2 ** attempt)), 60)
        time.sleep(wait + random.uniform(0, 1))
    return resp

Reference

Gateway sets no concurrency cap on model requests and no max_tokens ceiling beyond the model’s own. For a hard spend ceiling, use a project budget or per-key spend limit, which Gateway checks before calling any vendor.

Next steps