> This page is for Gateway.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.merge.dev/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.merge.dev/_mcp/server.

# Prompt caching

> How Gateway handles cache markers on explicit routes and session hints on automatic routes, and how to read cache token usage on each surface.

Prompt caching lets a vendor reuse a prompt prefix it already processed, so cached tokens bill at a lower rate and skip reprocessing. Use it when requests share a long system prompt, tool schemas, a document, or a growing agent conversation.

Caching follows the vendor route, not the model, and never fails a request: if a route can't honor a cache marker, Gateway removes it.

## How caching works

Each route belongs to one caching family.

| Family      | What you send                                   | What Gateway does                                                             |
| ----------- | ----------------------------------------------- | ----------------------------------------------------------------------------- |
| `explicit`  | `cache_control` markers on the content to cache | Forwards the markers. Places one for you when the prompt is evidently reused. |
| `automatic` | Nothing, or a session hint                      | Strips markers. The vendor caches repeated prefixes on its own.               |
| `none`      | Nothing                                         | Strips markers. No caching.                                                   |

On an `explicit` route with no marker, Gateway adds one when the request has an earlier assistant turn, tool definitions, or a session hint: top-level on `anthropic`, and on the latest user, system, or tool message elsewhere, so the cached prefix grows with the conversation. OpenAI-shaped clients such as Codex, Cursor, and the OpenAI SDK cache against Claude without client changes. A single-turn request with no tools or session hint goes as-is, since a cache write only pays off on reuse.

| Family      | Vendors and routes                                                             |
| ----------- | ------------------------------------------------------------------------------ |
| `explicit`  | Anthropic, Claude on Bedrock, Amazon Nova on Bedrock, Z.AI                     |
| `automatic` | OpenAI, Azure, Google Gemini, DeepSeek, Qwen, Mistral, xAI, Fireworks, Baseten |

`/v1/models` doesn't list the family; check the route's `pricing.cache_read_per_million` and `pricing.cache_write_per_million`. A route with no cache-read price bills cached reads at the full input rate. Automatic caching is best-effort, and vendors only cache prefixes above a minimum length.

## Mark a cached prefix

On `explicit` routes, put `cache_control` on the last block of the stable prefix. That block and everything before it are cached, so put unchanging content (instructions, tool definitions, shared documents) first.

**`cURL`**

```bash title="cURL"
curl https://api-gateway.merge.dev/v1/responses \
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-haiku-4-5-20251001",
    "max_tokens": 1024,
    "input": [
      {
        "type": "message",
        "role": "user",
        "content": [
          {"type": "text", "text": "Acme Corp onboarding handbook: ...long stable document...", "cache_control": {"type": "ephemeral"}},
          {"type": "text", "text": "What is the PTO policy for a new hire in their first 90 days?"}
        ]
      }
    ]
  }'
```

**`Python`**

```python title="Python"
import os
from merge_gateway import MergeGateway

client = MergeGateway(api_key=os.environ["MERGE_GATEWAY_API_KEY"])

response = client.responses.create(
    model="anthropic/claude-haiku-4-5-20251001",
    max_tokens=1024,
    input=[
        {
            "type": "message",
            "role": "user",
            "content": [
                {"type": "text", "text": "Acme Corp onboarding handbook: ...long stable document...", "cache_control": {"type": "ephemeral"}},
                {"type": "text", "text": "What is the PTO policy for a new hire in their first 90 days?"},
            ],
        }
    ],
)
```

**`Anthropic SDK`**

```python title="Anthropic SDK"
import os
from anthropic import Anthropic

client = Anthropic(
    api_key=os.environ["MERGE_GATEWAY_API_KEY"],
    base_url="https://api-gateway.merge.dev/v1/anthropic",
)

message = client.messages.create(
    model="anthropic/claude-haiku-4-5-20251001",
    max_tokens=1024,
    system=[{"type": "text", "text": "Acme Corp onboarding handbook: ...long stable document...", "cache_control": {"type": "ephemeral"}}],
    messages=[{"role": "user", "content": "What is the PTO policy for a new hire in their first 90 days?"}],
)
```

Markers work on text, image, document, audio, and video blocks, and on a whole message. Or send one top-level `cache_control`: on `anthropic`, Gateway forwards it and Anthropic moves the breakpoint forward as the conversation grows; on other explicit routes, Gateway converts it to a marker on the latest message.

Markers accept Anthropic's `ttl` values, `5m` (default) or `1h`, passed through unchanged. Anthropic rejects a request whose top-level marker and targeted block carry different `ttl` values. Every cache write bills at the route's single write rate, whatever the `ttl`.

## Send a session hint

On `automatic` routes, keep the stable content first. To group related requests, add a session hint with the `X-Session-Id` header or `session_id` body field. The header wins when both are sent, and values over 256 characters are truncated, not rejected.

**`cURL`**

```bash title="cURL"
curl https://api-gateway.merge.dev/v1/responses \
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Session-Id: chat_9f2a7c" \
  -d '{
    "model": "openai/gpt-5.5",
    "input": [
      {"type": "message", "role": "system", "content": "You are a support agent for Acme Corp. Follow this policy: ...long stable document..."},
      {"type": "message", "role": "user", "content": "How do I reset a customer'\''s API key?"}
    ]
  }'
```

**`OpenAI SDK`**

```python title="OpenAI SDK"
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MERGE_GATEWAY_API_KEY"],
    base_url="https://api-gateway.merge.dev/v1/openai",
    default_headers={"X-Session-Id": "chat_9f2a7c"},
)

completion = client.chat.completions.create(
    model="openai/gpt-5.5",
    messages=[
        {"role": "system", "content": "You are a support agent for Acme Corp. Follow this policy: ...long stable document..."},
        {"role": "user", "content": "How do I reset a customer's API key?"},
    ],
)
```

On OpenAI and Azure, Gateway sends the hint as the vendor's cache key, scoped to your organization so another tenant's identical value never shares your cache. On `explicit` routes it signals reuse, so Gateway places a marker. [Cache-aware routing](/merge-gateway/routing/cache-aware-routing) uses it to keep a session on the vendor whose cache is warm. On `/v1/openai`, a body `prompt_cache_key` also works as the hint.

## Read cache usage

On `/v1/responses` and `/v1/anthropic`, `input_tokens` counts the full prompt, including cached reads and writes.

```json
{
  "usage": {
    "input_tokens": 5620,
    "output_tokens": 180,
    "total_tokens": 5800,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 5533
  }
}
```

A new prefix reports a write in `cache_creation_input_tokens`, which is `null` on automatic routes whose vendor doesn't count writes. Reuse reports reads in `cache_read_input_tokens`. Track that for hit rate, and `usage.cost` for the dollar effect.

Reads bill at `cache_read_per_million`. Writes bill at `cache_write_per_million` (1.25x the input rate on Anthropic and Claude on Bedrock), or the input rate on routes that don't price writes.

## Reference

| Surface                       | Markers                                                                                                        | Session hint                         | Cache usage fields                                                                                              |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------- |
| `/v1/responses`               | `cache_control` on blocks or messages, or top level                                                            | `session_id` or `X-Session-Id`       | `cache_read_input_tokens`, `cache_creation_input_tokens`                                                        |
| `/v1/anthropic`               | `cache_control` on content, system, and `tool_result` blocks, or top level                                     | `X-Session-Id`                       | `cache_read_input_tokens`, `cache_creation_input_tokens`                                                        |
| `/v1/openai/chat/completions` | `cache_control` on a message                                                                                   | `X-Session-Id` or `prompt_cache_key` | `prompt_tokens_details.cached_tokens`, `prompt_tokens_details.cache_write_tokens`. Omitted when neither occurs. |
| `/v1/openai/responses`        | None                                                                                                           | `X-Session-Id` or `prompt_cache_key` | `input_tokens_details.cached_tokens`, `input_tokens_details.cache_write_tokens`. Always present.                |
| `/v1/ai-sdk/chat/completions` | `providerOptions.anthropic.cacheControl` on a part or message (`merge-gateway-ai-sdk-provider` 0.3.0 or later) | `X-Session-Id`                       | `prompt_tokens_details.cached_tokens`, `prompt_tokens_details.cache_write_tokens`                               |
| `/v1/langchain`               | None                                                                                                           | `X-Session-Id`                       | `prompt_tokens_details.cached_tokens`, `prompt_tokens_details.cache_write_tokens`                               |

Streams carry the usage fields on the final chunk.

No cache tokens? Check the route's family (markers on `automatic` and `none` routes are stripped), that the marked prefix is byte-for-byte identical across requests, that repeats arrive before the entry expires, and that the prefix meets the vendor's minimum length. A routing policy or alias spanning families caches only on some requests, so pin `vendor` or keep its routes in one family when you depend on hits.

## Next steps

#### [Cache-aware routing](/merge-gateway/routing/cache-aware-routing)

Keep a session on the route whose cache is warm

#### [Cost governance and savings](/merge-gateway/cost/cost-governance-and-savings)

Set budgets and track savings from cached reads

#### [Model catalog](/merge-gateway/models)

Compare cache read and write prices per route