Reasoning

Reasoning lets supported models spend additional tokens planning before they answer. Use it for multi-step coding, math, analysis, tool-use planning, and other tasks where a slower request is worth a better final answer.

Gateway treats reasoning as a vendor-route capability. The same canonical model can expose different reasoning behavior depending on the vendor that serves the request, so check the exact route before sending reasoning controls.

How reasoning works

Gateway supports three reasoning patterns:

1. Gateway-controlled thinking uses the top-level thinking request field. Gateway translates it for routes that support that request style, and it is the portable spelling: write it once and it works across vendors.
2. Provider-native reasoning controls use provider-specific fields such as reasoning_effort, when the selected route advertises that control.
3. Adaptive thinking lets the model decide how much to think, steered by an effort level rather than a token budget. The newest Claude models take this shape and reject a manual budget, so Gateway rewrites a thinking block into it for them (see Adaptive thinking on Claude).

In responses, Gateway normalizes separate provider reasoning into thinking content blocks.

1
{
2
  "type": "thinking",
3
  "thinking": "The model's reasoning text appears here.",
4
  "signature": "ErUBCkYIBRgCIkC8..."
5
}

Always render or log these blocks separately from final text. Some providers treat reasoning text as intermediate output, and some applications should hide it from end users.

signature is populated on Anthropic routes and is null on providers that do not sign reasoning (Gemini thought summaries, for example). It is not decorative: if you send the assistant turn back in a later request, the signature must come back with it. See Replay reasoning in multi-round tool loops.

Discover support

Use GET /v1/models and inspect the vendor route you plan to use.

$
curl "https://api-gateway.merge.dev/v1/models?vendor=bedrock" \
>
  -H "Authorization: Bearer YOUR_API_KEY"

A reasoning-capable route has supports_reasoning: true and a reasoning object.

1
{
2
  "model": "anthropic/claude-sonnet-4-6",
3
  "vendors": {
4
    "bedrock": {
5
      "capabilities": {
6
        "supports_reasoning": true,
7
        "reasoning": {
8
          "configurable": true,
9
          "disable_supported": true,
10
          "default_enabled": false,
11
          "controls": ["thinking.budget_tokens"],
12
          "effort_values": ["low", "medium", "high"],
13
          "output_style": "reasoning_content"
14
        }
15
      }
16
    }
17
  }
18
}

Check the route you will actually execute. If you use a routing policy, inspect every candidate route that policy can select.

Enable Gateway-controlled thinking

Use the thinking field when the route supports Gateway-controlled thinking. The budget_tokens value is required when type is "enabled" because providers that support explicit thinking need a budget.

Keep sending this shape even on routes that no longer accept it upstream. Gateway rewrites it to whatever the resolved route wants, so thinking.budget_tokens stays the one spelling that works everywhere.

$
curl https://api-gateway.merge.dev/v1/responses \
>
  -H "Authorization: Bearer YOUR_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "anthropic/claude-sonnet-4-6",
>
    "vendor": "bedrock",
>
    "max_tokens": 4096,
>
    "thinking": {
>
      "type": "enabled",
>
      "budget_tokens": 1024
>
    },
>
    "input": [
>
      {
>
        "type": "message",
>
        "role": "user",
>
        "content": "Design a migration plan for splitting a monolith into services. Include risks and sequencing."
>
      }
>
    ]
>
  }'

Pick a reasoning budget that leaves enough room for the final answer. If you set max_tokens: 4096 and budget_tokens: 1024, the provider still needs remaining output capacity for visible text.

Use provider-native controls

Some routes advertise provider-native controls instead of the top-level thinking field. For example, GPT-OSS routes may expose reasoning_effort.

Only send provider-native controls when /v1/models lists them under reasoning.controls.

1
{
2
  "model": "openai/gpt-oss-120b",
3
  "vendor": "bedrock",
4
  "reasoning_effort": "low",
5
  "input": [
6
    {
7
      "type": "message",
8
      "role": "user",
9
      "content": "Review this incident timeline and identify the most likely root cause."
10
    }
11
  ]
12
}

Provider-native controls are not interchangeable. A control that works on one vendor route can be ignored or stripped on another route for the same canonical model.

Adaptive thinking on Claude

The newest Claude models (Opus 4.7 and 4.8, Sonnet 5, Fable 5, and Mythos 5) do not accept a manual thinking budget. They decide how long to think on their own, steered by an effort level, and reject thinking: {"type": "enabled", "budget_tokens": N} outright.

Send the thinking block anyway. Gateway rewrites it for those models into adaptive thinking with a summarized display, plus an effort level derived from the budget you asked for:

The summarized display is deliberate. These models omit thinking output unless asked, so without it the response would carry empty thinking blocks. A disabled directive is dropped for the same reason: reasoning is always on for this generation.

The adaptive shape is also accepted directly on the Anthropic-compatible surface, if you would rather write it than have it translated:

1
{
2
  "thinking": { "type": "adaptive", "display": "summarized" },
3
  "output_config": { "effort": "high" }
4
}

An explicit display is preserved rather than overwritten, and an effort level the route does not accept returns 400 unsupported_reasoning_effort.

Earlier Claude models on the direct anthropic vendor take an effort level too, translated back into a thinking budget. Gateway picks that budget against the request’s max_tokens and keeps a reserve for the answer, so high under a tight max_tokens runs at a lower level with a reasoning_effort_adjusted warning instead of failing Anthropic’s rule that max_tokens must exceed the thinking budget. Raise max_tokens when you want the level you asked for.

Effort levels per route

Effort levels are validated against the vendor route that serves the request, not the canonical model. Two vendors hosting the same model often accept different sets, and the level you can use is whatever reasoning.effort_values lists for the route you land on.

List the levels every vendor accepts for a model:

$
curl -s "https://api-gateway.merge.dev/v1/models?limit=500" \
>
  -H "Authorization: Bearer YOUR_API_KEY" \
>
  | jq -r '.data[] | select(.model == "moonshot/kimi-k3")
>
      | .vendors | to_entries[]
>
      | "\(.key): \(.value.capabilities.reasoning.effort_values // ["(no effort control)"] | join(", "))"'

moonshot: none, minimal, low, medium, high, xhigh, max
modal: none, minimal, low, medium, high, xhigh, max
togetherai: none, minimal, low, medium, high, xhigh, max
morph: low, medium, high, xhigh, max
makora: none, low, high, max
baseten: none, minimal, low, medium, high, xhigh
fireworks: (no effort control)

Same model, seven routes, five different answers. A request for max served by baseten runs at xhigh; served by fireworks it runs at the model’s default with no effort control at all.

Ceilings also move by model generation, which is why the route’s own effort_values is the answer rather than a rule of thumb. Three cases catch people out:

• minimal belongs to the original gpt-5 family only. The gpt-5, gpt-5-mini, and gpt-5-nano routes accept it; gpt-5.1 and later reject it, and so do the o-series routes.
• Claude on Bedrock stops at high. Bedrock’s Converse API drops the field adaptive thinking needs, so xhigh and max are unavailable there even though the direct anthropic vendor accepts both.
• max on gpt-5.6 and later runs at max. Gateway dispatches those requests down a path that carries the level through rather than clamping it. The clamp does still apply below a 2,048-token output budget, because that path fails outright on an exhausted budget where the alternative returns a partial answer.

Which route serves you depends on:

• A vendor or vendors pin in the request.
• Your own provider keys. With BYOK preference on, requests go to the vendors you hold keys for, even when another vendor accepts more levels.
• Routing policies, and how Gateway picks a vendor among the routes that survive your organization’s filters.

A request that names an effort level steers that choice: candidates narrow to routes that can honor the level before the cheapest one wins, as long as at least one candidate can. So on an unpinned request you usually land on a route that accepts the level you asked for, and only fall back to the closest level when no route accepts yours. An explicit vendor pin skips the narrowing, because the pin is the stronger instruction.

If your requested level is not in the resolved route’s list, the Gateway does not reject the request. It sends the closest level the route accepts and tells you what it did.

The reasoning_effort_adjusted warning

When the applied level differs from the one you asked for, the response carries a warning. detail.applied_effort is what actually ran.

1
{
2
  "warnings": [
3
    {
4
      "code": "reasoning_effort_adjusted",
5
      "message": "reasoning effort 'max' was adjusted to 'xhigh': the highest level the resolved route accepts",
6
      "detail": {
7
        "requested_effort": "max",
8
        "applied_effort": "xhigh",
9
        "model": "moonshot/kimi-k3",
10
        "vendor": "baseten"
11
      }
12
    }
13
  ]
14
}

Streaming responses carry the same warning on the final chunk. On /v1/openai/responses it appears in the same warnings array.

Watch for reasoning_exhausted alongside it. That one means the model spent its whole thinking budget and never wrote an answer, so the response is a 200 with finish_reason: length, empty content, and full billing. Raise the budget or lower the effort level and retry. See Warnings for the rest of the vocabulary.

Disable reasoning

If a route advertises disable_supported: true, you can request thinking.type: "disabled".

1
{
2
  "model": "deepseek/deepseek-v4-flash",
3
  "vendor": "deepseek",
4
  "thinking": {
5
    "type": "disabled"
6
  },
7
  "input": [
8
    {
9
      "type": "message",
10
      "role": "user",
11
      "content": "Summarize this changelog in three bullets."
12
    }
13
  ]
14
}

Use disable controls for latency-sensitive requests, short classification tasks, or prompts where reasoning tokens are not worth the extra cost. If the route does not advertise disable_supported, Gateway may strip unsupported disable fields before sending the request upstream.

Read thinking blocks

Reasoning output appears as thinking content blocks before or alongside text blocks.

1
{
2
  "output": [
3
    {
4
      "type": "message",
5
      "role": "assistant",
6
      "content": [
7
        {
8
          "type": "thinking",
9
          "thinking": "I need to compare the constraints, then propose a sequence.",
10
          "signature": "ErUBCkYIBRgCIkC8..."
11
        },
12
        {
13
          "type": "text",
14
          "text": "Start by isolating the billing workflow..."
15
        }
16
      ]
17
    }
18
  ]
19
}

In code, branch on content.type.

1
for item in response.output:
2
    for block in item.content:
3
        if block.type == "thinking":
4
            save_internal_reasoning(block.thinking)
5
        elif block.type == "text":
6
            print(block.text)

Some providers may include reasoning-like text inside a normal text block. Treat the public /v1/models route metadata as the source of truth for whether Gateway expects separate reasoning blocks.

Replay reasoning in multi-round tool loops

A tool loop sends the model’s previous turn back on the next request. When reasoning is enabled on an Anthropic route, that turn contains a signed thinking block, and the signature has to travel with it.

Each surface exposes the signature differently, because the OpenAI wire format has no field for provider reasoning.

On the native and Anthropic surfaces, echo the assistant content blocks back verbatim and the signature travels with them. On the OpenAI-shaped surfaces, both fields are Gateway extensions, so your client has to carry them explicitly.

1
message = response.choices[0].message
2
3
# Echo BOTH fields back, or the reasoning is dropped on the next round.
4
messages.append({
5
    "role": "assistant",
6
    "content": message.content,
7
    "thinking": message.thinking,
8
    "thinking_signature": message.thinking_signature,
9
    "tool_calls": message.tool_calls,
10
})
11
messages.append({"role": "tool", "tool_call_id": call_id, "content": tool_output})

Redacted reasoning

Anthropic sometimes encrypts reasoning instead of returning it in the clear. Gateway preserves those blocks, and they carry the same replay requirement.

On /v1/anthropic the block keeps its provider shape:

1
{ "type": "redacted_thinking", "data": "EncryptedPayload==" }

On /v1/responses it arrives as a thinking block with empty text and the payload on redacted_data:

1
{ "type": "thinking", "thinking": "", "redacted_data": "EncryptedPayload==" }

There is nothing readable to show a user. Pass it back unchanged with the rest of the turn.

Reasoning with routing policies

Routing policies can choose among multiple vendors and models. If a request includes thinking, every route that might serve the request must support Gateway-controlled reasoning, or the request can fail during capability checks.

For deterministic behavior, pin model and vendor when you need a specific reasoning mode.

1
{
2
  "model": "anthropic/claude-sonnet-4-6",
3
  "vendor": "bedrock",
4
  "thinking": {
5
    "type": "enabled",
6
    "budget_tokens": 1024
7
  },
8
  "input": [
9
    {
10
      "type": "message",
11
      "role": "user",
12
      "content": "Find the safest rollback plan for this deployment."
13
    }
14
  ]
15
}

Use a routing policy when you only need a reasoning-capable route, not a specific provider behavior. In that case, configure the policy with models whose selected vendor routes all support the same reasoning control.

Streaming

Reasoning works with stream: true on supported routes. Gateway accumulates provider reasoning deltas and returns them as thinking content in the streamed response.

1
{
2
  "model": "anthropic/claude-sonnet-4-6",
3
  "vendor": "bedrock",
4
  "stream": true,
5
  "max_tokens": 4096,
6
  "thinking": {
7
    "type": "enabled",
8
    "budget_tokens": 1024
9
  },
10
  "input": [
11
    {
12
      "type": "message",
13
      "role": "user",
14
      "content": "Compare two database migration strategies."
15
    }
16
  ]
17
}

Streaming can delay visible text because the model may spend tokens on thinking before it emits final answer text.

Reasoning composes with streaming, structured output, and tool calling at the same time. None of those features is non-streaming-only.

How reasoning arrives depends on the surface:

• /v1/responses streams cumulative frames. Each chunk carries the whole thinking block accumulated so far, not a delta, so read the latest frame rather than concatenating. The signature appears once the block is complete.
• /v1/anthropic streams thinking_delta events, then a signature_delta carrying the signature, matching Anthropic’s own stream. Redacted blocks arrive as a complete content_block_start of type redacted_thinking, since there is nothing to stream incrementally.
• /v1/openai/chat/completions and /v1/ai-sdk/chat/completions stream reasoning text on delta.thinking and deliver delta.thinking_signature on the final chunk, because the signature only exists once the block is finished.

Common errors

Next steps