Reasoning

Reasoning lets supported models spend additional tokens planning before they answer. Use it for multi-step coding, math, analysis, tool-use planning, and other tasks where a slower request is worth a better final answer.

Gateway treats reasoning as a vendor-route capability. The same canonical model can expose different reasoning behavior depending on the vendor that serves the request, so check the exact route before sending reasoning controls.

How reasoning works

Gateway supports two reasoning patterns:

1. Gateway-controlled thinking uses the top-level thinking request field. Gateway translates it for routes that support that request style.
2. Provider-native reasoning controls use provider-specific fields such as reasoning_effort, when the selected route advertises that control.

In responses, Gateway normalizes separate provider reasoning into thinking content blocks.

1
{
2
  "type": "thinking",
3
  "thinking": "The model's reasoning text appears here.",
4
  "signature": "ErUBCkYIBRgCIkC8..."
5
}

Always render or log these blocks separately from final text. Some providers treat reasoning text as intermediate output, and some applications should hide it from end users.

signature is populated on Anthropic routes and is null on providers that do not sign reasoning (Gemini thought summaries, for example). It is not decorative: if you send the assistant turn back in a later request, the signature must come back with it. See Replay reasoning in multi-round tool loops.

Discover support

Use GET /v1/models and inspect the vendor route you plan to use.

$
curl "https://api-gateway.merge.dev/v1/models?vendor=bedrock" \
>
  -H "Authorization: Bearer YOUR_API_KEY"

A reasoning-capable route has supports_reasoning: true and a reasoning object.

1
{
2
  "model": "anthropic/claude-sonnet-4-6",
3
  "vendors": {
4
    "bedrock": {
5
      "capabilities": {
6
        "supports_reasoning": true,
7
        "reasoning": {
8
          "configurable": true,
9
          "disable_supported": true,
10
          "default_enabled": false,
11
          "controls": ["thinking.budget_tokens"],
12
          "output_style": "reasoning_content"
13
        }
14
      }
15
    }
16
  }
17
}

Check the route you will actually execute. If you use a routing policy, inspect every candidate route that policy can select.

Enable Gateway-controlled thinking

Use the thinking field when the route supports Gateway-controlled thinking. The budget_tokens value is required when type is "enabled" because providers that support explicit thinking need a budget.

$
curl https://api-gateway.merge.dev/v1/responses \
>
  -H "Authorization: Bearer YOUR_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "anthropic/claude-sonnet-4-6",
>
    "vendor": "bedrock",
>
    "max_tokens": 4096,
>
    "thinking": {
>
      "type": "enabled",
>
      "budget_tokens": 1024
>
    },
>
    "input": [
>
      {
>
        "type": "message",
>
        "role": "user",
>
        "content": "Design a migration plan for splitting a monolith into services. Include risks and sequencing."
>
      }
>
    ]
>
  }'

Pick a reasoning budget that leaves enough room for the final answer. If you set max_tokens: 4096 and budget_tokens: 1024, the provider still needs remaining output capacity for visible text.

Use provider-native controls

Some routes advertise provider-native controls instead of the top-level thinking field. For example, GPT-OSS routes may expose reasoning_effort.

Only send provider-native controls when /v1/models lists them under reasoning.controls.

1
{
2
  "model": "openai/gpt-oss-120b",
3
  "vendor": "bedrock",
4
  "reasoning_effort": "low",
5
  "input": [
6
    {
7
      "type": "message",
8
      "role": "user",
9
      "content": "Review this incident timeline and identify the most likely root cause."
10
    }
11
  ]
12
}

Provider-native controls are not interchangeable. A control that works on one vendor route can be ignored or stripped on another route for the same canonical model.

Disable reasoning

If a route advertises disable_supported: true, you can request thinking.type: "disabled".

1
{
2
  "model": "deepseek/deepseek-v4-flash",
3
  "vendor": "deepseek",
4
  "thinking": {
5
    "type": "disabled"
6
  },
7
  "input": [
8
    {
9
      "type": "message",
10
      "role": "user",
11
      "content": "Summarize this changelog in three bullets."
12
    }
13
  ]
14
}

Use disable controls for latency-sensitive requests, short classification tasks, or prompts where reasoning tokens are not worth the extra cost. If the route does not advertise disable_supported, Gateway may strip unsupported disable fields before sending the request upstream.

Read thinking blocks

Reasoning output appears as thinking content blocks before or alongside text blocks.

1
{
2
  "output": [
3
    {
4
      "type": "message",
5
      "role": "assistant",
6
      "content": [
7
        {
8
          "type": "thinking",
9
          "thinking": "I need to compare the constraints, then propose a sequence.",
10
          "signature": "ErUBCkYIBRgCIkC8..."
11
        },
12
        {
13
          "type": "text",
14
          "text": "Start by isolating the billing workflow..."
15
        }
16
      ]
17
    }
18
  ]
19
}

In code, branch on content.type.

1
for item in response.output:
2
    for block in item.content:
3
        if block.type == "thinking":
4
            save_internal_reasoning(block.thinking)
5
        elif block.type == "text":
6
            print(block.text)

Some providers may include reasoning-like text inside a normal text block. Treat the public /v1/models route metadata as the source of truth for whether Gateway expects separate reasoning blocks.

Replay reasoning in multi-round tool loops

A tool loop sends the model’s previous turn back on the next request. When reasoning is enabled on an Anthropic route, that turn contains a signed thinking block, and the signature has to travel with it.

Each surface exposes the signature differently, because the OpenAI wire format has no field for provider reasoning.

On the native and Anthropic surfaces, echo the assistant content blocks back verbatim and the signature travels with them. On the OpenAI-shaped surfaces, both fields are Gateway extensions, so your client has to carry them explicitly.

1
message = response.choices[0].message
2
3
# Echo BOTH fields back, or the reasoning is dropped on the next round.
4
messages.append({
5
    "role": "assistant",
6
    "content": message.content,
7
    "thinking": message.thinking,
8
    "thinking_signature": message.thinking_signature,
9
    "tool_calls": message.tool_calls,
10
})
11
messages.append({"role": "tool", "tool_call_id": call_id, "content": tool_output})

Redacted reasoning

Anthropic sometimes encrypts reasoning instead of returning it in the clear. Gateway preserves those blocks, and they carry the same replay requirement.

On /v1/anthropic the block keeps its provider shape:

1
{ "type": "redacted_thinking", "data": "EncryptedPayload==" }

On /v1/responses it arrives as a thinking block with empty text and the payload on redacted_data:

1
{ "type": "thinking", "thinking": "", "redacted_data": "EncryptedPayload==" }

There is nothing readable to show a user. Pass it back unchanged with the rest of the turn.

Reasoning with routing policies

Routing policies can choose among multiple vendors and models. If a request includes thinking, every route that might serve the request must support Gateway-controlled reasoning, or the request can fail during capability checks.

For deterministic behavior, pin model and vendor when you need a specific reasoning mode.

1
{
2
  "model": "anthropic/claude-sonnet-4-6",
3
  "vendor": "bedrock",
4
  "thinking": {
5
    "type": "enabled",
6
    "budget_tokens": 1024
7
  },
8
  "input": [
9
    {
10
      "type": "message",
11
      "role": "user",
12
      "content": "Find the safest rollback plan for this deployment."
13
    }
14
  ]
15
}

Use a routing policy when you only need a reasoning-capable route, not a specific provider behavior. In that case, configure the policy with models whose selected vendor routes all support the same reasoning control.

Streaming

Reasoning works with stream: true on supported routes. Gateway accumulates provider reasoning deltas and returns them as thinking content in the streamed response.

1
{
2
  "model": "anthropic/claude-sonnet-4-6",
3
  "vendor": "bedrock",
4
  "stream": true,
5
  "max_tokens": 4096,
6
  "thinking": {
7
    "type": "enabled",
8
    "budget_tokens": 1024
9
  },
10
  "input": [
11
    {
12
      "type": "message",
13
      "role": "user",
14
      "content": "Compare two database migration strategies."
15
    }
16
  ]
17
}

Streaming can delay visible text because the model may spend tokens on thinking before it emits final answer text.

Reasoning composes with streaming, structured output, and tool calling at the same time. None of those features is non-streaming-only.

How reasoning arrives depends on the surface:

• /v1/responses streams cumulative frames. Each chunk carries the whole thinking block accumulated so far, not a delta, so read the latest frame rather than concatenating. The signature appears once the block is complete.
• /v1/anthropic streams thinking_delta events, then a signature_delta carrying the signature, matching Anthropic’s own stream. Redacted blocks arrive as a complete content_block_start of type redacted_thinking, since there is nothing to stream incrementally.
• /v1/openai/chat/completions and /v1/ai-sdk/chat/completions stream reasoning text on delta.thinking and deliver delta.thinking_signature on the final chunk, because the signature only exists once the block is finished.

Common errors

Next steps