Structured outputs

Structured outputs constrain a model’s response to JSON — either any JSON object (json_object) or JSON matching a schema you provide (json_schema). Gateway translates your request into each provider’s native mechanism (OpenAI structured outputs, Gemini responseSchema, Anthropic output_format), and fails loudly with a clear error when a route can’t honor it. It never degrades silently to best-effort text.

Two modes

Request shape

Pass response_format on /v1/responses (or the OpenAI-compatible surfaces):

1
{
2
  "type": "json_schema",
3
  "json_schema": {
4
    "name": "person",
5
    "schema": {
6
      "type": "object",
7
      "properties": {
8
        "name": { "type": "string" },
9
        "age": { "type": "integer" }
10
      },
11
      "required": ["name", "age"],
12
      "additionalProperties": false
13
    },
14
    "strict": true
15
  }
16
}

Your schema passes through to the provider as you wrote it — nested objects, $ref/$defs, and anyOf included. Gateway performs no schema rewriting beyond per-vendor JSON Schema dialect fixes (for example, rewriting draft-07 definitions into 2020-12 $defs for vendors that require it).

The strict flag

strict is tri-state and caller-controlled:

• Omitted — the provider’s own default applies. Gateway never injects a value.
• true — providers with a strict mode (OpenAI-style) guarantee exact schema conformance, but impose constraints: every property must be listed in required, additionalProperties must be false, and some keywords are unsupported. Schemas with optional fields or open objects will be rejected by the provider with a descriptive 400.
• false — the schema guides generation without strict-mode constraints. Use this for schemas with optional fields, unions, or other shapes strict mode rejects.

Providers without a strict knob (Anthropic, Gemini) constrain output by grammar from your schema regardless; the flag is ignored there.

Examples

$
curl https://api-gateway.merge.dev/v1/responses \
>
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "openai/gpt-5.1",
>
    "input": "Invent a person.",
>
    "response_format": {
>
      "type": "json_schema",
>
      "json_schema": {
>
        "name": "person",
>
        "schema": {
>
          "type": "object",
>
          "properties": {"name": {"type": "string"}, "age": {"type": "integer"}},
>
          "required": ["name", "age"],
>
          "additionalProperties": false
>
        },
>
        "strict": true
>
      }
>
    }
>
  }'

Discovering capable models

Structured-output support is a per-route (model + vendor) capability. Check supports_structured_outputs in the /v1/models response:

1
"vendors": {
2
  "openai": {
3
    "capabilities": {
4
      "supports_structured_outputs": true
5
    }
6
  }
7
}

Routing behavior

Capability checks apply to the exact execution route before the request is sent upstream:

• Routing policies filter out candidates that can’t honor response_format before selection. If no capable candidate remains, the request fails with 400 unsupported_params naming the requirement.
• Direct model requests to a route without support fail with 400 unsupported_params identifying the model, vendor, and requested response format type.
• Fallbacks re-check capabilities on the fallback route — a request never silently lands on a route that ignores your schema.

json_object rides the same capability gate as json_schema: a route that can’t do schema-constrained output can’t guarantee JSON-object output either.

See Errors for the unsupported_params and invalid_response_format error shapes.

Streaming

Structured output works with stream: true — the JSON arrives as ordinary text deltas that you can parse incrementally. One caveat: if a mid-stream provider failure triggers a fallback, the stream emits a fallback_restart event and restarts generation from scratch. Incremental JSON parsers must discard everything accumulated before that event.

Malformed requests

A response_format that names json_schema but omits the schema — or uses an unknown type — is rejected with 400 invalid_response_format rather than being silently ignored.