Structured outputs

Structured outputs constrain a model’s response to JSON — either any JSON object (json_object) or JSON matching a schema you provide (json_schema). Gateway translates your request into each provider’s native mechanism (OpenAI structured outputs, Gemini responseSchema, Anthropic output_format), and fails loudly with a clear error when a route can’t honor it. It never degrades silently to best-effort text.

Two modes

Request shape

Pass response_format on /v1/responses (or the OpenAI-compatible surfaces):

{
  "type": "json_schema",
  "json_schema": {
    "name": "person",
    "schema": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "age": { "type": "integer" }
      },
      "required": ["name", "age"],
      "additionalProperties": false
    },
    "strict": true
  }
}

Your schema passes through to the provider as you wrote it — nested objects, $ref/$defs, and anyOf included. Gateway performs no schema rewriting beyond per-vendor JSON Schema dialect fixes (for example, rewriting draft-07 definitions into 2020-12 $defs for vendors that require it).

The strict flag

strict is tri-state and caller-controlled:

• Omitted — the provider’s own default applies. Gateway never injects a value.
• true — providers with a strict mode (OpenAI-style) guarantee exact schema conformance, but impose constraints: every property must be listed in required, additionalProperties must be false, and some keywords are unsupported. Schemas with optional fields or open objects will be rejected by the provider with a descriptive 400.
• false — the schema guides generation without strict-mode constraints. Use this for schemas with optional fields, unions, or other shapes strict mode rejects.

Providers without a strict knob (Anthropic, Gemini) constrain output by grammar from your schema regardless; the flag is ignored there.

Examples

curl https://api-gateway.merge.dev/v1/responses \
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.1",
    "input": "Invent a person.",
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "schema": {
          "type": "object",
          "properties": {"name": {"type": "string"}, "age": {"type": "integer"}},
          "required": ["name", "age"],
          "additionalProperties": false
        },
        "strict": true
      }
    }
  }'

Discovering capable models

Structured-output support is a per-route (model + vendor) capability. Check supports_structured_outputs in the /v1/models response:

"vendors": {
  "openai": {
    "capabilities": {
      "supports_structured_outputs": true
    }
  }
}

Routing behavior

Capability checks apply to the exact execution route before the request is sent upstream:

• Routing policies filter out candidates that can’t honor response_format before selection. If no capable candidate remains, the request fails with 400 unsupported_params naming the requirement.
• Direct model requests to a route without support fail with 400 unsupported_params identifying the model, vendor, and requested response format type.
• Fallbacks re-check capabilities on the fallback route, so a request never silently lands on a route that ignores your schema.
• A completion that violates a strict schema counts as a failed attempt. With strict: true, a 200 whose body does not satisfy your schema is discarded and the request moves to the next candidate model under a Priority or Intelligent policy. If every candidate violates it, the last violating body is served rather than an error, so validate the response you get. Streaming responses are never classified this way, and a turn that returns tool calls or stops on the output-token limit is exempt.

Two things follow from that last rule. Provider strict modes are constrained decoding, so they enforce structure but not value keywords like minimum and maximum, and how strictly they enforce varies by vendor. Failing over on a violation is what turns your schema contract into something honored end to end rather than per vendor. And each discarded attempt is billed at the route that ran it, so a strict schema that most of your candidates cannot satisfy costs more than one call.

json_object rides the same capability gate as json_schema: a route that can’t do schema-constrained output can’t guarantee JSON-object output either. Bedrock routes are an exception in the other direction. They serve json_schema fine but reject json_object with 400 unsupported_params pointing you at json_schema, because Bedrock has no native JSON mode and grammar-constraining against a permissive object schema drives every model tested into returning a literal {}.

See Errors for the unsupported_params and invalid_response_format error shapes.

Streaming

Structured output works with stream: true — the JSON arrives as ordinary text deltas that you can parse incrementally. One caveat: if a mid-stream provider failure triggers a fallback, the stream emits a fallback_restart event and restarts generation from scratch. Incremental JSON parsers must discard everything accumulated before that event.

Malformed requests

A response_format that names json_schema but omits the schema — or uses an unknown type — is rejected with 400 invalid_response_format rather than being silently ignored.