Multi-turn conversations

A conversation is multi-turn when the model needs to know what was already said. Gateway gives you two ways to do that, and the first one needs nothing turned on.

Send the conversation in input

Include the prior turns in input on every request. The model sees the whole exchange, and Gateway keeps nothing.

$
curl https://api-gateway.merge.dev/v1/responses \
>
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "openai/gpt-4o",
>
    "input": [
>
      {"type": "message", "role": "user", "content": "Say hello in one word."},
>
      {"type": "message", "role": "assistant", "content": "Hello"},
>
      {"type": "message", "role": "user", "content": "What word did you just say?"}
>
    ]
>
  }'

This works on every model and every route, it has no setup, and your application keeps its own transcript. It is the right default.

Let Gateway hold the conversation

If you would rather not carry the transcript, Gateway can store a turn and let you continue from it by id.

This is off by default and takes two things: your organization must have the response store enabled, and each request you want to be resumable must send store: true.

Turn 1: ask for the turn to be stored

$
curl https://api-gateway.merge.dev/v1/responses \
>
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "openai/gpt-4o",
>
    "input": [{"type": "message", "role": "user", "content": "Say hello in one word."}],
>
    "store": true
>
  }'

The response carries an id:

1
{ "id": "resp_9754aec2adc6059b", "output": [ ... ] }

Turn 2: continue from it

$
curl https://api-gateway.merge.dev/v1/responses \
>
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "openai/gpt-4o",
>
    "previous_response_id": "resp_9754aec2adc6059b",
>
    "input": [{"type": "message", "role": "user", "content": "What word did you just say?"}],
>
    "store": true
>
  }'

Gateway replays the stored conversation ahead of your new input. Send store: true again on each turn you want to be resumable, including this one.

What to expect

A stored response expires an hour after its last use. The clock resets every time you resume, so an active conversation never expires however long it runs. An idle one does.

Treat the store as a convenience, not a system of record. It is held in memory and is best-effort: a response can expire, be evicted, or be lost to a failover. Keep your own transcript, or be ready to rebuild it. If a chain is gone, Gateway tells you plainly:

1
{
2
  "error": {
3
    "type": "invalid_request_error",
4
    "code": "previous_response_not_found",
5
    "param": "previous_response_id",
6
    "message": "previous_response_id 'resp_...' was not found. ..."
7
  }
8
}

The fix is always the same: send the full conversation in input, as in the first section.

It does not reduce your token cost. Gateway sends the full conversation to the provider either way. What you save is carrying the transcript in your own application, not tokens.

Reasoning is not replayed. Thinking and reasoning blocks are dropped when a stored turn is replayed, because a reasoning signature is only valid for the vendor that produced it and your next turn may be routed elsewhere. The visible conversation is preserved.

Retrieving and deleting a stored response

$
# Read a stored conversation
$
curl https://api-gateway.merge.dev/v1/responses/resp_9754aec2adc6059b \
>
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY"
$
$
# Delete it, ending the chain
$
curl -X DELETE https://api-gateway.merge.dev/v1/responses/resp_9754aec2adc6059b \
>
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY"

Both are scoped to your organization, and to your project or customer when your API key is scoped to one. An id belonging to another scope reads as not found.

Which approach to use

Use the response store when carrying the transcript is the awkward part, such as a long-running agent loop where turns arrive seconds apart. Use input everywhere else, and keep it as your fallback either way.