API OverviewResponses

Create Response

Create an LLM response.

Supports both streaming and non-streaming modes via the stream parameter.

Authentication

AuthorizationBearer
Your production key sent as a bearer token.

Request

This endpoint expects an object.
inputlist of objectsRequired
Conversation history
modelstring or nullOptional

Model ID in format ‘provider/model-name’. Optional if a routing policy is configured.

customerstring or nullOptional

Embedded Routing: the Customer id to scope this request to, as a Merge UUID (the id returned when the customer was created, not your origin_id). Applies that customer’s routing policy, provider keys, budget, and usage attribution. A non-UUID value returns 422; a UUID that matches no customer returns 404 (customer_not_found). Requires embedded routing enabled for your organization.

routing_policy_idstring or nullOptional

Embedded Routing: route this request with a specific customer routing policy instead of the customer’s default. Requires customer.

toolslist of objects or nullOptional
Available tools
tool_choicestring or map from strings to any or nullOptional

Tool choice policy: ‘auto’, ‘none’, ‘required’, or {‘type’: ‘function’, ‘function’: {‘name’: ’…’}}

max_tokensinteger or nullOptional
Maximum tokens to generate
temperaturedouble or nullOptional0-2
Sampling temperature
top_pdouble or nullOptional0-1
Nucleus sampling parameter
stoplist of strings or nullOptional
Stop sequences
response_formatobject or nullOptional
Response format
streambooleanOptionalDefaults to false
Whether to stream the response
tagslist of objects or nullOptional

Tags to attach to this request for categorization (key-value pairs)

project_idstring or nullOptional

Optional project ID (UUID) to associate with this request

include_routing_metadatabooleanOptionalDefaults to false
Include detailed routing metadata in response
vendorstring or nullOptional

Restrict routing to a specific vendor (hosting platform)

vendorslist of strings or nullOptional
Ordered list of acceptable vendors. First available wins.
service_tierenum or nullOptional

Processing tier. ‘flex’ = discounted best-effort (slower, may be throttled); ‘priority’ = premium low-latency (not currently priced on any route, so it fails closed). Omit or ‘standard’ for normal processing. Allowed only on routes that price the requested tier, otherwise the request fails closed with 400.

service_tier_fallbackbooleanOptionalDefaults to false

If the provider throttles the requested tier (429/503), retry once at the standard tier, billed at standard rates. The response service_tier shows the tier that actually served.

Response

Successful Response
idstring
created_atdatetime
modelstring

The model that generated the response, in ‘provider/model-name’ format

outputlist of objects
usageobject
Token usage statistics.
object"response"Defaults to response
vendorstring or null
The execution vendor that served the request
provider_request_idstring or null
The upstream provider's request ID
routingobject or null

Routing metadata (only present when include_routing_metadata=true)

service_tierstring or null
The tier that actually served the request and was billed. Differs from the requested tier on a throttle fallback.

Errors

422
Unprocessable Entity Error