Decisions

Decision models answer typed questions about a piece of state instead of generating text. You give them an email, a ticket, a log line, or any JSON object, plus a set of questions you name, and each answer comes back as a choice, a score, or a yes/no probability with its own confidence. Reach for them when you want to route, triage, score, or gate something and a free-text response would only have to be parsed back into a value.

They are considerably cheaper and faster than asking a chat model for JSON, because they emit a handful of structured tokens rather than a completion.

Not a chat model

Decision models have no chat surface. Sending one to /v1/responses returns 400 unsupported_endpoint_for_model pointing you here, and they never appear as candidates in a routing policy. There is nothing to stream, and no sampling parameters to set: the model returns a probability distribution, not a generation.

Request shape

Send state, the content being judged, and questions, a map of questions you name:

{
  "model": "typesafe/jev-1.13",
  "state": "Our invoice shows $4,200 but the contract says $3,800. Third month running. Fix this today or we cancel.",
  "questions": {
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this message?",
      "criteria": ["not urgent", "somewhat urgent", "urgent", "critical"]
    },
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "billing": "Invoice and payment issues",
        "support": "Product and technical issues",
        "sales": "Upgrades and renewals"
      }
    },
    "churn_risk": {
      "type": "noul",
      "instructions": "Is this customer at risk of churning?"
    }
  }
}

state takes a string, an object, or an array, so a chat transcript or an application state blob can go in as-is without being flattened into prose. The keys you choose in questions are the keys you get back in answers, so name them for the code that reads them.

Question types

The descriptions in a choice question’s criteria are how the model learns what each option means, so they carry real weight. A question that omits a required criteria, or uses a type outside these three, is rejected with a 422 before any provider call.

Response shape

{
  "object": "decision",
  "model": "jev-1.13.0",
  "vendor": "typesafe",
  "answers": {
    "urgency": {
      "type": "score",
      "score": 2.84,
      "confidence": 0.84,
      "legend": {"0": "not urgent", "1": "somewhat urgent", "2": "urgent", "3": "critical"},
      "probabilities": {"0": 0.0, "1": 0.0, "2": 0.16, "3": 0.84}
    },
    "team": {
      "type": "choice",
      "choice": "billing",
      "confidence": 1.0,
      "probabilities": {"billing": 1.0, "support": 0.0, "sales": 0.0}
    },
    "churn_risk": {"type": "noul", "noul": 0.95}
  },
  "usage": {"input_tokens": 403, "output_tokens": 70, "total_tokens": 473, "cost": 1.6926e-05}
}

Three things to note. score is a probability-weighted position on your scale rather than a bucket index, which is why you get 2.84 instead of 3. legend echoes your levels back so you can label the value without re-deriving it. And noul carries no confidence, because the probability is the answer.

model is the concrete version that served the request, which may differ from the id you sent if you used an alias.

Confidence means different things by type

A four-option choice whose top option sits at p_max = 0.46 reports confidence: 0.27, because picking at random would already land 0.25. That makes choice confidence a direct read on “how much better than guessing is this”, which is usually what you want when deciding whether to auto-route or escalate to a person.

The consequence is that one threshold does not fit both. Gating at confidence > 0.5 asks a score question for a peak of 0.5, but asks a four-option choice question for a peak of about 0.625. Set thresholds per question type, and calibrate them against your own data rather than porting a number across.

If you want the raw peak instead, read probabilities directly. It is always present and always sums to 1.

Batch your questions

Input tokens are billed once per call no matter how many questions you attach, and there is a fixed overhead of roughly 250 tokens per request. One question against a short state costs about 274 input tokens; three questions against a longer one costs about 403. Ten separate calls cost roughly ten times what one call with ten questions costs.

Ask everything you want to know about a piece of state in a single request. The limit is 100 questions per call.

Examples

curl https://api-gateway.merge.dev/v1/decisions \
  -H "Authorization: Bearer $MERGE_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "typesafe/jev-1.13",
    "state": "Our invoice shows $4,200 but the contract says $3,800.",
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Invoice and payment issues",
          "support": "Product and technical issues"
        }
      }
    }
  }'

Pricing

Decision models are billed on input tokens only. Output tokens are reported in usage so the figure is auditable, but they cost nothing. Each response carries its own usage.cost in USD, computed from the route that served it.

Limits and errors

Exceeding the token budget returns 400 max_tokens_exceeded. Unknown request fields are rejected rather than ignored, so a parameter that a chat endpoint would accept, such as stream, returns a 422 instead of being silently dropped.

You can pin execution with vendor, and attribute usage to an end customer with customer, exactly as on other Gateway endpoints. Model aliases are not supported here: send a model id.

Available models

GET /v1/models lists decision models alongside everything else, with output: ["decision"] in their capabilities. Remember to pass limit when you search the catalog, since the endpoint pages at 50 by default.

Pin a specific version rather than a floating alias. Vendors ship new versions under the same alias, and a decision model’s answers are something you calibrate thresholds against, so a silent version change moves your thresholds underneath you.

Next steps