Merge API Documentation

{
  "id": "string",
  "suite_id": "string",
  "target_model": "string",
  "status": "string",
  "trigger_source": "string",
  "suite_config_snapshot": {},
  "pass_count": 1,
  "fail_count": 1,
  "error_count": 1,
  "created_at": "2024-01-15T09:30:00Z",
  "updated_at": "2024-01-15T09:30:00Z",
  "execution_mode": "gateway",
  "metadata": {},
  "migration_id": "string",
  "triggered_by": "string",
  "passed": true,
  "total_cost_usd": 1.1,
  "judge_cost_usd": 1.1,
  "estimated_cost_usd": 1.1,
  "trace_id": "string",
  "error": "string",
  "trials": 1,
  "started_at": "2024-01-15T09:30:00Z",
  "completed_at": "2024-01-15T09:30:00Z",
  "total_cases": 0,
  "completed_cases": 0,
  "pass_rate": 1.1,
  "ci_low": 1.1,
  "ci_high": 1.1
}

Trigger a run of the suite (single-call).

201 with the created run; execution is asynchronous — poll GET /v1/evals/runs/{id} until a terminal status, then branch on passed. An idempotency_key that already exists for the suite returns the original run with 200 instead of creating a duplicate. 409 run_in_flight once the suite’s concurrent-run cap is reached.

Trigger a run of the suite (single-call). 201 with the created run; execution is asynchronous — poll ``GET /v1/evals/runs/{id}`` until a terminal status, then branch on ``passed``. An ``idempotency_key`` that already exists for the suite returns the original run with 200 instead of creating a duplicate. 409 ``run_in_flight`` once the suite's concurrent-run cap is reached.

Authentication

AuthorizationBearer

Your gateway API key (mg_ prefix), the same key used for /v1/responses

Path parameters

suite_idstringRequiredformat: "uuid"

Request

This endpoint expects an object.

target_modelstringRequired1-100 characters

trialsintegerOptional1-5Defaults to 1

modeenumOptionalDefaults to gateway

outputslist of objects or nullOptional

metadatamap from strings to strings or nullOptional

idempotency_keystring or nullOptional1-128 characters

request_overridesobject or nullOptional

Run-level request configuration applied to every case.

system is prepended as the first message of each case (the mechanism behind “eval prompt v14 against the same dataset”); params win over per-case request_params and pass the same allowlist.

Response

Successful Response

idstringformat: "uuid"

suite_idstringformat: "uuid"

target_modelstring

statusstring

trigger_sourcestring

suite_config_snapshotmap from strings to any

pass_countinteger

fail_countinteger

error_countinteger

created_atdatetime

updated_atdatetime

execution_modestringOptionalDefaults to gateway

metadatamap from strings to any or nullOptional

migration_idstring or nullOptionalformat: "uuid"

triggered_bystring or nullOptional

passedboolean or nullOptional

total_cost_usddouble or nullOptional

judge_cost_usddouble or nullOptional

estimated_cost_usddouble or nullOptional

trace_idstring or nullOptional

errorstring or nullOptional

trialsintegerOptionalDefaults to 1

started_atdatetime or nullOptional

completed_atdatetime or nullOptional

total_casesintegerOptionalDefaults to 0

completed_casesintegerOptionalDefaults to 0

pass_ratedouble or nullOptional

ci_lowdouble or nullOptional

ci_highdouble or nullOptional

Errors

422

Unprocessable Entity Error