> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.merge.dev/merge-gateway/evals-api/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.merge.dev/_mcp/server.

# Evals API

> Reference for the Evals API: suite and case sync, gateway and external runs, per-case results, and recurring schedules.

The Evals API runs eval suites from CI: sync cases from your repo, run them against a catalog model or grade your pipeline's own outputs, and fail the build on a red suite. See the [Evals guide](/merge-gateway/evals/overview) for concepts and graders, and [running evals from CI](/merge-gateway/evals/evals-api) for a walkthrough.

## Base URL and authentication

```text
https://api-gateway.merge.dev
```

Authenticate with the `mg_` API key you send `/v1/responses` with. The organization comes from the key; customer API keys are rejected.

```http
Authorization: Bearer mg_...
```

## Reference

| Endpoints                                                           | What they do                                                                                                                                                                                                        |
| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET`, `PUT /v1/evals/suites`, `GET /v1/evals/suites/{suite_id}`    | List suites, or create or update one by name (`201` created, `200` updated)                                                                                                                                         |
| `PUT /v1/evals/suites/{suite_id}/cases`                             | Sync cases by `external_id`. With `prune` (default `true`), synced cases missing from the payload are deleted. Dashboard-authored cases are never touched. Returns created, updated, deleted, and unchanged counts. |
| `GET`, `POST /v1/evals/suites/{suite_id}/runs`                      | List or trigger runs                                                                                                                                                                                                |
| `GET /v1/evals/runs/{run_id}`, `GET .../results`, `POST .../cancel` | Poll a run, read per-case results, or cancel a pending or running run                                                                                                                                               |
| `GET`, `PUT`, `DELETE /v1/evals/suites/{suite_id}/schedule`         | A recurring run every 6, 12, 24, or 168 hours. Pair it with [alerts](/merge-gateway/evals/running-evals#alerts).                                                                                                    |

A trigger takes `target_model`, `mode` (`gateway`, or `external` with your own `outputs`), `trials` (1 to 5, and a case passes only if every trial passes), `metadata` (up to 16 string pairs), `idempotency_key`, and `request_overrides` for the system prompt and params.

**The CI contract.** A trigger returns `201` immediately and runs asynchronously. Poll `GET /v1/evals/runs/{run_id}` until `status` is `completed`, `failed`, or `cancelled`, then branch on `passed`. Retrying a trigger with the same `idempotency_key` returns the original run with `200`. Each organization can trigger 10 runs per minute (`429` beyond that), and each suite can have 3 runs in progress (`409 run_in_flight`). See [Errors](/merge-gateway/errors#management-and-platform-apis) for other codes.

## Next steps

#### [Run evaluations](/merge-gateway/evals/running-evals)

Compare model responses against your cases

#### [Model migrations](/merge-gateway/evals/model-migrations)

Measure a proposed model change

> Build once. Connect to every API, tool, and LLM. Ship products and agents to production without owning the integration stack.


## API Docs

- Evals API > Runs [Get run](https://docs.merge.dev/merge-gateway/evals-api/runs/get-run.md)
- Evals API > Runs [Cancel run](https://docs.merge.dev/merge-gateway/evals-api/runs/cancel-run.md)
- Evals API > Runs [List run results](https://docs.merge.dev/merge-gateway/evals-api/runs/list-run-results.md)
- Evals API > Runs [List runs](https://docs.merge.dev/merge-gateway/evals-api/runs/list-runs.md)
- Evals API > Runs [Trigger run](https://docs.merge.dev/merge-gateway/evals-api/runs/trigger-run.md)
- Evals API > Suites [List suites](https://docs.merge.dev/merge-gateway/evals-api/suites/list-suites.md)
- Evals API > Suites [Create or update suite](https://docs.merge.dev/merge-gateway/evals-api/suites/create-or-update-suite.md)
- Evals API > Suites [Get suite](https://docs.merge.dev/merge-gateway/evals-api/suites/get-suite.md)
- Evals API > Cases [Sync cases](https://docs.merge.dev/merge-gateway/evals-api/cases/sync-cases.md)
- Evals API > Schedules [Get schedule](https://docs.merge.dev/merge-gateway/evals-api/schedules/get-schedule.md)
- Evals API > Schedules [Create or update schedule](https://docs.merge.dev/merge-gateway/evals-api/schedules/create-or-update-schedule.md)
- Evals API > Schedules [Delete schedule](https://docs.merge.dev/merge-gateway/evals-api/schedules/delete-schedule.md)

## OpenAPI Specification

The raw OpenAPI 3.1 specification for this API is available at:
- [OpenAPI JSON](https://docs.merge.dev/openapi.json)
- [OpenAPI YAML](https://docs.merge.dev/openapi.yaml)