Logs

LLM calls

Every model request your employees' AI clients made, with the routing decision, the tokens, and the cost

Every request an employee’s AI client sends through Gateway lands here, one row per model request. Read it when you need to know which model answered, whether your routing policy did what you expected, what the request cost, and where the time went. It is the model-side counterpart to Tool call logs: one covers what the AI asked a model, the other covers what it did to your systems.

It is the LLM calls tab of Logs, alongside tool calls and API requests. Viewing it needs the View logs permission, and the tab appears once LLM call logs are enabled for your organization.

What a row shows

Timestamp, employee, routing policy, model, project, status, and tokens as input over output. When routing sent the request somewhere other than the model that was asked for, the model cell shows both: the model the client requested, the model that answered, and whether the change came from intelligent routing or from a fallback after the primary provider failed. That one cell is usually the answer to “why did this request go to that model.”

Open a row for the rest:

  • Model and vendor, the requested model, the model used, the vendor that served it, and the routing strategy
  • Project and endpoint, the scope the request ran under and the API surface it arrived on
  • Routing policy, which policy evaluated the request, with a fallback or intelligent-routing note when either fired
  • Total response time, split into LLM time and Gateway overhead, and below that the per-stage timings for the stages that ran: routing, prompt injection, DLP, compression, and whatever gateway work is left over
  • Estimated cost and service tier, plus any web search charge the provider billed
  • Input and output tokens, and caching, with the cache type, hit rate, and the read, write, and miss token split
  • Trace, span, and thread ids, when the client sent them and tracing is enabled for your organization

Cost here comes from the same pipeline that bills you, not from public price tables.

Filtering

The filter bar takes date, status, model, provider, and project, plus a free-text search across the row. Statuses are success, error, rate limited, and blocked. Filters compose, so “errors on one model in the last hour” is a single query.

Beside the bar, Show shadow requests brings in the mirrored candidate calls a model migration makes. They are hidden by default, because they are traffic your policy generated rather than traffic an employee asked for.

For per-employee and per-team spend, rather than one request at a time, read Budget and the Dashboard’s Overview tab, which charts spend over time and ranks models and tools by it.

Payload logging

Prompt and response bodies appear in the request detail only when payload logging is on, under Settings → Organization. Leave it off and you keep every row, every routing decision, and every cost figure, without storing the content of what employees sent a model.

That control governs what Merge stores. It is independent of zero data retention, which governs what the model vendor keeps.

Tracing a multi-step run

A client can group the requests behind one prompt into a single trace by sending one HTTP header, and the trace, span, and thread ids then appear in the request detail once tracing is enabled for your organization. Claude Code, Codex CLI, and OpenCode already send a session identifier that Gateway turns into a thread with no configuration at all. Setting the headers deliberately, and reading a run as a span tree, is a Gateway surface: see Tracing.

Next

Configure what employees can send a model in the first place with Model guardrails.