Telemetry export
Telemetry export forwards the telemetry Gateway already records to an observability backend you own. Use it when your team works in Datadog or Grafana and wants LLM traffic in the same place as the rest of your system.
There is nothing to install and no code to change. Your traffic already flows through Gateway, so configuring a destination is enough. Gateway sends OpenTelemetry over HTTP/protobuf using the GenAI semantic conventions, so what arrives is standard OTLP.
Destinations
Choose Datadog if you have an LLM Observability seat. Prompts and completions render as conversations there, and spans are classified into the LLM dashboards.
Choose Datadog (APM) if you have APM only. Spans arrive as ordinary APM spans and appear in the same service map and traces as the rest of your application.
The Grafana Cloud destination sends traces to Tempo, where they are searchable and queryable with TraceQL. Grafana’s Agent/AI Observability app is fed by a separate first-party API, so it stays empty. Metrics are traces-only for this destination, explained under Metrics.
Custom OTLP endpoint is available on Enterprise plans. It accepts any OTLP/HTTP receiver, including an OpenTelemetry Collector, Honeycomb, New Relic, SigNoz, Langfuse, or self-hosted Tempo or Jaeger. Gateway stores the base endpoint and appends the signal path, following the OTEL_EXPORTER_OTLP_ENDPOINT convention, so one destination covers every signal you enable.
Custom endpoints authenticate with Authorization: Bearer <token>. Datadog and Grafana Cloud use their own schemes, a dd-api-key header and Basic auth, which their dedicated destinations handle for you. A self-hosted Grafana stack behind a Collector or Alloy is a custom endpoint like any other.
What a span contains
Each exported request becomes one span following the OTel GenAI semantic conventions:
merge.cost.usd is paid to a different party depending on merge.cost.basis. Under merge_billed it is what you pay Merge. Under provider_list_price, which applies to BYOK traffic, it is the provider’s list price for those tokens, paid to your provider. Summing both gives total LLM spend. Split on the basis if you want spend with Merge alone.
A request Gateway could not price carries no cost attribute at all. A missing cost means unknown, and a 0 means the request really cost nothing.
Prompts and completions
New destinations start with privacy mode on, which strips gen_ai.input.messages and gen_ai.output.messages before sending. Models, tokens, latency, and cost are unaffected.
Turning privacy mode off sends prompt content to that destination. It is a separate setting from Gateway’s own request logging, so a destination can receive prompts while Merge stores none. Gateway reads it again on every flush, so turning it back on stops content immediately, including for requests already queued.
Metrics
Metrics are off by default on every destination, including existing ones, and are enabled per destination. When enabled, Gateway sends these alongside the traces:
Metrics do not reach Grafana Cloud. Gateway aggregates each flush into OTLP delta points, which is the temporality Datadog’s metrics intake requires and the one Grafana Cloud’s OTLP gateway rejects, with invalid temporality and type combination. Traces are unaffected and keep arriving in Tempo.
Self-hosted Mimir accepts delta points once -distributor.otel-native-delta-ingestion is on, so metrics work there through a custom OTLP endpoint. That option is experimental in Mimir. Samples land as raw values that rate() and increase() cannot read, so query them with sum_over_time instead.
On Grafana Cloud, Tempo’s metrics generator derives span metrics from the traces Gateway already sends.
Datadog bills custom metrics separately from spans, so enabling metrics adds cost on your side. The same data is already on the spans. Enable metrics when you want long retention, cheap queries over wide time ranges, or alert rules, since metric series outlive indexed traces.
Two things to know when querying these in Datadog. Attribute keys keep their dots, so group by {merge.cost.basis} rather than {merge_cost_basis}. The three histograms arrive as distributions, and percentile queries such as p95: need percentiles enabled on the metric in Metrics > Summary first.
If you run an OpenTelemetry Collector, its spanmetrics connector derives equivalent metrics from the traces Gateway already sends, which may cover this for you.
Sampling
A destination can sample, sending a fraction of requests. Errors are always sent regardless of the rate. On a sampled destination with metrics enabled, points are scaled to estimate the full population, and the rate used is recorded on each series.
Set up a destination
- In the dashboard, open Settings → Telemetry export and create a destination
- Choose the destination type and enter the site, zone, or endpoint it needs
- Add the credential: a Datadog API key, a Grafana Cloud instance ID and token, or a bearer token for a custom endpoint
- Leave privacy mode on unless you want prompts and completions sent
- Enable metrics if you want them, and set a sample rate to send a fraction of requests
Delivery starts on the next request. Spans usually appear within a few minutes. Datadog’s LLM Observability intake adds a further 3–5 minute delay.
Delivery behavior
Gateway batches spans per destination and retries transient failures with backoff. A rejected credential or a malformed endpoint fails permanently, so Gateway stops after the first attempt. Repeated failures suspend delivery to that destination until it recovers.
Telemetry export runs outside the request path. A destination that is failing, suspended, or misconfigured leaves your LLM calls untouched.
Custom endpoints must use https. Gateway refuses redirects, and refuses endpoints that resolve to private, loopback, link-local, or cloud metadata addresses.