Observe

Stream logs to a SIEM

Pull Tool Call and API Request logs into Datadog, Splunk, Microsoft Sentinel, Elastic, and more.

Agent Handler exposes two cursor-paginated, pull-based log endpoints so your SIEM or observability platform can retrieve logs on a schedule, without you building a bespoke integration:

Full request/response schemas are in the API reference.

The log endpoints are an Enterprise feature. If they’re not enabled for your organization, the endpoints return 403. Contact your account team to enable them.

How the endpoints work

Both endpoints behave identically:

AspectDetail
AuthAuthorization: Bearer <PRODUCTION_KEY> — a production API key from Settings → API Keys.
OrderOldest-first, so you can tail the log forward without gaps.
PaginationCursor-based. Each response returns next_cursor; pass it back as ?cursor=<…> for the next page. Stop when has_more is false (then next_cursor is null). Persist next_cursor between polls to resume exactly where you left off.
Filterspage_size (1–1000, default 100), created_after / created_before (ISO 8601), plus tool_name / status / connector_id (tool calls) and method / response_status / connector_id / toolcall_id (API requests).
RetentionOnly the last 30 days are available. If a collector is down longer than 30 days, the oldest records age out and can’t be recovered — alert on collector staleness.
RedactionCredential values in headers (Authorization, API keys, cookies) and secret-named body fields are redacted before they leave the platform.

A poll loop looks like this:

$# First poll: start from the beginning of the retention window.
$curl -s "https://ah-api.merge.dev/api/v1/logs/tool-calls/?page_size=1000&created_after=2026-05-13T00:00:00Z" \
>  -H "Authorization: Bearer $PRODUCTION_KEY"
$# → {"results": [ ... ], "next_cursor": "eyJ0…", "has_more": true}
$
$# Next polls: pass the cursor from the previous response.
$curl -s "https://ah-api.merge.dev/api/v1/logs/tool-calls/?page_size=1000&cursor=eyJ0…" \
>  -H "Authorization: Bearer $PRODUCTION_KEY"
$# Repeat until "has_more": false, persist the last next_cursor, resume later from it.

Native pull vs. forwarder

Platforms fall into two camps:

  • Native pull — the platform has a built-in REST/API poller that follows cursor pagination for you. Just configure it. Microsoft Sentinel, Elastic, Cribl Stream, and Sumo Logic are in this camp.
  • Forwarder — the platform is push-only (it expects you to send logs to its intake), so you run a small scheduled job that walks the cursor loop above and posts batches to the platform. Datadog and Splunk are most reliably integrated this way.

Either way, the source side is identical: page through the endpoint with cursor, persist next_cursor, stop on has_more: false.

Platform setup

Forwarder. Datadog log collection is push-based — there’s no native input that follows cursor pagination against an arbitrary REST API. Run a scheduled job (Lambda, container, cron, or a Datadog Agent custom check) that pulls from the endpoint and POSTs to the Datadog Logs intake API.

  1. Create a Datadog API key (Organization Settings → API Keys). This is the DD-API-KEY header value — an API key, not an application key.
  2. Pick the intake host for your Datadog site (mismatched site is the most common failure):
    • US1 — https://http-intake.logs.datadoghq.com
    • US3 — https://http-intake.logs.us3.datadoghq.com
    • US5 — https://http-intake.logs.us5.datadoghq.com
    • EU1 — https://http-intake.logs.datadoghq.eu
    • AP1 — https://http-intake.logs.ap1.datadoghq.com
  3. Store the Agent Handler PRODUCTION_KEY and the Datadog API key in a secret manager, and persist the cursor in durable storage (DynamoDB, SSM, Redis, a DB row).
  4. Implement the pull→push loop and schedule it (e.g. every 1–5 minutes). Persist next_cursor after a page ships successfully so a crash re-ships at most one page.
1import requests
2
3cursor = state.load()  # None on first run
4ah = {"Authorization": f"Bearer {AH_PRODUCTION_KEY}"}
5dd_url = "https://http-intake.logs.datadoghq.com/api/v2/logs"   # match your site
6dd = {"Content-Type": "application/json", "DD-API-KEY": DD_API_KEY}
7
8while True:
9    params = {"page_size": 1000}
10    if cursor:
11        params["cursor"] = cursor
12    else:
13        params["created_after"] = "2026-05-13T00:00:00Z"  # within the 30-day window
14    body = requests.get(
15        "https://ah-api.merge.dev/api/v1/logs/tool-calls/", headers=ah, params=params, timeout=30,
16    ).json()
17
18    batch = [
19        {"ddsource": "agent-handler", "service": "ah-tool-calls",
20         "ddtags": "env:prod", "message": rec}
21        for rec in body["results"]
22    ]
23    if batch:
24        # Chunk conservatively (≤ ~500 logs / ≤ ~4 MB per POST).
25        requests.post(dd_url, headers=dd, json=batch, timeout=30).raise_for_status()
26
27    cursor = body["next_cursor"]
28    state.save(cursor)
29    if not body["has_more"]:
30        break  # next scheduled run resumes from the saved cursor

Gotchas

  • The intake auth header is DD-API-KEY with an API key — not Authorization: Bearer. The Bearer token is only for the Agent Handler side.
  • A single log is capped at ~1 MB (≤ 25 KB recommended). Chunk batches conservatively.
  • Datadog intake doesn’t dedup. Persist the cursor only after a successful POST, and index a stable record ID so you can dedup downstream if a retry re-sends a page.
  • Set each log’s timestamp (epoch ms) from the record’s created time, or logs land at receive-time and look mis-ordered.
  • Don’t use the Observability Pipelines HTTP/S Client source — it polls a fixed URL and can’t carry a cursor.

Docs: Send logs (HTTP intake) & sites · Log collection · Agent custom check send_log