Every request, fully visible.
Logs, alerts, and callbacks for every LLM call. See what happened, get notified when things break, pipe data to your existing tools. Cost is the provider’s number — never ours, never reconstructed.
Latest request
- Default log retention
- 90 days
- Cost source
- x-nemo-request-cost
- Latency percentiles
- p50 · p95 · p99
- PII masking
- Optional
Financial records retained indefinitely
Provider header — never reconstructed
Per model, per key, per org
Per-org policy: zero / meta / full / redacted
Successes, rate-limits, cache hits, guardrail blocks: one stream.
Searchable, filterable, expandable per row. Logged async: zero latency added to the request path.
Provider → Backend → Callbacks (async)
NemoRouter captures the provider cost directly: no extra hop where the cost header can be lost or rewritten. Logs are recorded, callbacks fan out async with a retry queue.
Async log path
Provider
Vertex / Anthropic / OpenAI
Returns x-nemo-request-cost header — the source of truth.
Nemo routing core
Routing engine
Computes cost, attaches metrics, emits log event.
Nemo Backend
NemoRouter
Records request_log, fans out to enabled callbacks async.
Callbacks
Langfuse · Datadog · S3 · Slack
Fire-and-forget delivery with retry queue.
Provider cost header is the source of truth
Cost integrity
One number, four surfaces, zero drift
We never compute cost ourselves. The same x-nemo-request-cost value lands in the request_log row, the credit ledger, every active callback, and the analytics rollup: one upstream value, four downstream consumers, no recomputation. Missing-header rows are flagged for reconcile, never silently zeroed.
- Source: provider response header (x-nemo-request-cost)
- Sinks: request_log, credit ledger, callbacks, analytics
- Daily ledger parity check: sum(transactions) == balance
- Zero-cost settlements are the canonical revenue-leak signal
Cost-header integrity
Full reference: callbacks, alerts, data policy, latency metrics
- Request logs
- Every call captured with model, status, latency, cost, tokens, and full prompt/response if your data policy allows. Per-row expand, filter by model/status/time/key/team. 90-day retention default; longer on Enterprise.
- Logging callbacks
- Langfuse (traces + spans + generations), Datadog (metrics + correlation IDs), S3 (raw JSON archive), Slack (real-time event feed), and custom HTTPS webhooks with signed payloads. One toggle each, async, fire-and-forget with retry queue.
- Latency metrics
- p50 / p95 / p99 + max per model, key, team, org. Time-to-first-token and total latency tracked separately, streaming-aware (TTFT measured at first SSE event). CSV export for long-tail diagnosis.
- Budget & spend alerts
- Balance thresholds, daily and monthly spend caps, and per-budget utilization. Multi-channel delivery: email + Slack + Teams + webhook, with per-org and per-key thresholds.
- Data policy: 4 levels
- Zero (billing-essential only), metadata-only (default: model, tokens, cost, latency, status), full (complete prompt + response), or PII-redacted (full with Presidio masking pre-storage). Switch any time per org; applies immediately.
- Async: zero added latency
- Log ingestion happens after the response returns (< 100 ms to the in-product viewer). Failed callbacks never affect request success. The cost-header passthrough is the only synchronous invariant.
Common observability questions
What data is logged by default?
Metadata only: model, tokens, cost, latency, status code, and request ID. No prompt or response content is stored by default. Enable Full or PII-redacted logging in your dashboard data-policy settings if you need request bodies for prompt replay or fine-tuning datasets.
Can I integrate with my existing monitoring?
Yes. Callbacks to Langfuse, Datadog, S3, Slack, and custom webhooks. Enable any integration with a single toggle from your dashboard. Each callback receives structured JSON with full request metadata, so you can build custom dashboards and alerts in your existing tools.
Is observability available on all plans?
Yes. All plans include full observability: logs, alerts, callbacks, and data-policy controls. There is no feature gating. Every NemoRouter customer gets the same observability capabilities regardless of tier.
How long are logs retained?
90 days for request logs by default. Financial records (credit transactions, spend tracking, invoices) are retained indefinitely. Configure S3 callbacks for custom long-term archival beyond the 90-day window. Enterprise contracts can extend default retention.
Does logging add latency?
Logging is async and adds zero latency to your LLM requests. Log ingestion happens after the response is returned to the caller. Callbacks to external services (Langfuse, Datadog, S3, webhooks) are also processed asynchronously and never block the request path. The only synchronous observability concern is the x-nemo-request-cost header, which is the source of truth for cost.
90-day retention · 4 callbacks · 8 alerts
See every request — without DIY observability infrastructure
Sign up, enable a callback or two, watch the data flow into your existing stack. No agents to install, no schema to invent.