Notes on shipping reliable LLM infrastructure.

Features & releases·9

Architecture & internals·13

Comparison

How we compare·13

Guides

How-tos & best practice·16

All posts

Browse the catalog

Filter by category, or browse the full archive of deep-dives.

63 posts found

An LLM Gateway for Coding Agents

Coding agents burst into hundreds of model calls per task across many tools. Here is how a gateway gives them budgets, fallback, and per-task attribution so an autonomous loop can't run up a surprise bill.

2026-06-128 min

An LLM Gateway for RAG: Embeddings and Chat, One Key

RAG apps call two model types — embeddings and chat — often from different providers. Here is how a single gateway unifies both behind one key, with shared cost tracking, budgets, and fallback.

2026-06-128 min

Migrate Off OpenRouter in an Afternoon

Switching LLM gateways sounds like a project. It's usually a base-URL change. Here is how to migrate from OpenRouter to NemoRouter without rewriting your app — and what you gain by doing it.

2026-06-127 min

Per-Customer LLM Billing for AI Apps

If you sell an AI product, you need to know what each customer costs to serve. Here is how request tags, team scoping, and exact cost tracking turn one gateway bill into per-customer economics.

2026-06-128 min

Building a Tamper-Evident Audit Trail for Admin Actions

"Who rotated that key?" should be a query, not a guess. Here is how to build an audit trail for an LLM gateway with complete attribution, consistent IP capture, and tamper-evidence auditors trust.

2026-06-118 min

Guides

A SOC 2 Checklist for LLM Gateways

Routing company data through LLMs raises the same controls auditors already ask about. Here is a practical SOC 2 checklist for an LLM gateway — access, audit, retention, encryption, and tenant isolation.

2026-06-119 min

Org, Team, Member: Scoping Keys, Budgets, Guardrails

A flat "everyone in the org" model breaks the moment a company has two teams. Here is how NemoRouter scopes resources across org, team, and key — and why the active team comes only from the authenticated context.

2026-06-119 min

Guides

Virtual Keys vs Master Key: Designing Blast Radius

One leaked key shouldn't compromise everything. Here is how separating a master key for management from per-use virtual keys bounds the damage of a leak — and why customer traffic never touches the master key.

2026-06-118 min

Server-Side Prompt Templates With Versioning

Hardcoded prompts scattered across services are impossible to change safely. Here is how server-side prompt templates with versioning let you edit, roll back, and A/B prompts without a redeploy.