Frameworks
Connect nRouter to popular AI frameworks including LangChain, LlamaIndex, Vercel AI SDK, CrewAI, AutoGen, and Instructor via OpenAI-compatible endpoints.
Last updated
nRouter speaks the OpenAI REST shape, so every framework with an OpenAI integration works without a custom plugin. Point the framework's OpenAI client at https://api.nrouter.ai/v1, pass your sk-nrouter-... key, and everything else — guardrails, prompt templates, caching, rate-limits, and cost tracking — runs server-side.
Supported Frameworks
| Framework | Best for | Guide |
|---|---|---|
| LangChain | Chains, agents, RAG | Setup → |
| LlamaIndex | Document indexing, RAG | Setup → |
| Vercel AI | Next.js / React UIs | Setup → |
| CrewAI | Multi-agent orchestration | Setup → |
| AutoGen | Microsoft agent framework | Setup → |
| Instructor | Structured outputs, Pydantic validation | Setup → |
What you get for free
Without changing a line of framework code:
- Guardrails — PII redaction, prompt injection blocking, keyword filters
- Caching — repeat prompts return instantly from cache
- Prompt templates — versioned, A/B-tested, injected server-side
- Rate limits — per-key / per-org RPM/TPM enforcement
- Cost tracking — priced responses carry
x-nr-request-cost, paired withx-nr-cost-status. When a call cannot be priced the cost header is absent rather than0 - Model fallbacks — switch providers by changing the model name string
What you don't have to build
- PII detection (Presidio, Microsoft Presidio Analyzer)
- Prompt injection detection
- Per-tenant rate limiting
- Cross-provider cost normalization
- Prompt versioning + rollout
- Per-request guardrail allow-listing
All of these are server-side. The framework only sees the LLM response.
Rust SDK
The official nRouter Rust SDK providing standard async-openai integration and a native client for x-nr-* metadata parsing, streaming, and cost tracking.
LangChain
Use nRouter with LangChain in Python and JavaScript. Integrate smart routing, automatic retries, guardrails, and cost tracking into your LangChain pipelines.