LangChain
Use ChatOpenAI with base_url set to the NemoRouter endpoint. Every chain, tool, and agent runs unchanged.
pip install langchain-openaiProvider docsIntegrations
If it speaks the OpenAI API,
it works with NemoRouter.
A fully OpenAI-compatible REST API: every framework, SDK, and tool that already talks to OpenAI talks to NemoRouter. Change two lines — keep one key, one bill, and built-in guardrails across 240+ models.
No bespoke plugins needed: compatibility is the integration
- OpenAIAPI surfaceDrop-in /v1/chat/completions, /embeddings, /models
- 7SDK languagesPython, Node, Go, Java, Ruby, PHP, cURL
- 7Frameworks testedLangChain, LlamaIndex, Vercel AI, CrewAI & more
- 2Lines to switchbase_url + api_key — nothing else changes
Quickstart
Three steps, two lines of code.
There is no NemoRouter SDK you are forced to adopt. These snippets are generated from the canonical SDK examples: the same source the playground and keys page use.
Step 1
Create a key
Sign up at nemorouter.ai, generate a NemoRouter virtual key (sk-nemo-…), and grab the base URL.
Step 2
Point your client
Set base_url to the NemoRouter endpoint and api_key to your key. That is the entire integration.
Step 3
Ship
Your framework, SDK, and observability tooling all run unchanged — now with one key, one bill, and built-in guardrails.
Install
pip install nemoroutersdk| 1 | # Cache: enabled (org default). Pass nemo_cache: false to skip. |
| 2 | from nemoroutersdk import NemoRouter, NemoGuardrailBlockedError |
| 3 | |
| 4 | client = NemoRouter() # reads NEMOROUTER_API_KEY from env |
| 5 | |
| 6 | try: |
| 7 | response = client.chat.completions.create( |
| 8 | model="gemini-2.5-flash-lite", |
| 9 | temperature=1, |
| 10 | max_tokens=1024, |
| 11 | messages=[ |
| 12 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 13 | ], |
| 14 | # nemo_cache=False, # Uncomment to skip cache |
| 15 | ) |
| 16 | print(response.choices[0].message.content) |
| 17 | |
| 18 | # Auto-captured metadata |
| 19 | meta = client.last_response |
| 20 | if meta: |
| 21 | print(f"Cost: ${meta.cost}") |
| 22 | print(f"Guardrails: {meta.guardrails_applied}") |
| 23 | |
| 24 | except NemoGuardrailBlockedError as e: |
| 25 | print(f"Blocked by guardrail: {e}") |
Base URL is always https://api.nemorouter.ai/v1. Works with the official OpenAI SDKs and any OpenAI-compatible client.
Frameworks
The frameworks your team already uses.
Each of these is a standard OpenAI-API client. NemoRouter does not ship a bespoke plugin for them. It does not need to. They work because the API surface is identical.
LlamaIndex
Point the OpenAI LLM class at NemoRouter. Query engines and retrievers keep working. Only the endpoint moves.
pip install llama-index-llms-openaiProvider docsVercel AI SDK
Configure the @ai-sdk/openai provider with a custom baseURL. streamText and generateText route through NemoRouter.
npm install ai @ai-sdk/openaiProvider docsHaystack
OpenAIGenerator and OpenAIChatGenerator accept an api_base_url: set it to NemoRouter and your pipelines are unchanged.
pip install haystack-aiProvider docsCrewAI
CrewAI reads OpenAI-style env vars. Set OPENAI_API_BASE to the NemoRouter endpoint and your crews run on managed models.
pip install crewaiProvider docsAutoGen
AutoGen LLM configs take a base_url field. Drop in the NemoRouter endpoint and every agent message routes through the gateway.
pip install autogen-agentchatProvider docsGoogle ADK
ADK speaks the OpenAI-compatible interface. Point it at NemoRouter to keep one key and one bill.
pip install google-adkAny OpenAI client
Instructor, Marvin, DSPy, Semantic Kernel… If a library accepts an OpenAI base URL or honors OPENAI_API_BASE, it works. No SDK fork, no shim.
SDK matrix — one endpoint, every language
Python
pip install nemoroutersdk
NemoRouter SDK or openai
Python (OpenAI)
pip install openai
openai.OpenAI(base_url=…)
Node.js / TypeScript
npm install openai
new OpenAI({ baseURL })
Go
go get github.com/sashabaranov/go-openai
openai.ClientConfig{ BaseURL }
Java
Maven / Gradle — openai-java
OpenAIOkHttpClient.builder()
Ruby
gem install ruby-openai
OpenAI::Client.new(uri_base:)
PHP
composer require openai-php/client
OpenAI::factory()->withBaseUri()
cURL / HTTP
Any HTTP client
POST /v1/chat/completions
Live, runnable snippets for every language are on the playground and your dashboard keys page. If a library honors OPENAI_API_BASE, it already works.
Why route through Nemo
The integration buys you more than a base URL.
Pointing your client at NemoRouter is two lines. But those two lines move every request behind a managed gateway: one key for every model, guardrails at the gateway instead of in your framework code, and real cost accounting on every call.
- One key for every model: switch models by changing a string, no per-provider keys
- PII redaction, prompt-injection detection, and secret scanning run at the gateway
- Per-key, per-team spend tracked in real time. Credits reserve and settle automatically
- Langfuse, Datadog, S3, and Slack callbacks are dashboard toggles, not webhook code you maintain
Your code
OpenAI SDK, unchanged
base_url + api_key — two lines, nothing else moves
Nemo gateway
Guardrails · budgets · cost tracking
Every request guarded, attributed, and settled exactly
Providers
240+ models, one bill
We manage every provider key and relationship
Observability & tooling
Logging callbacks, built in.
Request logs, traces, and metrics forwarded to the tools your platform team already runs. Configured from Logging → Callbacks, with zero changes to your application code. Alerts also fan out over generic webhooks.
Langfuse
Traces, spans, and token usage for prompt analytics and evals. Configured once under Logging → Callbacks.
Datadog
Request logs and latency metrics alongside the rest of your service telemetry.
Amazon S3
Raw request logs archived to a bucket you control: retention, audit, batch analysis.
Slack
Budget and spend thresholds routed to a channel: email, Slack, Teams, or webhook.
UpcomingOn the roadmap — Terraform provider & CLI
Terraform provider: manage keys, teams, budgets, and guardrails as code. Planned, not yet published to the Terraform registry.
CLI tool: a nemo command for keys, usage, and config from the terminal. In design, no install command to publish yet.
These are full product builds, not marketing pages. We track them as committed roadmap items, not shipped features. Want early access? sales@nemorouter.ai
Two lines · five minutes
Plug NemoRouter into your stack today.
Keep your framework, keep your SDK, keep your observability tooling. Change the base URL and the key: get one bill, built-in guardrails, and real cost tracking.
OpenAI-compatible · one key · guardrails included