Routing

One API. 169+ models.

Call Alibaba US (Qwen, DeepSeek — our largest catalog), OpenAI (GPT, o-series), Azure Foundry, Google Vertex AI (Gemini), and Anthropic (Claude) live today through a single endpoint. Cost, latency, and weighted routing, deterministic A/B splits, configurable fallback chains — config, not redeploy.

Get started See the pipeline

routing-decision · request_id 7f2a

Live routing trace

Routing strategycost-optimized

Active providerGoogle Vertex AI

Zero-downtime fallbackAnthropic → Microsoft Azure

Routing compute< 1 ms P99

Automatic failoverSingle credit holdZero double-billing

Routing strategies: 4
Retries: Configurable
A/B splits: Deterministic
Fallback chains: Opt-in

Strategies

Four strategies. One config field.

Choose across 4 strategies for your workload — cost, latency, intent across 11 intent categories, or your own weighted split. The router does the rest, per org, no redeploy.

costprice per token

Picks the cheapest capable model in your candidate set. Optimize the bill, not the benchmark.

latencyrecent success-weighted latency

Steers each request to the candidate with the lowest recent success-weighted average latency for your org — a moving average over recent successful calls, weighted toward the newest.

weightedyour split

Splits traffic across the candidates in one Smart Router by the weight you give each. You call the router by the alias you named it.

intentsemantic task classification

Strategy::Intent and nrouter/auto classify query complexity across 11 intent categories (5 light, 6 heavy) via Cortex OP_CLASSIFY_INTENT to steer queries to the optimal price-performance model tier.

How it works

A routing decision, end to end

Strategy match, primary provider, fallback chain, settle. A substitution is surfaced on the x-nr-model response header — the decision is made in process, with no extra network hop.

Routing decision flow

Request
POST /v1/chat/completions
Bearer sk-nrouter-..., model="gemini-2.5-flash"
Strategy match
cost · candidate set
Alias resolves to the cheapest capable model in your set.
Primary provider
Vertex AI · gemini-2.5-flash
429, 503, or 529 triggers the fallback chain.
Fallback chain
Anthropic → Azure → OpenAI
Ordered backup list per model; each link retried in turn.
Settled
cost + latency logged
Strategy, primary, fallback, retries: all in the request log.

Drop-in

Two-line migration from OpenAI

OpenAI-compatible

Change the base URL and the key. That’s it

The unmodified OpenAI SDK works against nRouter on every supported language. Streaming, tool use, JSON mode, embeddings, image generation — all handled natively by the gateway.

Python, Node.js, Go, Java, Ruby, PHP, cURL
Base URL: https://api.nrouter.ai/v1
Auth: Bearer sk-nrouter-...
Drop-in for chat.completions, embeddings, and images

diff · client.py

Two-line change

- base_urlhttps://api.openai.com/v1

+ base_urlhttps://api.nrouter.ai/v1

- api_keysk-proj-...

+ api_keysk-nrouter-...

Code lines changed2

OpenAI SDK7 languagesstreaming-safe

Full routing reference — strategies, fallbacks, retries, A/B

Fallback chains: Ordered backup list per model — on error and on rate limit. A 429, 503, or 529 triggers the next link, cross-provider (Vertex → Anthropic → Azure). A failed attempt never bills you twice: fallback chains run under a single credit reservation per customer request. Configured, not coded.
Retries + timeouts: Per-org retry count (default 3) and request timeout, with a per-request override. Provider retry-after hints are honored on the retry.
Deterministic A/B splits: Split a model or prompt into variants and bucket traffic deterministically. The same authenticated virtual key stays on the same variant, so retries are consistent and results are attributable.
Never depend on one model: A Smart Router walks its candidates in order and skips any that no longer resolves to a served model, so one retired model does not take your alias down. If none of them is servable the call is refused, naming your alias — never quietly redirected to a model you did not choose.
OpenAI-compatible surface: Same chat.completions, embeddings, and images endpoints. Base URL https://api.nrouter.ai/v1, auth Bearer sk-nrouter-... Streaming, tool use, and JSON mode routed and streamed natively.
Live model catalog: Capabilities and fallback chains live in the catalog, not your repo. New models are added within hours of provider launch — public read-only catalog at /api/public/models.

FAQ

Common routing questions

What latency does routing add?

Routing decisions are evaluated entirely in-process using an asynchronous native Rust core, adding sub-millisecond (<1ms P99) routing compute overhead with zero additional network hops between your call and the provider.

How many models are supported?

169+ models live now on Alibaba US, OpenAI, Azure Foundry, Google Vertex AI & Anthropic; Alibaba US (Qwen, DeepSeek — our largest catalog), OpenAI (GPT, o-series), Azure Foundry, Google Vertex AI (Gemini), and Anthropic (Claude) live today. We add new models within hours of provider launch. Check the model catalog for the full list.

Can I use custom routing strategies?

Yes. Create a Smart Router in your dashboard, give it a candidate set and a strategy — cost (cheapest capable candidate), latency (lowest recent success-weighted average latency), or weighted (your own split across those candidates) — then call it by the alias you gave it. The router resolves a candidate per request.

What happens when a provider goes down?

nRouter automatically retries with the configured fallback chain. If the primary model returns a 429, 503, or 529, we transparently retry on a backup model.

One key. One bill. Every model.

Replace your homegrown failover layer in an afternoon

Sign up, paste your virtual key, change the base URL. Cost / latency / weighted routing, A/B splits, and fallback chains are unlocked on every plan.

Get started Browse models