Rust SDK

The official nRouter Rust SDK providing standard async-openai integration and a native client for x-nr-* metadata parsing, streaming, and cost tracking.

The official nRouter Rust SDK (nrouter) provides typed, high-performance bindings for the nRouter unified AI gateway (https://api.nrouter.ai/v1). Engineered for zero-cost abstractions and asynchronous concurrency on the Tokio runtime, it offers two distinct entry points: a drop-in client for the standard async-openai ecosystem, and a native HTTP client exposing x-nr-* request metadata, real-time cost tracking, and streaming capabilities.

Routing Rust services through nRouter centralizes governance: server-side guardrails prevent prompt injection before model inference, virtual keys enforce hard team spending limits, and multi-cloud fallback cascades guarantee 99.99% availability across upstream provider outages.

Prerequisites & Installation

The SDK requires Rust 1.75+ (2021 Edition) and the Tokio async runtime.

Add nrouter and tokio to your Cargo.toml. The examples below also use serde_json and async-openai:

[dependencies]
nrouter = "3.1.2"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
serde_json = "1"
async-openai = { version = "0.41.3", features = ["full"] }  # only for nrouter::client()

Or install using cargo add:

cargo add nrouter serde_json
cargo add tokio --features "macros,rt-multi-thread"
cargo add async-openai@0.41.3 --features full   # only for nrouter::client()

Setup & Configuration

Configure authentication by setting your virtual key in your environment:

export NROUTER_API_KEY="sk-nrouter-your-key-here"

The SDK targets https://api.nrouter.ai/v1 by default.

Configuration Parameters

When using the native client, configure connection parameters via nrouter::http::ClientBuilder:

Parameter	Type	Default	Description
`api_key`	`&str`	`std::env::var`	Your nRouter virtual key (`sk-nrouter-...`).
`base_url`	`&str`	`https://api.nrouter.ai/v1`	Unified gateway base URL. Must include `/v1`.
`timeout`	`Duration`	`60s`	Maximum total duration per request.
`connect_timeout`	`Duration`	`10s`	TCP connect and TLS handshake timeout.
`default_headers`	`HeaderMap`	Empty	Headers attached to every outgoing request, such as `x-nr-routing`.

use std::time::Duration;
use nrouter::http::ClientBuilder;

let client = ClientBuilder::new()
    .api_key(std::env::var("NROUTER_API_KEY")?)
    .base_url("https://api.nrouter.ai/v1")
    .timeout(Duration::from_secs(45))
    .connect_timeout(Duration::from_secs(5))
    .header("x-nr-routing", "latency")
    .build()?;

Implementation Patterns

1. Using the async-openai Client

If your codebase already uses the async-openai crate, initialize the client using nrouter::client():

use async_openai::types::chat::{
    ChatCompletionRequestUserMessageArgs, CreateChatCompletionRequestArgs,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Automatically reads NROUTER_API_KEY and targets https://api.nrouter.ai/v1
    let client = nrouter::client()?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("gpt-5.4-mini")
        .messages(vec![ChatCompletionRequestUserMessageArgs::default()
            .content("Hello, nRouter!")
            .build()?
            .into()])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or_default());

    Ok(())
}

2. Using the Native Client (Cost & Metadata)

To access exact USD request costs, token counts, and gateway headers, use nrouter::http::Client:

use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = nrouter::http::Client::from_env()?;

    let out = client
        .chat_completions(&json!({
            "model": "gpt-5.4-mini",
            "messages": [{"role": "user", "content": "Explain zero-copy deserialization in Rust."}]
        }))
        .await?;

    println!("Response: {:?}", out.body["choices"]);

    // Inspect real-time gateway cost
    match out.meta.cost {
        Some(usd) => println!("Cost: ${usd}"),
        None => println!("Cost status: {:?}", out.meta.cost_status),
    }

    println!("Request ID: {:?}", out.meta.request_id);
    println!("Model: {:?}", out.meta.model);

    Ok(())
}

3. Server-Sent Events (SSE) Streaming

The native client incrementally parses Server-Sent Events (SSE) across all text wires:

use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = nrouter::http::Client::from_env()?;

    let mut stream = client
        .messages_stream(&json!({
            "model": "claude-haiku-4-5-20251001",
            "max_tokens": 256,
            "messages": [{"role": "user", "content": "Write a short poem about memory safety."}]
        }))
        .await?;

    while let Some(chunk) = stream.next().await? {
        print!("{}", chunk.delta);
    }
    println!();

    println!("Request ID: {:?}", stream.meta.request_id);

    Ok(())
}

Stream Cancellation: Dropping the stream handle drops the underlying TCP connection immediately. nRouter detects client disconnects and cancels downstream generation, eliminating unread token waste.

4. Per-Request Gateway Overrides

Pass prompt template IDs and cache preferences within the JSON body:

use serde_json::json;

let body = json!({
    "model": "gpt-5.5",
    "messages": [{"role": "user", "content": "Summarize release notes."}],
    "nrouter_prompt_template_id": "tmpl_release_notes_v1",
    "nrouter_prompt_variables": {"version": "v3.1.2"},
    "nrouter_cache": false
});

let out = client.chat_completions(&body).await?;

Production Best Practices

Deterministic Routing & Fallbacks

Support mission-critical systems by configuring model fallbacks and routing headers:

use serde_json::json;

let body = json!({
    // Primary model with automatic fallback
    "model": "gpt-5.4-mini,claude-haiku-4-5-20251001",
    "messages": [{"role": "user", "content": "Analyze telemetry."}]
});

x-nr-routing: latency: Directs traffic to the lowest-latency active provider deployment.
x-nr-routing: cost: Prioritizes the most cost-effective deployment matching the model specification.
Model Fallback Chain: If the primary provider reports 5xx errors or capacity exhaustion, nRouter fails over instantly to the secondary model.

Response Telemetry & FinOps Tracking

out.meta provides full access to gateway metadata:

request_id: Unique request identifier across edge WAF and provider egress.
latency_ms: Edge-measured time from arrival until headers ready.
cost: Exact USD request cost.
cost_status: exact or unpriced.
input_tokens / output_tokens: Token accounting figures.

Troubleshooting & Error Handling

Gateway refusals map to variants of NRouterError:

use nrouter::NRouterError;

match client.chat_completions(&body).await {
    Err(NRouterError::GuardrailBlocked(err)) => {
        eprintln!("Blocked by guardrail policy: {:?}", err);
    }
    Err(NRouterError::Credit(err)) => {
        eprintln!("Insufficient credits or budget reached: {:?}", err);
    }
    Err(e) if e.is_retryable() => {
        eprintln!("Retryable failure (rate limit or 503): {:?}", e);
    }
    Err(e) => eprintln!("Non-retryable error: {:?}", e),
    Ok(out) => println!("Success: {:?}", out.body),
}

Common Error Codes

Variant	HTTP Status	Description	Recommended Action
`Request`	400	Invalid request parameters	Verify JSON schema and model parameter bounds.
`GuardrailBlocked`	400	Blocked by configured guardrail	Review prompt input; adjust guardrail rules in dashboard.
`Authentication`	401	Invalid or revoked virtual key	Check `NROUTER_API_KEY` environment variable.
`Credit`	402	Balance exhausted or budget ceiling reached	Refill organization credits or increase key budget ceiling.
`NotFound`	404	Model not found	Check model catalog for available model IDs.
`RateLimit`	429	Rate limit or TPM limit exceeded	Implement exponential backoff; check retry headers.
`Service`	503	Gateway or upstream service error	Specify fallback models in the request.
`Transport`	—	Network transport error	Check local network connectivity.

is_retryable() returns true for RateLimit, Service, and Transport errors.

Next Steps

Go SDK Guide — High-concurrency Go client
Python SDK Guide — Official Python SDK
Chat Completions API — Direct HTTP wire specifications

Rust SDK

On this page