Connect LlamaIndex to nRouter for retrieval-augmented generation (RAG) and agent workflows. Configure LLM and embedding endpoints with enterprise guardrails.

LlamaIndex's OpenAI LLM and OpenAIEmbedding work directly with nRouter. Set api_base to https://api.nrouter.ai/v1 — guardrails, caching, and rate-limits auto-apply.

Installation

pip install llama-index-llms-openai llama-index-embeddings-openai nrouter-sdk

Setup

import os
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.core import Settings

NROUTER_KEY = os.environ["NROUTER_API_KEY"]
NROUTER_BASE = "https://api.nrouter.ai/v1"

Settings.llm = OpenAI(
    model="claude-sonnet-4-5-20250929",
    api_key=NROUTER_KEY,
    api_base=NROUTER_BASE,
)
Settings.embed_model = OpenAIEmbedding(
    model_name="text-embedding-3-small",
    api_key=NROUTER_KEY,
    api_base=NROUTER_BASE,
)

Chat

from llama_index.core.llms import ChatMessage

response = Settings.llm.chat([
    ChatMessage(role="user", content="What is quantum computing?"),
])
print(response.message.content)

Per-Request Overrides

Pass nrouter_* fields via additional_kwargs:

llm_with_prompt = OpenAI(
    model="gpt-5.5",
    api_key=NROUTER_KEY,
    api_base=NROUTER_BASE,
    additional_kwargs={
        "nrouter_prompt_template_id": "your-summarizer-id",
        "nrouter_prompt_variables": {"language": "Spanish", "max_length": "100"},
        "nrouter_cache": False,
    },
)

Guardrails are not passed here. You assign them in the dashboard at key, team, or organization scope — the narrowest scope that mentions a guardrail wins — and they run automatically on every request that scope covers.

RAG with Query Engine

from llama_index.core import VectorStoreIndex, Document

documents = [
    Document(text="nRouter is an LLM gateway with the models available in your live catalog."),
    Document(text="Guardrails protect every request: PII, injection, keywords."),
]
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

response = query_engine.query("How does nRouter handle safety?")
print(response)

The user query is checked by guardrails before it reaches the model. Prompt-injection attempts on RAG queries are blocked before retrieval starts.

Next Steps

LangChain — Same pattern for chain-style RAG
Python SDK — Without LlamaIndex

LlamaIndex