Model Benchmarks
Empirical evaluations across MMLU, HumanEval, GPQA, MATH, and SWE-bench with measured latency percentiles and zero-markup pricing.
How a nRouter benchmark is run
A benchmark is only worth printing if someone else can reproduce it. Every study we publish follows the same four rules, and ships the config so you can run it yourself.
Isolate the gateway
Run against the zero-cost mock provider so provider latency is constant. Whatever moves is gateway overhead. Nothing else.
Warm, then sample
Discard cold-start requests, then collect a large fixed sample at steady state. Cold starts are reported separately, never blended into p50.
Report the tail
Publish p50, p95, and p99. Never just the average. A gateway that looks fast at p50 and stalls at p99 is a slow gateway.
Show the config
Every run ships its config: region, instance size, guardrails enabled, concurrency, sample size, and date. A benchmark you cannot reproduce is marketing.
How results will be published
A dated report, not a moving number. Each study is timestamped and versioned. Older reports stay live so you can see the trend, not just today’s headline.
Full run configuration. Region, instance size, concurrency, guardrails enabled, sample size, and provider mix — everything needed to reproduce the run.
p50, p95, and p99 — always the tail. Every latency claim ships all three percentiles. A single average is not a benchmark; it is a hope.
Failover and error rates alongside latency. Reliability and speed are reported together. A gateway that is fast only when nothing fails is not actually fast.
Want the methodology in detail, or to run a benchmark against your own workload before committing? Email sales@nrouter.ai.
Availability is a number we sign
The 99.9% uptime SLA is a contractual commitment in the legal SLA, not a figure we picked because it looked good on a slide. The gateway runs on managed autoscaling container infrastructure and managed Supabase Postgres.
99.9%
Committed, monitored, and public — component health and incident history are never behind a login.
Measure it yourself
The fastest benchmark is your own traffic
Point a real workload at nRouter and watch the overhead in your own dashboard. No fabricated leaderboard to trust. Just the gateway, your requests, and the numbers.