Point a router alias at a candidate set with the Latency strategy, keep the automatic cross-provider failover, tune the two settings that decide how fast a bad provider is abandoned, and prove the effect on your own p95.
Routing is the one cost lever you can pull from a dashboard. Point an alias at a set of models, choose cost or latency or weighted, and change what a request costs without touching a line of application code.
Request count and dollar cost tell different stories, and the gap between them is where the savings are. Here are the four shapes an overlay of cost and usage produces, which one to chase first, and what makes the numbers trustworthy enough to act on.
An LLM gateway is one endpoint in front of every model provider that owns six cross-cutting jobs — auth, cost, limits, safety, observability and failover. What it does, what happens to a request inside it, and when you need one.
Quality is a property of a task, not of a model. Here is how to inventory your traffic by task, point a router alias at a candidate set, prove each downgrade with an A/B test, and read the saving off the Cost vs Usage report.
A coin flip on every request is not an experiment — it is noise with a dashboard. Here is how deterministic hash-based assignment gives each user a stable variant for the life of a test, why the experiment id belongs in the hash, and what the gateway refuses to let a caller override.
When a provider 5xxs, overloads or times out, the request should still return. Here is how an ordered fallback chain advances only on retryable failures, bills exactly one hop, and turns a 38% error rate into 0.125%.
Vendor-neutral buyer's-guide decision tree across the three durable LLM routing-intelligence shapes — benchmark-anchored (Unify-style), ML-classifier (NotDiamond-style), and operator-controlled (nRouter-style). Three questions, one shape, one product. Pick the failure mode your team is best equipped to own.
Not Diamond returns a model recommendation and charges $0.05 per million tokens routed — you still hold every provider key and make the call yourself. What that leaves you to build, and how deterministic A/B tests compare to a trained router when you have to reproduce a decision.
nRouter vs Unify AI for teams weighing benchmark-driven model arbitration against operator-pinned routing. Why a vendor leaderboard is not your eval, and how to compute cost-per-passing-answer from your own requests.
Eight axes that actually matter when picking an LLM gateway in 2026. Shortlist matrix across OpenRouter, Portkey, Helicone, nRouter. Decision tree by buyer profile, 90-minute evaluation.
An agent run is not one request, it is eleven. Here is how to route each step to the model that fits it, keep the run alive when a provider fails mid-chain, and read one settled cost for the whole run instead of guessing.
