A feature matrix does not survive a finance review. The internal case for an LLM gateway is four measurable outcomes — total spend, unit cost, performance under load, and shipping speed — each with a number attached.
A forecast is only a forecast if the worst case is bounded. Set hard dollar ceilings at the org, team, user and key scope, decompose next month's number into them, and know exactly which code your client gets when one bites.
Routing is the one cost lever you can pull from a dashboard. Point an alias at a set of models, choose cost or latency or weighted, and change what a request costs without touching a line of application code.
Request count and dollar cost tell different stories, and the gap between them is where the savings are. Here are the four shapes an overlay of cost and usage produces, which one to chase first, and what makes the numbers trustworthy enough to act on.
Configure log destinations once at the gateway instead of instrumenting every call site. Here is how to add and verify a callback today, what the Beta does and does not deliver yet, and the live paths that get data out in the meantime.
A budget caps dollars over a window and answers 402; a rate limit caps RPM/TPM right now and answers 429. Here is how to classify the risk, set each control in the dashboard, and write client code that tells the three rejections apart.
Auto top-up keeps your balance from hitting zero mid-traffic, but on its own it removes the only thing that stops a runaway. Here is the three-number configuration — threshold, top-up amount, and a Block-mode budget — that makes it safe.
A budget is a dollar allowance attached to a scope and a window. Here is how to create one on each of the four scopes, which status code each returns when it fires, and how to prove the cap bites before you trust it in production.
Quality is a property of a task, not of a model. Here is how to inventory your traffic by task, point a router alias at a candidate set, prove each downgrade with an A/B test, and read the saving off the Cost vs Usage report.
A rebuilt savings model for a $40k/month LLM bill. Batch and prompt caching carry published rates; provider capacity reservations do not, so that lever is a labelled assumption with a stated range. Every line is separated into sourced and assumed.
Provider pricing does not normalize on its own — per-token, per-image, per-second, provisioned. Here is how nRouter turns that into one settled cost per request that your app, your ledger and your dashboard all read, and why an unknown cost is reported as absent rather than as zero.
