Four Outcomes That Get an LLM Gateway Funded

The position

An LLM gateway is not bought on capabilities; it is funded on four outcomes — total spend, cost per request, performance under load, and time to ship the next AI feature. Write the case in that language, with a number on each line, or it stalls at the first finance review.

Most gateway pages sell mechanisms. The person who signs the purchase order is not buying a mechanism. They are buying a change to a line on a spreadsheet and a change to how fast the team ships, and they will approve neither unless you have written both down in their units.

This post is the internal case, in the order a VP of Engineering, CTO, or Head of Platform actually has to pitch it. Every claim below is either arithmetic you can redo or a link to a page you can check. If you are still establishing what the category even is, start with what an LLM gateway is and come back.

What we were asked for instead

The request we keep getting — and keep declining — is the feature matrix. Send us the capability grid, twenty rows deep, with checkmarks across four vendors. Procurement wants it, and it is a reasonable thing to want.

Stated at its strongest, the case for the matrix is solid:

All true. We fill these grids in when asked, and we do not lie in them. But a completed matrix answers the question "what does it do?" and the budget owner asked "what changes for the business?" Those are different questions, and the second one is the one that releases money.

What that would cost you

Here is the concrete failure mode, and it is not hypothetical. A matrix-led case reaches finance and gets one of two responses: "we already have observability" or "come back with a number." Both are correct responses to a document that lists capabilities and forecasts nothing.

The cost of that is measured in quarters, not dollars. Meanwhile:

1. The spend keeps growing unattributed. Every month without a gateway is a month whose provider invoice cannot be split by feature, team, or customer. That history is not reconstructible later — you can start attributing tomorrow, but you cannot attribute last quarter.
2. The next incident has no containment. Without per-key ceilings, a retry loop shipped on a Friday is a budget event, not a log line. Four ceilings every request passes is the shape of what you are missing.
3. Each new model is a project. Teams without a single seam integrate providers one at a time, which means the cheapest qualifying model for a workload stays unadopted because adopting it costs two sprints.
4. The matrix ages badly. Capability grids are true on the day they are filled in. Prices and model families move faster than procurement cycles — every hyperscaler publishes its rates on a page it revises without notice, Azure OpenAI Service and Google Vertex AI among them — so the grid is stale before the contract is signed.

A capability list has no unit. An outcome does. Rewrite the case as four outcomes and the same facts start moving.

The four outcomes, mapped

Each row starts in the language of the person who signs. Finance reads the middle column. Engineering validates the right one. Nobody has to translate "OpenAI-compatible endpoint" into "we will ship faster" — the translation is already done.

Capabilities described here are nRouter's, verified 2026-08-23. SOC 2 Type II is in progress, not certified — see the trust page. "Type II" here means an attestation over a period against the AICPA's trust services criteria, which is worth knowing before a procurement call treats the phrase as a checkbox.

The memo that gets funded is one page and follows this order:

1. The number today. Last month's provider invoice, total, and the fraction you cannot attribute to a team.
2. The four outcomes, each with the target you will hit and the date you will measure it.
3. The blast-radius incident you are preventing, named, with its plausible cost.
4. The migration cost, honestly: a base URL, a key, and a rollout window.
5. The exit, honestly: what it takes to leave, and how long.

Line 5 is the one people skip and the one that wins the room. A buyer who has been told the exit cost believes the other four lines.

Outcome 1 — total spend: the platform fee is the line a vendor choice can actually move

Token prices are set by the provider, on the provider's own page — OpenAI's is the reference most teams price against. Your vendor choice does not move them. What a vendor choice does move is the overhead layered on top — and that overhead is where gateway pricing quietly differs by an order of magnitude.

Work the arithmetic for a team spending $20,000/month with providers. On pay as you go the fee is added on top at purchase as a flat 4% of the credits: exactly $800/month on $20,000 of credits. On Pro, where the platform fee is 0%, it is a flat $50/month, or $500/yr (about $41.67/month); every plan's numbers sit on the pricing page.

That is exactly $750/month saved against the monthly Pro plan, or $9,000/year, moved by one line in a billing settings page. The crossover is public and does not need a sales call: the fee on a month's spend is 4% of that spend, so Pro pays for itself once it passes $50, which is exactly $1,250/month of provider spend on the monthly plan and around $1,042/month on the annual one. Below that, pay as you go is cheaper, and we would rather you stay on it — the full walkthrough is in from credits to Pro.

The second half of "total spend" is what you are not charged extra for. Guardrails, A/B tests, prompt management, evals and per-team budgets are on every plan; plans vary the platform fee and the rate limits and nothing else. That is a deliberate commercial position, argued in Every Feature on Every Plan: We Charge a Fee, Not a Gate. For a finance reviewer, it removes the line item that usually appears in month four labelled "enterprise upgrade to enable the thing we already assumed we had."

One honest note for the memo: the evaluation itself begins with a card, because there is no free tier and the $5 minimum is a real charge with the platform fee on top. Put that line in the memo rather than discovering it in the procurement call.

Outcome 2 — cost per request: you cannot cut what you cannot attribute

"Reduce AI costs" is a different claim from "spend less money," and conflating them is why so many cases fall apart under questioning. Total spend can fall because usage fell. Unit cost falling means you got the same work done for less, and proving that requires per-request cost attached to the request.

Every response carries the exact cost of that call on x-nr-request-cost, paired with x-nr-cost-status. When the cost is not knowable, the header is absent and the status reads unpriced — never a fabricated $0, for the reasons in cost honesty.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.nrouter.ai/v1",
    api_key=os.environ["NROUTER_API_KEY"],
)

resp = client.chat.completions.with_raw_response.create(
    model="claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": prompt}],
)

cost = resp.headers.get("x-nr-request-cost")          # absent when unpriced
status = resp.headers.get("x-nr-cost-status")         # "exact" | "unpriced"

With that number on the request you can do the thing the finance case actually needs: tag spend by team, customer and feature (attribution tags), then find the model that is quietly expensive rather than the one that is obviously popular (cost vs usage). Routing cheap traffic to a cheaper qualifying model is the lever everyone reaches for first; attribution is what tells you which traffic is cheap. Do them in that order.

Forecasting before you have data is guesswork, so use the cost calculator for the memo's first draft and replace its numbers with measured ones within a month.

Outcomes 3 and 4 — performance and shipping speed come from the same seam

These read as two outcomes and are one mechanism. Because every request passes through one place, you can change what happens to it without changing application code — and that single property produces both the reliability story and the velocity story.

Performance under load. A provider degrading is not an outage if the next call goes elsewhere. Fallback chains handle the failover, and the subtlety worth putting in the memo is that a retry is a second call and therefore a second bill — a naive failover double-charges for one user-visible request. Meanwhile ceilings keep one misbehaving job from consuming the quota of everything else: budgets and rate limits are different controls for different failure shapes, and knowing which to reach for is most of the operational skill. When the ceiling does bite, your application sees a 429 or a 402, and handling 429 and 402 is the client-side half. Report the result in percentiles, not averages — p50, p95, p99 is what an SLO is written in.

Shipping speed. Adopting a different model is a string change, and adopting nRouter itself is a base URL and a key:

- client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
+ client = OpenAI(
+     base_url="https://api.nrouter.ai/v1",
+     api_key=os.environ["NROUTER_API_KEY"],
+ )

That diff is the whole "faster deployment" claim, and it is the one a skeptical engineer can verify in ten minutes with the quick start. Which models the key reaches is a live question rather than a fixed list — the models page is the current answer, and Anthropic, OpenAI and AWS Bedrock are all live today.

The trade we are making

A gateway is a hop, and we are not going to pretend otherwise.

Our availability becomes part of yours. A direct SDK call has one dependency; through us it has two. That is why the status page is public and why fallback behaviour is documented rather than implied.

You do not bring your own provider keys. nRouter holds the provider relationship. If you have negotiated committed-spend pricing directly with a provider, you cannot carry that rate through us, and our fee sits on top of a public rate rather than your discounted one. For some teams that single fact decides the evaluation, and it should.

Outcome-led cases are harder to write than feature grids. The matrix can be delegated; the memo cannot. Somebody senior has to know last month's number. That is real work we are asking you to do before the product helps you at all.

Who this is wrong for

• Teams with a large committed-spend contract. If you already beat public rates, run the arithmetic against your actual rate card before anything else.
• Single-service, single-provider deployments. One key, one service, nobody asking for attribution — a direct SDK call is genuinely correct and a gateway is overhead.
• Organisations that forbid a third party in the request path. Self-hosting is the answer, and no feature parity argument changes a compliance boundary.
• Anyone who needs a free evaluation tier. There is not one. The $5 minimum is a real charge.
• Buyers who need every governance feature in a signed enterprise agreement before piloting. You can have that conversation via enterprise, but the features themselves are already on the cheapest plan, which surprises people.

How you can hold us to it

Every outcome above is checkable without talking to us.

1. Total spend: compare the pricing page against the table in outcome 1. If they disagree, the pricing page is authoritative and this post is a defect worth an email.
2. Unit cost: read x-nr-request-cost off a live response, sum a day of them, and reconcile against your ledger. Reading a spend ledger is the how.
3. Ceilings: set a $1 budget on a test key and prove the 402 arrives before the provider call, using hard spend limits.
4. Performance: the status page is public history, not a marketing claim.
5. Feature parity: create a guardrail on pay as you go. If it asks you to upgrade, outcome 1's last paragraph is false.
6. Compliance posture: the trust page states SOC 2 Type II as in progress. If you ever see us write "certified" before it is, that is the sentence to hold us to.

Try it

Write the one-page memo from the five-line template above, then spend an afternoon replacing its guesses with measurements. Create an account at signup, load the $5 minimum, point one existing service at https://api.nrouter.ai/v1 with NROUTER_API_KEY, and let it run for a day. You will have a real per-request cost distribution, a real attribution split, and a real answer on the Pro crossover — which is a considerably better memo than any capability grid. The vendor-neutral version of the same exercise is the LLM gateway buyer's guide.

See also

• LLM gateway buyer's guide 2026 — the vendor-neutral evaluation this memo sits inside.
• Every Feature on Every Plan: We Charge a Fee, Not a Gate — the commercial position that makes outcome 1's last row true.
• From credits to Pro — the crossover arithmetic worked all the way through, with the annual case.
• Attribute LLM spend by team, customer, and feature — how outcome 2 stops being an estimate.
• Provider fallback chains: surviving an OpenAI outage — outcome 3's mechanism, including the double-billing trap.
• The four ceilings every LLM request passes — the containment story to put in the incident line of the memo.
• Pricing — the authoritative fee, minimum purchase, and Pro price in their live form.

Sources

Verified 2026-08-23. If any linked page has changed since, email hello@nrouter.ai and we will correct this post.

• nRouter plans and fees: nrouter.ai/pricing — $0 subscription with a platform fee of 4% of the credits, added on top; Pro at $50/mo or $500/yr at 0%, $5 minimum purchase.
• OpenAI API pricing: openai.com/api/pricing — per-model rates referenced by the unit-cost argument.
• Anthropic pricing: anthropic.com/pricing — per-model rates for the Claude family.
• AWS Bedrock pricing: aws.amazon.com/bedrock/pricing — Bedrock is live on nRouter and AWS publishes its own rates.
• OpenAI API reference: platform.openai.com/docs/api-reference — the request shape the https://api.nrouter.ai/v1 endpoint is compatible with.
• OpenAI pricing, developer-docs copy: platform.openai.com/docs/pricing — the per-million-token rates the unit-cost argument is priced against.
• Azure OpenAI Service pricing: azure.microsoft.com — one of the pages that moves under a capability matrix.
• Google Vertex AI generative-AI pricing: cloud.google.com — another.
• AICPA/CIMA trust services criteria (2017, revised points of focus 2022): aicpa-cima.com — what a SOC 2 Type II attestation is evaluated against.

OpenAI, Anthropic, AWS, Microsoft and Google are trademarks of their respective owners. nRouter is not affiliated with or endorsed by them. All claims above are sourced from their public pricing or documentation on the date shown.