Managed LLM Gateway vs Self-Hosted: Why We Carry the Pager

The position: nRouter is a managed LLM gateway, and we are not shipping a self-hostable build of it — because the hard part of a gateway is not standing it up, it is operating it correctly on the day a provider fails while money is moving through it.

That is a real decision with a real cost to you, so this post argues it properly: the strongest version of the request we keep declining, the concrete bill self-hosting hands you, the mechanism we offer instead, what our choice takes away, and the teams for whom self-hosting is genuinely the right answer.

The position

A gateway is not a piece of software you install once. It is a control plane that sits between your application and every model provider you use, and it is in the path of three things you cannot get wrong: money, tenancy, and availability. A request that is billed twice, attributed to the wrong team, or dropped during a provider incident is not a bug you fix next sprint — it is a number in someone's invoice and a conversation with your customer.

Running that correctly is a standing job, not a setup task. We decided the honest thing to do was to take the job rather than hand you a binary and a README, and to be explicit that this is a trade, not a favour.

What we were asked for instead

The request arrives in three forms, and each is reasonable:

1. "Ship a self-hostable build." We already run the thing; publishing an image is a small marginal effort for us and a large win for a team with a platform group that would rather own it.
2. "Our data cannot leave our network." For regulated workloads, an in-VPC deployment is not a preference, it is a control the auditor asks about by name. A managed service is a vendor in the data path, and a vendor in the data path is a question you have to answer.
3. "We do not want the dependency." If nRouter changes its pricing, its terms, or its mind, a self-hosted deployment keeps running. A managed one does not. That is a legitimate continuity concern and we are not going to pretend it is paranoia.

Stated at full strength, the case for self-hosting is: you keep control of the data path, the upgrade cadence, and your own destiny, and you pay for it in engineering time you were going to spend anyway. That is a good argument. It is right for some teams. It is not right for most, and the reason is in the next section.

What that would cost you

Standing a gateway up is an afternoon. Operating one is a permanent line item. Here is the work that does not end, laid out honestly:

None of those rows is exotic. Each is the kind of work a competent platform engineer does well and a busy feature team does late.

The cost is not the server. Servers are cheap. The cost is that a gateway sits in the money path, and money-path software has to be right under conditions you do not control: a provider returning a 200 with no usage block, a rate limit that arrives mid-stream, a fallback that fires and produces a second billable call for one customer request. Getting each of those right is a week of careful work plus a permanent test you now maintain. Getting one of them wrong is a refund and an apology.

The number that decides it for most teams

Our platform fee is a flat 4% of the credits on pay-as-you-go — added on top at purchase, so the credits you buy land in full and the fee works out to 4% of your spend — and 0% on Pro at $50/mo or $500/yr. That gives you an unusually clean way to price the decision, because the managed alternative has an actual number attached:

The crossover is exactly $1,250/mo of provider spend for the monthly plan, and around $1,042/mo for annual at $500/yr. Above that line, a flat $50 a month is what a fully operated gateway costs you.

So the self-hosting question becomes concrete: can your team run a money-path control plane — on-call, upgrades, pricing tables, ledger correctness, tenant isolation and audit evidence — for less than $50 a month of loaded engineering time? For a platform group that already runs this class of system, sometimes yes. For a six-person team shipping features, that question answers itself. See the four ceilings every request passes for the specific enforcement layers you would be rebuilding.

What we do instead

Managed here does not mean "we host the same software and add a bill." It means the operational surface is ours and none of it is yours:

• We run the request path. Routing, retries, and provider fallback chains that survive an outage without your code changing.
• We hold the provider accounts. You never sign up with a provider or paste a provider key — the reasoning is in why we do not do BYOK.
• We keep the ledger. Credits are reserved before a call and settled after it, so a failure releases the hold instead of charging you for a request that never happened. The mechanism is in reserve-and-settle, and the output is a ledger you can read line by line — here is how.
• We keep the prices current. When a provider changes a rate, the change is ours to chase, not yours. When a call cannot be priced, it is reported as unpriced rather than as zero — that stance is cost honesty.
• We ship the governance layer on day one. Guardrails, budgets, roles, audit trails, A/B tests, prompt templates and evals are on every plan, because we charge a platform fee instead of gating features.

What you operate is a key and a budget. The quick start is a base-URL change and an NROUTER_API_KEY; everything above happens behind https://api.nrouter.ai/v1.

The trade we are making

Here is the part a marketing page would skip.

You take a dependency. Your request path now includes a company that is not you. If we have an incident, you have an incident. That is the actual price of not carrying the pager, and no amount of architecture removes it — you can only decide whether you would rather debug your own gateway at 3am or read our status page at 3am.

Your traffic transits our systems. For most teams that is an ordinary vendor relationship, handled with a data-processing agreement and the controls described on security and trust. For some workloads it is disqualifying, and we would rather say so here than discover it on call three.

You do not control the upgrade cadence. We change the gateway; you do not schedule the change. We keep the customer-facing API stable and additive for exactly this reason, but "stable and additive" is a commitment, not a configuration flag you hold.

You cannot fork us. If we make a decision you hate, your recourse is to migrate, not to patch. Because the API is OpenAI-compatible, that migration is a base URL and a key — which is deliberate, and is the same property that makes moving off another gateway an afternoon rather than a quarter. Portability is the honest answer to lock-in, and it is the one we chose to invest in.

Who this is wrong for

Name the segment plainly. Self-host, and do not let us talk you out of it, if:

• You have hard data-residency or air-gap requirements. If model traffic cannot leave your network or your region, a managed gateway in another network is not a fit. No commercial argument beats a control requirement.
• You have large pre-negotiated provider commitments. If you have committed spend with a provider that you must draw down directly, our model does not let you — see why no BYOK for the full reasoning and why this exclusion is deliberate rather than accidental.
• You already run this class of system. If you have a platform team operating money-path infrastructure with on-call and change management today, the marginal cost of one more control plane is genuinely low for you, and total control over routing internals may be worth more than a flat fee.
• You need to modify the routing behaviour itself. Not configure it — modify it. Custom scoring, in-house model endpoints with bespoke auth, an internal policy engine in the hot path. Configuration we support; a fork we cannot.

If two or more of those describe you, self-hosting is the right call and this post is not an argument against it. It is an argument against defaulting into it because "we could run it ourselves" is technically true.

How you can hold us to it

A position is worth nothing if you have to take our word for it. Everything below is checkable without asking us:

1. The ledger. Every credit movement has a row — reserved, settled, released, refunded. If a request failed and you were charged, the row is missing or wrong and you can point at it: how to read it.
2. The cost header. Every response carries the x-nr-* headers, including the per-request cost. When a call cannot be priced, the cost header is absent and the status header says unpriced. It is never a fabricated $0.
3. The pricing page. The fee, the plans, and the minimum live on pricing, not in a quote. If this post and that page disagree, the page wins and the post is a defect.
4. The catalog. Models is the live list. A model we do not serve is not on it.
5. The status page. Incidents go on status. A managed vendor that hides its incidents is a managed vendor you should not use.
6. The exit. The API is OpenAI-compatible on purpose. Point your SDK's base URL somewhere else and you are gone — the strongest commitment a managed vendor can make, because it is the one that costs us if we get lazy.
7. The compliance posture. SOC 2 Type II is in progress, not complete, and we say so on trust. If you need the report today, ask before you build, and use the SOC 2 checklist for gateways to interrogate us properly.

Try it

The fastest way to test the claim is to stop reading it. Point an existing OpenAI-compatible client at https://api.nrouter.ai/v1, set NROUTER_API_KEY, and send one request:

curl https://api.nrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $NROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"MODEL_FROM_YOUR_CATALOG","messages":[{"role":"user","content":"ping"}]}' \
  -i | grep -i '^x-nr-'

The -i | grep is the point: you are reading the cost of the call you just made, on the call you just made. Then open the playground to try a second model without touching your code, and the billing view to see that same request as a ledger row.

Signup is card-required and takes a real $5 minimum charge with the platform fee on top — there is no free tier and we do not pretend otherwise. Start at signup, or bring questions to the community first.

See also

• Why we don't do BYOK — the sibling decision; managed operation and managed provider keys are the same choice seen from two sides.
• Every Feature on Every Plan: We Charge a Fee, Not a Gate — why the managed fee buys the whole feature set rather than an entry tier.
• Cost honesty — what we do when a call cannot be priced, and why it is never reported as zero.
• Reserve-and-Settle: Never Overspend a Credit Balance — the ledger mechanism that makes "we run the money path" a checkable claim.
• Provider fallback chains — what happens to your traffic during a provider outage, and who is awake for it.
• A SOC 2 checklist for LLM gateways — the questions to ask any gateway vendor, including us.
• Pricing — the fee, the plans and the crossover number, from the source rather than from this post.

Sources

Verified 2026-08-23. Corrections to hello@nrouter.ai and we will update.

nRouter's own numbers — the 4% platform fee, the $50/mo and $500/yr Pro plan, and the $5 minimum purchase — come from our pricing page and are authoritative there, not here. The provider pages below are the upstream rate cards a self-hosted gateway would have to track and keep current on its own; they are listed as the concrete form of the "provider drift" and "pricing accuracy" rows in the table above.

• nRouter pricing: nrouter.ai/pricing
• nRouter model catalog: nrouter.ai/models
• Anthropic pricing: anthropic.com/pricing
• OpenAI API pricing: openai.com/api/pricing
• AWS Bedrock pricing: aws.amazon.com/bedrock/pricing
• Google Vertex AI generative-AI pricing: cloud.google.com/vertex-ai/generative-ai/pricing
• AICPA SOC 2 overview: aicpa-cima.com