NotDiamond alternative: a router picks a model, a gateway runs the call

Short answer: Not Diamond picks a model. It does not hold your provider keys, cap your spend, or issue a key per customer — its own quickstart has you keep OPENAI_API_KEY and ANTHROPIC_API_KEY in your .env next to the Not Diamond key. nRouter is the layer underneath that decision, and it makes the decision a different way.

Most head-to-head posts start by claiming the two products overlap. This one starts by insisting they mostly don't, because that is the useful thing to say.

Not Diamond describes itself as an intelligent model router with prompt optimization: you hand it a prompt and a candidate list, it predicts which model will answer best, and you save money by not sending easy queries to expensive models. That is a genuine and narrow product, priced narrowly — $0.05 per million tokens routed. It is not trying to be a gateway, and reading it as one is the mistake that makes this comparison confusing.

The reason people still search for an alternative is that the routing decision turns out to be the smallest problem in the stack. The bigger ones show up a month later: nobody can say what a given customer cost you, a leaked key means rotating credentials in four places, and when quality regresses on Tuesday nobody can reproduce which model answered on Monday. This post is about that gap.

What a router returns, and what you do with it

The integration shape tells you almost everything. In the Python SDK, you call the router with your messages and the candidate models you are willing to use:

from notdiamond import NotDiamond

client = NotDiamond()          # reads NOTDIAMOND_API_KEY

session_id, provider = client.model_router.select_model(
    messages=[{"role": "user", "content": "classify this support ticket"}],
    llm_providers=["openai/gpt-5.5", "anthropic/claude-sonnet-4-5-20250929"],
    tradeoff="cost",           # or "latency"; default maximises quality
)

# ...and now YOU call `provider`, with YOUR key for that provider.

Three observations, none of them criticisms:

1. The candidate set is yours. llm_providers is a list you author. The router picks within it; it does not discover models for you or add a new flagship on your behalf.
2. tradeoff is the whole control surface. Quality by default, cost, or latency. That is a deliberately small knob, and small knobs are a virtue in a product that is trying to do one thing.
3. The call is still yours to make. The SDK offers a create helper that dispatches client-side, but the dispatch happens from your process, with your credentials, against the provider directly. Nothing sits in the request path holding a budget or a guardrail.

That third point is where a router and a gateway actually differ, and it is structural rather than a feature gap.

The .env file is the whole argument

Not Diamond's own Python SDK README asks you to create a .env containing NOTDIAMOND_API_KEY and the provider keys for every model you want to route between — OPENAI_API_KEY, ANTHROPIC_API_KEY, and so on; the routing quickstart sets up the Not Diamond half of that pair.

Read that literally and the operating model falls out of it:

• Every provider key you route to is a key you hold, rotate and blast-radius manage, in every environment and every service.
• A per-customer or per-service key strategy is something you build on top, because the router has no notion of your customers.
• Spend attribution is yours to assemble from each provider's own billing export, keyed by whatever you managed to tag on the way out.

nRouter inverts every line of that. There is one nRouter key and a credit balance; the provider credentials sit on our side and are not available to customers at all. That is a real trade with real losers — a team that has negotiated its own provider rate, or needs a model we do not carry, is worse off — and the argument including those losers is No BYOK: One nRouter Key Instead of Ten Provider Keys.

The per-customer half is the part router users usually miss until it bites: minting a scoped key per service, per tenant or per agent run is how spend becomes attributable at all, and how one leaked credential stops being an incident across every provider you use. That mechanism is described in Virtual Keys vs Master Key: Scoping a Key Per Job.

$0.05 per million tokens routed, converted

Let us be straightforwardly fair about the price, because it is Not Diamond's strongest number and hedging it would be dishonest.

At $0.05 per million tokens routed, a workload pushing 200 million tokens a month through the router pays $10 a month for the routing decision. Against a model bill that is very likely four figures, the router's own fee is close to noise. If the router's recommendations cut even a few percent off that bill, it has paid for itself many times over. Nothing in this post argues that Not Diamond is expensive, because it isn't.

The comparison that matters is not fee against fee — it is what the fee buys. Not Diamond's meter covers a recommendation. nRouter's pay-as-you-go fee (4% of your credits, added on top), or $50/month flat at 0% on Pro, covers the request path: credentials, ceilings, guardrails, logs, attribution and the calls themselves. The numbers are on /pricing, and the reason none of that sits behind a plan tier is in Every Feature on Every Plan.

If you run both — router in front, gateway underneath — you are paying both meters, and that is a perfectly coherent architecture. It is just not the architecture most people have in mind when they type "alternative."

Side-by-side: recommendation layer versus request path

NotDiamond and Not Diamond are trademarks of Not Diamond, Inc. nRouter is not affiliated with or endorsed by Not Diamond, Inc. Every Not Diamond claim above is sourced from Not Diamond's own documentation and pricing page on the dates listed in the sources below; if any have changed, email hello@nrouter.ai and we will update the post.

Routing quality is an eval problem, not a model problem

The pitch for a trained router is that it knows something you don't about which model handles which prompt. Maybe it does — the published work on learned routing reports real gains, and it is worth reading rather than dismissing: RouteLLM trains routers on preference data, Hybrid LLM routes per query on a predicted quality gap, and FrugalGPT cascades upward from cheap models. Note what all three have in common: each result is a number produced by an evaluation harness on a named dataset. The only way to find out whether it holds on your traffic is to run the equivalent harness yourself, and doing that requires three things a router does not give you:

1. A stable assignment. If the same input can be routed differently on two consecutive days, you are measuring the router's variance along with the models'. nRouter assigns A/B variants by hashing a stable identifier, so a given user or session lands on the same variant every time — the mechanics are in Hash-Based A/B Tests.
2. A cost per variant, from the same source as the bill. Comparing quality without comparing cost tells you to route everything to the largest model. Every nRouter call carries x-nr-request-cost on the response, and when the cost cannot be determined the header is absent rather than zero — the discipline behind that is Cost Honesty.
3. A quality signal you chose. Task success, human rating, an LLM judge — a router optimises the objective it was trained on, which is not necessarily the one you would have picked.

The concrete alternative to a trained router is unglamorous and it works: pin two candidates, split traffic deterministically, and read cost against quality per variant. Route the classification job to gpt-5-mini or claude-haiku-4-5 and the ambiguous 10% to gpt-5 or claude-sonnet-4-5-20250929, measure for two weeks, then change the split. Cost-vs-Quality LLM Routing: Which Tasks Can Go Cheap works through how to decide which tasks tolerate the smaller model, and /docs/guides/ab-testing is the configuration surface.

The trade is honest in both directions: a trained router adapts per query and a fixed split does not. A fixed split is reproducible and a trained router is not. Pick the one whose failure mode you can live with.

Reproducing a decision three weeks later

This is the scenario that sends teams looking for something else, so it is worth walking through concretely.

A customer reports that answers got worse "sometime last month." You need to answer four questions: which model handled their requests, when the mix changed, what it cost, and whether anything else changed at the same time.

With a recommendation layer, the model choice lives in the router's session records and the actual call lives in your own logs, if you logged it. Joining those two is work, and the join key has to be something you thought to record before the incident.

With the gateway in the path, all four answers come from the same request log — model, variant, key, team, cost, latency and outcome on one row, queryable by time range and by customer. What belongs on that row, and specifically what should be redacted before it lands, is in What an LLM Request Log Should Contain; the change history for keys, budgets and guardrails is a separate query described in Who Rotated That Key?.

The general principle: the component that made the decision should be the component that recorded it. Splitting those across two vendors is how a half-hour question becomes a two-day one.

What NotDiamond genuinely does better

The router itself. We do not ship a trained per-query classifier and we are not planning to. If your central belief is that model selection should be learned from data rather than authored by an operator, that belief is not silly and Not Diamond is a serious implementation of it, including the option to train a custom router on your own traffic.

Prompt optimization. Automatic prompt optimization is a first-class part of their product and it is not part of ours. Prompt management — versioning, rollback, A/B — is a different thing, and if the automatic-improvement half is what you want, that is a real reason to keep them.

The price. $0.05 per million tokens routed is a small number for a decision that can move a large one. It is one of the cleanest value propositions in this category.

No catalog constraint. Because you supply the keys and make the call, Not Diamond can route to any model you can reach, including ones outside our catalog. If your candidate list includes a self-hosted model or a provider we do not carry, that is dispositive and no amount of governance makes up for it.

Switch cost, and the thing that does not carry over

The mechanical part is small. Drop the router call, point an OpenAI-compatible client at the gateway, and name a model:

- session_id, provider = client.model_router.select_model(
-     messages=messages,
-     llm_providers=["openai/gpt-5.5", "anthropic/claude-sonnet-4-5-20250929"],
-     tradeoff="cost",
- )
- resp = call_provider_yourself(provider, messages)   # your keys, your dispatch

+ from openai import OpenAI
+ gw = OpenAI(base_url="https://api.nrouter.ai/v1",
+             api_key=os.environ["NROUTER_API_KEY"])
+ resp = gw.chat.completions.create(model="gpt-5-mini", messages=messages)

The part that does not carry over is the learned policy. Whatever the router worked out about your traffic — that these prompts are fine on a small model and those are not — is encoded in a model you cannot export into a routing rule. You are re-deriving it, deliberately, as an A/B split you can read.

That is a genuine cost of switching and it should be planned as one. The practical approach is to run the split alongside the router for a fortnight before cutting over, so the comparison is measured rather than asserted.

What you stop building

The router leaves these to you. The gateway does not:

• Spend ceilings that stop a call. Org, team, user and key scopes, enforced before the request goes out — set up per /docs/guides/budget-controls and explained in Hard LLM Spend Caps.
• A key per customer, service or agent run, revocable without a deploy.
• Guardrails on the way in and out — PII redaction, pattern blocks — rather than a middleware you wrote and now maintain.
• Fallback when a provider is down, without retry logic in every caller.
• One cost per request, from the same source as your invoice, instead of reconciling several provider exports at month end.

If none of those five are problems you currently have, the honest recommendation is to keep the router and come back when one of them starts hurting. Adopting a gateway to solve a problem you don't have is how stacks get heavy.

When to stay on NotDiamond

• You are buying the trained router specifically. It is their product; it is not ours. Wanting it is a complete answer.
• Your candidate set includes models we do not serve — self-hosted, fine-tuned, or a provider outside /models.
• Prompt optimization is doing real work for you.
• Your governance is already solved elsewhere. A team with keys, budgets, guardrails and logging already handled needs a routing decision, not a second platform.
• Your traffic is genuinely heterogeneous per query. A workload where every request is a different shape is the case a trained router is built for; a workload of three repeating job types is the case a fixed split handles fine.

The two products can also simply coexist: let the router recommend, and send the resulting call through a gateway that holds the credentials, enforces the ceiling and writes the log. Nothing prevents that, and for some teams it is the correct answer.

Try it

Pay as you go starts at $5 — a card is required, the first $5 is a real charge ($5.20 with the platform fee on top). There is no free tier here.

→ Create an account, mint a key, and set up one deterministic A/B split across two models on the workload you currently route. Two weeks of cost and quality per variant will tell you more than any comparison table, this one included.

Questions go to hello@nrouter.ai or the public nRouter Slack.

See also

• Hash-Based A/B Tests: Same User, Same Model Variant, Every Call — the reproducible alternative to a per-query trained decision.
• Cost-vs-Quality LLM Routing: Which Tasks Can Go Cheap — how to decide which jobs tolerate the smaller model before you split traffic.
• No BYOK: One nRouter Key Instead of Ten Provider Keys — why the .env full of provider keys is the real difference here.
• Virtual Keys vs Master Key: Scoping a Key Per Job — the per-customer attribution a routing layer cannot give you.
• What an LLM Request Log Should Contain — the row that answers "which model served this customer last month."
• LLM routing strategies 2026 — the wider map of routing approaches if you are still choosing an intelligence model.
• Pricing — what the platform fee covers, and the crossover math.

Sources

Verified 2026-08-23. Every Not Diamond claim traces to Not Diamond's own documentation or pricing page. If a page has changed since, email hello@nrouter.ai and we will re-audit and re-date this section.

• Not Diamond pricing — $0.05 per million tokens routed: notdiamond.ai/pricing
• Not Diamond routing quickstart — where NOTDIAMOND_API_KEY is set (the older /docs/quickstart path now 404s; re-checked 2026-08-23): docs.notdiamond.ai/docs/quickstart-routing
• Not Diamond documentation — select_model, llm_providers, tradeoff: docs.notdiamond.ai
• Not Diamond Python SDK — the .env listing NOTDIAMOND_API_KEY, OPENAI_API_KEY and ANTHROPIC_API_KEY together: github.com/Not-Diamond/notdiamond-python
• RouteLLM: Learning to Route LLMs with Preference Data — the strongest published case for the learned-router thesis this post argues against on operational, not accuracy, grounds: arxiv.org/abs/2406.18665
• Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing — per-query routing evaluated on a named dataset: arxiv.org/abs/2404.14618
• FrugalGPT — the cascade alternative to a classifier, and the same measurement problem: arxiv.org/abs/2305.05176
• OpenAI API reference — the Chat Completions contract the migration diff keeps unchanged: platform.openai.com/docs/api-reference
• Official OpenAI client library — the OpenAI(...) constructor the base-URL swap is made on: github.com/openai/openai-python
• nRouter pricing: /pricing
• nRouter model catalog: /models