Reserve-and-Settle: Never Overspend a Credit Balance

The answer: before a request reaches a provider, its maximum plausible cost is placed on hold against your balance. If the hold does not fit, you get 402 and no provider is contacted. After the call, the hold is replaced by the real cost and the difference returns to you. On a routing failure, a connection failure, an upstream error or a timeout, the whole hold returns. A balance never goes negative and every movement is a ledger row you can read.

Credits are the product. Not a metaphor for the product — the actual thing you buy, the thing your spend controls are denominated in, and the thing that has to be exactly right at three in the morning when a batch job is firing four hundred requests a minute at an account with eleven dollars left on it.

Two properties make that possible, and they are absolute rather than aspirational: a balance may never go negative, and every change to it is recorded in a ledger. Everything in this post exists to keep those two sentences true under conditions where the obvious implementation quietly stops keeping them.

The obvious implementation is check-then-call. It is what everybody writes first, it passes every test written by a human sending requests by hand, and it fails the moment two requests arrive at the same time — which is the traffic shape an LLM gateway exists to serve.

The last dollar, ten requests at once

Here is the failure, at the smallest scale where it is still real. An organization has $1.00 of credit. A batch job fires ten requests concurrently, each of which will cost about $0.30.

# The version everyone writes first.
if balance >= estimated_cost:        # 1. check
    response = await provider.call() # 2. act  (200ms – 30s)
    balance -= actual_cost           # 3. debit

Every one of the ten requests reaches line 1 before any of them reaches line 3. Every one of them reads a balance of $1.00 and a cost of $0.30 and concludes, correctly at the moment it looked, that there is room. All ten proceed. All ten are billed by the provider. The balance lands at −$2.00.

Nothing here is a bug in the usual sense. There is no typo, no off-by-one, no missing null check. The code is correct for one request at a time and wrong for two, and the gap between the check and the debit — a provider round trip, so anywhere from 200 milliseconds to half a minute — is a window wide enough to drive a fleet through.

The consequences are not symmetrical, either. An overspend of two dollars is a rounding error. The same race on an account with a $50,000 monthly ceiling and a misbehaving agent is a very different conversation, and the mechanism that produces it is identical.

Why the naive fixes do not fix it

Three repairs get proposed, in roughly this order, and each fails for a different reason worth knowing.

The last row is the trap. A race that fires once a week under load is a race that will be closed as "could not reproduce" twice before someone catches it. Rarity is not safety.

What actually resolves it is noticing that the check and the commitment are the same decision and must not be two steps. If deciding "there is room" and taking that room are one indivisible act, then the second request cannot see the room the first one already took — because by the time it looks, the room is gone.

Reserve, settle, release: the contract

The debit splits into three phases, and the important move is that the money-holding phase happens before the provider is contacted:

reserve   Hold the maximum plausible cost of this call against the balance.
          Indivisible: it either takes the hold or takes nothing.
          If it does not fit → 402 immediately. No provider is contacted.

forward   Send the request upstream.

settle    Replace the hold with the call's real cost.
          The difference returns to your available balance.

release   On any failure path, the entire hold returns.

Stated as promises you can hold us to, rather than as steps:

• A refusal for insufficient credit costs nothing. The 402 happens before any provider connection is opened, so there is no upstream charge and no partial work to pay for.
• You are charged the real number, not the estimate. The estimate exists for the duration of the call and then stops existing. It never appears on your invoice.
• Every request that takes a hold either settles it or releases it. There is no third outcome. A hold that is neither settled nor released would silently shrink your headroom for a call that cost nothing, and we treat that as a defect of the same severity as a negative balance.
• Both ends of every movement are ledgered. The hold, the settlement, the release, the top-up and the platform fee are each a row. Your balance is not a number we maintain and hope is right; it is the sum of those rows.

That last property is what makes the whole thing auditable rather than merely asserted. When finance asks where $4.12 went, the answer is a filter on a ledger, not an investigation — the reading guide is How to Read Your LLM Credit Ledger.

Available balance is your balance minus what is held

One consequence deserves its own section because it is the part that surprises people looking at the dashboard mid-burst.

Available balance = balance − active holds. That is the number every spend decision is made against, and it is the number the Billing page shows you. During a burst of a hundred in-flight requests, your available balance is visibly lower than your balance, and it climbs back as calls settle and their over-reservations return.

balance                $50.00
active holds          − $3.40   ← 100 calls in flight, held at their ceilings
                      ────────
available              $46.60   ← what the next request is checked against

This is exactly how a card authorisation behaves at a hotel or a fuel pump, and for the same reason: the final amount is not known when the decision to allow the transaction has to be made. If you have ever seen a pending charge larger than the receipt that eventually arrived, you have already used this system.

The practical implication is that a burst can be refused with an available balance that looks like it should have covered it. That is the system working. The alternative — allowing the request because the unheld balance covers it — is the race from the top of this post, wearing a nicer interface.

Worked example: $1.00 and ten concurrent calls

Same scenario as the failure above, run through the contract. Balance $1.00. Ten concurrent requests. Each holds $0.30 (its ceiling, not its expected cost).

Three calls run. Seven are refused cleanly and cost nothing. The balance never goes below zero at any observable moment.

Now settle them. Each call actually cost $0.11, because the model produced far fewer output tokens than the requested ceiling:

settle #1   hold 0.30 → real 0.11   returns 0.19
settle #2   hold 0.30 → real 0.11   returns 0.19
settle #3   hold 0.30 → real 0.11   returns 0.19
                                    ───────────
balance     1.00 − (0.11 × 3)     =  0.67
holds                                0.00
available                            0.67

$0.67 available, $0.33 spent, seven clean refusals, zero negative balance, thirteen ledger rows (three holds, three settlements, three releases of the difference, plus the movements that produced the opening $1.00). The over-reservation existed for the duration of three provider calls and then was gone.

Notice what the customer paid: $0.33, the real cost. Not $0.90, the reserved amount. The conservative estimate bought safety during the window and cost nothing afterwards.

Why the estimate is deliberately high

The hold is the maximum plausible cost of the call, not the expected cost. For a chat completion that means the measured input tokens priced at the model's input rate, plus the requested output ceiling priced at its output rate. For most calls that is several times the eventual number.

Over-reserving and under-reserving are not symmetrical mistakes, which is the entire argument:

One of those is an inconvenience and the other is a hole in the product. So the estimate errs high, every time, and the difference comes back within milliseconds of the call completing.

There is one class of request where the ceiling matters a great deal: image, video and audio generation, where a single call can cost orders of magnitude more than a chat completion and where a bad floor is genuinely dangerous. That is treated separately in Multimodal Cost Safety: An Unpriced Image Call Is Never $0.

Edge cases we had to decide

These are the calls without an obvious right answer. Each is the case, the behaviour, and the reason.

1. When the cost of a completed call cannot be determined, we settle at the reserved amount rather than releasing it. This one is counter-intuitive and it is the most important decision in the post. Releasing the hold would make the request free — and free requests are how a gateway leaks money while every dashboard reads green. The provider performed the work and will bill us for it. Settling at the reservation is the conservative reading, it is visible to you (see the headers below), and it never invents a number. The principle behind it is Cost Honesty: We Read the Number, We Don't Invent It.

2. When an output guardrail blocks a completion, we settle rather than release. The provider generated those tokens and charged for them; the fact that policy prevented you from receiving the content does not un-generate them. You are charged for work genuinely performed and you do not receive content that violates your own policy. The alternative — releasing — would make a blocked response a free call, which is the same hole as case 1 with a different door. The policy side is Inline LLM Guardrails.

3. When the upstream call fails, we release the entire hold. A routing failure, a connection failure, a non-2xx from the provider or a timeout all return the full reservation. No provider produced billable work, so no charge exists to settle. Your balance returns to exactly what it was before the request, and the ledger shows the hold and its release as a matched pair rather than as nothing at all.

4. When a request falls back to a second provider, one hold covers the whole attempt. A retry is a second call and could be a second bill; a reservation taken per attempt would double-hold your balance during an outage, which is precisely the moment you can least afford surprise refusals. The hold is taken once per customer request and only the route that actually served it settles. The failover mechanics are Provider Fallback Chains: Surviving an OpenAI Outage.

5. When a client disconnects mid-stream, we settle what was generated. Tokens already produced were paid for upstream whether or not anyone read them. Releasing would let a client cancel its way to free inference. The tokens not yet generated were never held against you beyond the ceiling, and the difference returns as normal.

What you see from the outside

None of the above requires taking our word for it.

On a successful call, the canonical x-nr-* headers report what settled:

The cost header is worth being precise about, because it is where edge case 1 becomes observable. When a cost cannot be determined, x-nr-request-cost is absent — not 0, not null, not an empty string — and x-nr-cost-status carries unpriced. An absent header is a fact your client can branch on. A zero would be a claim that the call was free, which would be false, and which would quietly corrupt every chargeback report built on top of it.

cost   = resp.headers.get("x-nr-request-cost")   # str | None
status = resp.headers.get("x-nr-cost-status")    # "exact" | "unpriced"

if cost is None:
    # Count the call. Do not invent a figure. Reconcile at the ledger.
    unpriced_calls += 1
else:
    attributed_spend += float(cost)

On a refusal for credit, you get 402 and no provider was contacted. A budget refusal looks similar but is a different control with different codes — org, team and user budgets return 402 with budget_exceeded, while a per-key budget returns 429 with key_budget_exceeded. The full table is Handling 429 and 402 Errors From an LLM Gateway, and where each check sits in the request path is The Four Ceilings Every LLM Request Passes.

In the dashboard, the Billing page shows available balance rather than raw balance, and the ledger lists every movement with its request id, so a settlement can be traced back to the exact call that produced it. Top-up behaviour and the loop protections around it are in Auto Top-Up Without Surprise Bills: Threshold, Amount, Cap.

How we prove it rather than assert it

A single-threaded test proves nothing about a concurrency property. Any implementation, including the broken one at the top of this post, passes a test that sends one request at a time.

The check that means something fires a large number of simultaneous requests at a deliberately tiny balance and then asserts two things:

1. The balance was never negative at any observed moment during the run.
2. The sum of every ledger row equals the final balance, to the cent, with no rounding slack.

Some calls succeed, the rest get a clean 402, and the two numbers agree. Run on every build, that catches a regression that reintroduces check-then-act loudly, at the moment it is written, rather than on the night it matters. The general argument for testing against real behaviour rather than a convenient stand-in is Why We Ban Mocks and Demo Mode.

Limits

Honest boundaries, because a money control described without them is a marketing claim.

A hold is not a budget and not a rate limit. It stops you spending money you do not have. It does not stop you spending money you do have on something you did not intend, and it does not stop a runaway loop that has plenty of balance behind it. Those are separate ceilings — Budgets vs Rate Limits is how to choose between them, and RPM and TPM Rate Limiting Per Key, Team, and Org is the velocity half.

Available balance moves during a burst. If you poll the balance and expect it to be stable while a hundred requests are in flight, you will see numbers that look wrong and are not. Read available balance, not raw balance, and read it as a snapshot.

Refusal is per request, not per job. A batch that partially completes and then hits 402 leaves you with some work done and some not. The gateway guarantees you were not overcharged; it cannot make a half-finished batch idempotent for you. That is application-level work, and it is easier if each workload has its own key — Virtual Keys vs Master Key.

Unpriced calls settle conservatively, which can be higher than the real cost. Edge case 1 protects the balance, not your wallet's optimum. If you are seeing unpriced on a model you use heavily, tell us — that is a pricing gap on our side, and hello@nrouter.ai is where it gets fixed.

Try it

Point any OpenAI-compatible client at https://api.nrouter.ai/v1, set NROUTER_API_KEY, and watch the balance rather than reading about it.

# 1. Read the settled cost off a normal call.
curl -i -sS https://api.nrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $NROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4-5-20250929","messages":[{"role":"user","content":"hi"}]}' \
  | grep -iE '^(HTTP|x-nr-)'

# 2. Fire twenty at once and watch available balance dip, then recover.
seq 20 | xargs -P 20 -I{} curl -s -o /dev/null -w '%{http_code}\n' \
  https://api.nrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $NROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4-5-20250929","messages":[{"role":"user","content":"hi"}]}'

Then open the Billing page during the burst. Available balance drops by the sum of the holds and climbs back as each call settles — and the ledger afterwards sums to exactly what the page shows.

Pay as you go starts at $5 — load your first $5 and get a $10 bonus, $15 in API credits, no subscription. → Get started, or read Billing first. Questions belong in the nRouter community.

See also

• How to Read Your LLM Credit Ledger — what each hold, settlement and release row looks like once it is written, and how to reconcile a month.
• Cost Honesty: We Read the Number, We Don't Invent It — why an unpriced call reports no cost header rather than a zero, and what to do with that in your own reporting.
• The Four Ceilings Every LLM Request Passes — where the credit check sits relative to budgets, rate limits and content checks.
• Provider Fallback Chains: Surviving an OpenAI Outage — why a failover attempt must not take a second hold, and what settles when the second route serves the call.
• Handling 429 and 402 Errors From an LLM Gateway — the client-side branch that tells a credit refusal from a budget block from a throttle.
• Auto Top-Up Without Surprise Bills: Threshold, Amount, Cap — keeping the balance above the floor without building a loop that tops up forever.
• Pricing — the $5 minimum, the platform fee charged on top, and the difference between Pay as you go and Pro.

Sources

External standards and vendor documentation referenced above. Verified 2026-08-23. If a linked page has changed and we have not refreshed, email hello@nrouter.ai and we will re-check. All nRouter figures come from nrouter.ai/pricing.

• RFC 9110, HTTP Semantics — status codes: datatracker.ietf.org/doc/html/rfc9110
• MDN, 402 Payment Required: developer.mozilla.org
• Martin Fowler, accounting narrative and ledger patterns: martinfowler.com/eaaDev/AccountingNarrative.html
• OpenAI API pricing: openai.com/api/pricing
• Anthropic pricing: anthropic.com/pricing