The short answer: the status code is ambiguous and the code field is not. 429 means either a rate limit (retry with backoff) or a per-key budget block (stop). 402 means either an empty credit balance (top up) or an org, team, or user budget block (raise the cap or wait for the window). Branch on code, and only one of those four is worth retrying.

The short answer

The difference between a resilient LLM client and a fragile one is almost entirely in the error path. Most clients retry blindly: same request, immediate loop, until something gives. That turns a twelve-second throttle into a self-inflicted outage, and it turns a budget block — which will not lift until a window resets or a human acts — into a tight loop that burns your latency budget and achieves nothing.

The fix is small and mechanical. An nRouter error carries a code alongside the HTTP status, and that code, not the status, tells you which of four different conditions you hit. Two of them look identical from the outside if you only read the number.

Status	`code`	Means	Right reaction
`429`	`rate_limit_exceeded`	RPM or TPM window is full	Back off with jitter, honour `Retry-After`, retry
`429`	`key_budget_exceeded`	This key's spend cap is hit	Stop. Surface the key and cap
`402`	`budget_exceeded`	An org, team, or user budget is hit	Stop. Surface the named budget
`402`	insufficient balance	The organization is out of credits	Top up or auto-topup, then resume

Everything below is how to implement that table without accidentally retrying three of its four rows.

When you need this

Your batch job takes hours longer than the model calls justify. Almost always a retry loop spinning against a non-transient rejection. Instrument which code your retries are being issued against and the answer usually falls out in minutes.

Users see a spinner that never resolves. A budget block surfaced as a generic "something went wrong, retrying" is a dead end for the person waiting. The honest message — "this workspace has reached its spending limit" — is both truer and more actionable.

A brief provider hiccup took your whole fleet down. Every client got throttled at the same instant, every client backed off by exactly the same amount, and every client retried at the same instant. That is a thundering herd, and jitter is the one-line fix.

What you need first

A funded organization and a virtual key. Create the key on the Keys page (/[organization]/keys); the sk-nrouter-… value is shown once at creation. See API Key Management.
NROUTER_API_KEY exported, and the base URL https://api.nrouter.ai/v1. Every block below runs as written.
A budget you can deliberately exceed. Attach a $0.01 Total budget to a throwaway key on /[organization]/budgets so you can trigger a real block rather than testing against a mock. Field reference: Budget Controls.
Somewhere to send an alert. Wire a channel under Observability → Alerts → Channels so a budget block pages a human instead of only failing a request — see Alerts & Notifications.

Step 1 — Read the `code`, not the status

Every rejection carries a machine-readable code and, for budgets, a message naming the budget, its window, the amount used and the limit. Parse it once, at the boundary, and hand the rest of your code a typed decision rather than an HTTP number.

import os, random, time
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["NROUTER_API_KEY"],
    base_url="https://api.nrouter.ai/v1",
)

RETRYABLE = {"rate_limit_exceeded"}
STOP = {"key_budget_exceeded", "budget_exceeded"}

def classify(status: int, code: str | None) -> str:
    if status == 429 and code in RETRYABLE:
        return "retry"
    if code in STOP:
        return "policy_stop"
    if status == 402:
        return "out_of_credits"
    if status in (400, 401, 403):
        return "fail_fast"
    return "retry" if status >= 500 else "fail_fast"

The decision tree that function encodes, written out:

response not ok?
 ├─ 429 + rate_limit_exceeded   → backoff with jitter, honour Retry-After, retry
 ├─ 429 + key_budget_exceeded   → STOP, surface the key and its cap
 ├─ 402 + budget_exceeded       → STOP, surface the named org/team/user budget
 ├─ 402 (no budget code)        → top up or auto-topup, then resume
 ├─ 400 guardrail_blocked       → fail fast, the content was rejected
 ├─ 401 / 403                   → fail fast, fix the key or the role
 └─ 5xx / timeout               → backoff and retry

The specs are the reason a code field has to exist at all. RFC 6585 §4 defines 429 Too Many Requests as "the user has sent too many requests in a given amount of time" and says nothing about which limit or whether it will ever clear; RFC 9110 §15.5.3 defines 402 Payment Required and explicitly reserves it for future use, leaving its meaning to each API. Two under-specified numbers, four distinct conditions — the machine-readable code is what closes the gap.

Note the asymmetry that catches people: a key budget arrives as 429 and an org, team, or user budget arrives as 402. Neither is retryable. The scope determines the status, and that mapping is documented in Budget Controls — the same mapping laid out gate by gate in Credits, budgets, rate limits, guardrails.

Step 2 — Retry a rate limit with backoff and jitter

A rate limit is genuinely transient: it clears as the sliding window rolls. The gateway returns 429 with a Retry-After header telling you how long to wait, and the body carries type: rate_limit_error and code: rate_limit_exceeded.

Honour Retry-After when it is present — RFC 9110 §10.2.3 defines it as either a date or a delay in seconds, so parse both shapes. When it is absent, back off exponentially and always add jitter, because without it every client throttled at the same instant retries at the same instant and recreates the burst that caused the throttle. This is not an nRouter convention: OpenAI's own rate-limit guide ships backoff examples and notes that "adding random jitter to the delay helps retries from all hitting at the same time" (rate limits), and AWS makes the same argument at length in Timeouts, retries, and backoff with jitter.

def call_with_backoff(messages, model="gpt-5.4-mini", max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except Exception as exc:
            status = getattr(exc, "status_code", None)
            code = (getattr(exc, "body", {}) or {}).get("code")
            if classify(status, code) != "retry":
                raise
            retry_after = float((getattr(exc, "response", None)
                                 and exc.response.headers.get("retry-after")) or 0)
            delay = retry_after or (2 ** attempt)          # 1s, 2s, 4s, 8s, 16s
            time.sleep(delay + random.uniform(0, delay * 0.25))   # jitter
    raise RuntimeError("rate limited after retries")

The official OpenAI SDK will also retry 429 for you if you set maxRetries, which is fine for a throttle and wrong for a budget block — the SDK cannot see your code field. If you rely on SDK retries, keep the cap low and add your own classification on top.

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.NROUTER_API_KEY,
  baseURL: 'https://api.nrouter.ai/v1',
  maxRetries: 2, // transient only — budget blocks are handled below, not here
});

Upstream providers surface their own headroom, and it is worth reading alongside ours when you are tuning a fleet: Anthropic returns anthropic-ratelimit-requests-remaining and anthropic-ratelimit-tokens-reset on every response (rate limits), and OpenAI enforces separate RPM, RPD, TPM and TPD dimensions, so the limit you hit is not always the one you were watching.

The server-side half of this — how RPM and TPM are computed and at which scope — is in RPM and TPM rate limiting.

Step 3 — Stop on a budget block and surface the scope

A budget block is a policy outcome, not a failure. Retrying it in a loop is a bug wearing the costume of resilience: the answer will be identical on the next call and every call after it until the window resets or someone raises the cap.

Every provider that meters spend ends up with this same shape. Anthropic's monthly spend cap returns HTTP 429 with error type rate_limit_error — the same type as a throttle — but deliberately omits the retry-after header, and the docs warn that "retrying, including the SDKs' automatic retries, fails until access resumes" (rate limits). The lesson generalises: a 429 you cannot distinguish from a throttle is a retry loop waiting to happen, which is why the code field matters more than the number.

The error message is designed to be surfaced rather than swallowed. It names the budget, the duration, the spend and the limit:

{
  "error": "Team budget 'Data Science — Monthly' (monthly) exceeded: $2000.12 used of $2000.00 limit.",
  "code": "budget_exceeded",
  "request_id": "req_..."
}

For a batch job, checkpoint and schedule a resume after the window resets — Daily budgets reset at midnight UTC, Weekly at midnight UTC on Monday, Monthly on the 1st. For an interactive flow, show the user the honest message. For an operations channel, page whoever owns the budget, quoting request_id so the call can be found on /[organization]/logs.

The prevention side is soft thresholds: set an alert at, say, 75% of the cap so the block is preceded by a warning rather than arriving as a surprise. Strategy in How to set hard spend limits.

Step 4 — Treat an out-of-credits 402 as a billing event

A 402 with no budget code means the organization's credit balance cannot cover the request. Retrying changes nothing — the balance is not going to refill on its own unless you have configured it to.

Two paths, and you should have both:

Auto-topup, so the balance refills before it reaches zero. Configure it on the Billing page (/[organization]/billing). The loop-prevention trade-offs — threshold, cooldown, what happens when a charge fails — are in Auto Top-Up Without Surprise Bills: Threshold, Amount, Cap.
A paging path, because auto-topup can itself fail (an expired card, a declined charge). A 402 that is not resolved by auto-topup should reach a human the same way a failed deploy does.

Credits and budgets are different objects and this is where the difference bites: credits are money in the account, budgets are permission to spend a slice of it. The distinction is unpacked in Gateway credits vs prepaid tokens.

Step 5 — Make every retry idempotent

Any retry you do issue must be safe to issue twice. For a plain chat completion this is usually harmless — you want a fresh answer and the second one supersedes the first. It stops being harmless the moment a call has side effects: a tool call that writes to your database, an agent step that advances state, a workflow node that sends an email.

Three rules that keep a backoff loop from double-applying work:

Carry your own idempotency key through the tool layer, not through the model call. The completion is not the side effect; the function your code runs in response to it is.
Record the attempt before you execute the effect, so a retry can detect that the effect already ran rather than inferring it from a response you never received.
Tag retried requests so your own analytics can separate genuine demand from retry amplification. The user field is the cheapest place to carry a stable identifier — see API Consumers.

Agent pipelines make this sharper because one user action fans out into many model calls, any of which can be retried independently. The attribution and safety considerations are covered in Multi-agent cost tracking and LLM routing for AI agent pipelines.

Verifying it worked

Do not ship error handling you have never seen execute. Force each condition on a disposable key and watch your client take the right branch.

Force a rate limit. Set a low RPM on a test key, then fire more calls than that in a minute:

for i in $(seq 1 12); do
  curl -s -o /dev/null -w "%{http_code} " \
    https://api.nrouter.ai/v1/chat/completions \
    -H "Authorization: Bearer $NROUTER_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"gpt-5.4-mini","messages":[{"role":"user","content":"ping"}]}'
done; echo

Force a key budget block. Attach a $0.01 Total budget to that key and send two calls. Confirm the second returns 429 with key_budget_exceeded and that your client does not retry it.
Force an org budget block. Attach a $0.01 Total budget at organization scope and confirm you get 402 with budget_exceeded — a different status for the same class of decision.
Read the headers on a success. Confirm x-nr-request-id is present on every response (it always is, and it is what support asks for), and that x-nr-request-cost carries the settled USD cost:

curl -sS -D - -o /dev/null https://api.nrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $NROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.4-mini","messages":[{"role":"user","content":"ping"}]}' \
  | grep -i '^x-nr-'

Check your retry counters. Whatever metric you emit per branch — retry, policy_stop, out_of_credits, fail_fast — should be non-zero for exactly the branches you just exercised. A counter that stayed at zero means the branch never ran and you have not tested it.

What goes wrong

You retried a budget block. The most common and the most expensive in wall time. It presents as a job that takes an unexplained multiple of its normal duration, with no error surfaced anywhere a human sees.

You treated every 402 as "out of credits". Half of them are budget blocks, and topping up the balance does not clear a team budget. Check for budget_exceeded before you trigger a payment path — otherwise you buy credits you did not need and the requests still fail.

Your backoff has no jitter. A fleet that backs off in lockstep is a fleet that retries in lockstep. Add uniform jitter proportional to the delay; a quarter of the delay is a reasonable default.

You retried a 400. A malformed body will be malformed the second time too, and a 400 carrying guardrail_blocked means content was deliberately rejected — it costs zero credits and it will be rejected again. See Guardrails.

You let the SDK's automatic retries hide the signal. A high maxRetries on the OpenAI SDK will absorb throttles silently, which feels good until you are trying to work out why latency doubled. Keep it low, log every retry, and do your own classification above it.

Try it

Every control that produces these responses — per-team and per-key budgets, RPM/TPM limits, guardrails, alert channels — is included for every nRouter customer on every plan. Plans vary the platform fee (4% on Pay as you go, 0% on Pro at $50/mo or $500/yr) and the default rate limits, never the feature set.

Load the $5 minimum — the platform fee rides on top — then create a throwaway key, give it a $0.01 total budget, and watch your client take the policy_stop branch for real. Start at app.nrouter.ai/signup, or fire a first call from the browser in the Playground.

Stuck on a code you cannot classify? Bring the request_id to the nRouter community.

Sources

Verified 2026-06-10. Status codes, error codes and header names above come from nRouter's own documentation; the HTTP semantics and provider behaviour are cited to their primary sources. If something has drifted, email hello@nrouter.ai and we will correct it.

Standards

429 Too Many Requests, and why it carries no built-in retry semantics: RFC 6585 §4
402 Payment Required, reserved and API-defined: RFC 9110 §15.5.3 · Retry-After grammar: RFC 9110 §10.2.3

Provider behaviour

OpenAI rate-limit dimensions (RPM/RPD/TPM/TPD) and the jitter recommendation: Rate limits
Anthropic's retry-after and anthropic-ratelimit-* headers, and the spend-cap 429 that carries neither: Rate limits · Errors
Why jitter, and how much: Timeouts, retries, and backoff with jitter, Amazon Builders' Library

nRouter

Budget scopes, status codes and reset windows: Budget Controls
Error responses and response headers: Chat Completions API
Authentication failures and key errors: Authentication
Guardrail blocks and their zero-cost property: Guardrails
Python and TypeScript SDK setup: Python SDK · Node.js SDK

429 vs 402 on an LLM Gateway: Which to Retry, Which to Stop