Budget Controls

NemoRouter provides granular budget and rate controls to prevent runaway costs and enforce usage policies. Set spending limits and rate controls at the organization, team, or individual key level.

Budget Types

NemoRouter supports two types of controls:

Both can be applied at multiple scopes and work together. A request is blocked if either the budget or rate limit is exceeded.

Spending Budgets

Spending budgets cap how much a key or team can spend over a defined period.

Creating a Budget

1. Navigate to the Budgets page in the dashboard
2. Click Create Budget
3. Configure the budget:

4. Assign the budget to a key or team

Budget Durations

Budget Scope

Budgets can be scoped to different levels:

Setting a budget at the organization, team, or user scope is restricted to org owners and admins (scope-ownership enforcement); key-level budgets can be set by the key's owner.

When a budget is exceeded, requests are blocked. The status code depends on the scope:

• Organization / team / user budgets return 402 with code budget_exceeded.
• Per-key budgets return 429 with code key_budget_exceeded.

{
  "error": "Team budget 'Data Science — Monthly' (monthly) exceeded: $2000.12 used of $2000.00 limit.",
  "code": "budget_exceeded",
  "request_id": "req_..."
}

Budget Examples

Limit a staging key to $50/month:

• Budget Name: "Staging Monthly"
• Max Spend: $50.00
• Duration: Monthly
• Assign to: staging-api-key

Cap the data science team at $2,000/month:

• Budget Name: "Data Science Monthly"
• Max Spend: $2,000.00
• Duration: Monthly
• Assign to: Team "Data Science"

Hard cap for a proof-of-concept:

• Budget Name: "POC Total Budget"
• Max Spend: $100.00
• Duration: Total (never resets)
• Assign to: poc-demo-key

Threshold Alerts

A budget can notify you before it hits the hard cap. Configure soft thresholds (for example 50%, 80%, and 100% of the limit) and each one dispatches an alert to your connected channels when crossed — so heavy spend surfaces while you can still act on it, not only when requests start getting blocked. Wire the destinations (email, Slack, Teams, webhook) under Observability → Alerts → Channels. The hard cap still enforces the block; the thresholds are the early-warning layer on top.

Enforcement Modes

Each budget chooses what happens at the cap. The mode lets you decide between a hard stop and a softer signal:

Rate Limits

Rate limits control the throughput of requests and tokens per minute.

Rate Limit Types

How Rate Limits Work

Rate limits use a sliding window. When a limit is exceeded, the API returns a 429 Too Many Requests response with a Retry-After header:

{
  "error": {
    "message": "Rate limit exceeded. Retry after 12 seconds.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded"
  }
}

Rate Limit Scope

Rate limits can be set at the key level. Higher-tier plans include higher default rate limits:

Custom rate limits set on a specific key override the plan defaults.

Handling Rate Limits in Code

Implement exponential backoff when you receive a 429 response:

import time
from openai import OpenAI, RateLimitError

client = OpenAI(
    api_key="sk-nemo-your-key-here",
    base_url="https://api.nemorouter.ai/v1",
)

def call_with_retry(messages, max_retries=3):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="gpt-4o",
                messages=messages,
            )
        except RateLimitError as e:
            if attempt == max_retries - 1:
                raise
            wait_time = 2 ** attempt  # 1s, 2s, 4s
            print(f"Rate limited. Retrying in {wait_time}s...")
            time.sleep(wait_time)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.NEMOROUTER_API_KEY,
  baseURL: "https://api.nemorouter.ai/v1",
  maxRetries: 3, // The OpenAI SDK handles retries automatically
});

Credits and Budgets

It's important to understand how credits and budgets interact:

• Credits are your account's payment balance. When you buy $100 in credits, you have $100 to spend on API calls.
• Budgets are spending limits that restrict how much of those credits a specific key or team can consume.

For example:

• Your organization has $1,000 in credits
• Team A has a $300/month budget
• Team B has a $500/month budget
• $200 in credits remains unbudgeted (available to keys without budgets)

Budgets do not reserve credits — they only cap spend. If Team A spends $300 and Team B spends $500, the remaining $200 is available to any key.

Monitoring Budgets

Track budget usage from the dashboard:

• Budgets page — View all budgets with current spend vs. limit
• Budgets report at /[organization]/advanced/budgets — The enforcement view (see below)
• Cost report at /[organization]/advanced/cost — Drill into spend by key, team, model, or time period
• API Keys page — See per-key spend at a glance

Spend Limits & Enforcement Panel

The Advanced → Budgets report opens with a live enforcement panel so you can see, at a glance, how close you are to your limits and whether anything is being blocked right now:

• Org spend limit — A meter showing current spend against your org-wide cap (the account backstop). When several org caps exist, the tightest one is shown, since it binds first.
• Top team budget — Your largest team allocation with its live utilization.
• Blocked by enforcement — How many budgets are rejecting requests right now. Every figure on this panel is the same spend the gateway enforces on, so the dashboard reflects exactly what your callers experience.
• Enforcement modes — A breakdown of how many budgets use Block, Warn, and Throttle.

Below the panel, the burn table lists every budget with its utilization bar and an Enforcement badge — Blocking when it's actively rejecting requests, Over · alerting when a warn/throttle budget is past its cap, or its configured mode otherwise.

Best Practices

• Set budgets on all production keys — Even generous ones. They're a safety net against bugs that make runaway API calls.
• Use daily budgets for development — Catch issues early with $10-20/day limits on dev keys.
• Use total budgets for POCs — Hard-cap spend on proof-of-concept projects.
• Monitor the Cost report (/[organization]/advanced/cost) weekly — Look for unexpected spend spikes before they become expensive.
• Separate high-volume and low-volume keys — Don't let a batch pipeline share a key with an interactive chat interface.

FAQ

Who is allowed to create and edit budgets?

Setting a budget at the organization, team, or user scope is restricted to org owners and admins (scope-ownership enforcement). A key-level budget can be set by a member when they create the key, so an individual can cap their own key at creation; editing a key's budget afterward is restricted to owners and admins.

What's the difference between a budget and my credit balance?

Credits are your account's actual payment balance — the money available to spend on API calls. A budget is a spending limit that caps how much of those credits a specific key, user, or team may consume; it never reserves or sets aside credits. So an org with $1,000 in credits can hand a team a $300/month budget while the other $700 stays available to other keys.

What happens when a budget is hit, and which error do I get?

By default the request is blocked at pre-flight before it reaches a provider. Organization, team, and user budgets return 402 with code budget_exceeded; per-key budgets return 429 with code key_budget_exceeded. Both come back with a request_id and a message naming the budget and the amount used vs. the limit.

Can I get warned before a budget starts blocking requests?

Yes. Configure soft thresholds (for example 50%, 80%, 100%) and each one dispatches an alert to your connected channels when crossed, so heavy spend surfaces while you can still act. The hard cap still enforces the block — thresholds are an early-warning layer on top, not a replacement for it. Wire the destinations (email, Slack, Teams, webhook) under Observability → Alerts → Channels.

Do I have to hard-block requests at the cap, or can I just get alerted?

Each budget picks its own enforcement mode. Block rejects requests at the cap (the default, and the right choice for production backstops), Warn fires an alert but keeps requests flowing, and Throttle fires an alert and flags the budget for rate reduction without hard-blocking. Use Warn or Throttle when you want visibility on a high-traffic key without cutting it off entirely.

What scopes can I set a budget at, and can they stack?

Budgets can be scoped to the organization (a backstop on the whole account), a team, an individual user within the org, or a single API key. They can be applied at multiple scopes at once and work together — a request is blocked if it exceeds any budget that applies to it, and when several org-wide caps exist the tightest one binds first.

Do budgets reset automatically?

It depends on the duration you choose. Daily resets at midnight UTC, Weekly resets at midnight UTC each Monday, and Monthly resets on the 1st of each month at midnight UTC. Total never resets — it's a hard lifetime cap, which is what you want for a proof-of-concept or a fixed project budget.

How are rate limits different from budgets, and what sets them?

Budgets cap dollar spend over a period; rate limits cap throughput per minute — RPM (requests per minute) and TPM (tokens per minute). Both are enforced together, and a request is blocked if either is exceeded. Rate limits are set at the key level: higher-tier plans get higher defaults (Enterprise gets custom SLAs), and a custom limit set on a specific key overrides the tier default.

How should my application handle a rate-limit response?

When a rate limit is exceeded the API returns 429 Too Many Requests with a Retry-After header telling you how long to wait. Implement exponential backoff on 429 — the Budget Controls guide shows Python and TypeScript examples, and the official OpenAI SDK can retry automatically via its maxRetries option.

Where do I see current spend and find out what's blocking requests right now?

The Budgets page lists every budget with current spend vs. limit, and the Advanced → Budgets report adds a live enforcement panel showing your org spend meter, top team budget, how many budgets are blocking right now, and a breakdown of Block/Warn/Throttle modes. Because blocks happen at pre-flight (before any provider call, so no spend log is written), the dashboard shows what's blocking now rather than a historical count of rejected requests — to see request-level outcomes, use Request Logs under Reports → Observability.

Next Steps

• Organization Setup — Configure your organization
• Team Management — Create teams and assign budgets
• Authentication — Create and manage API keys