A provider incident should never
be a storefront incident.
Smart routing fails over to a backup model the instant a provider degrades, autoscaling absorbs the spike, and per-surface budgets keep AI spend predictable from Black Friday to a quiet Tuesday.
Automatic failover · managed autoscaling · one bill
- AutomaticFailoverRetry on a backup model
- 99.9%Uptime SLASame SLA on every tier
- 4 → 0%Platform feeOpenRouter's published rate is 5.5% (verified July 2026)
Route. Detect. Retry. The customer never sees it.
The gateway monitors every provider endpoint. When one degrades or errors, the request retries on a configured backup model — no code change, no manual cutover, no war room. Weighted load balancing spreads traffic across deployments so no single endpoint is a bottleneck under peak load.
- Fallback chains retry on a backup model on error or timeout
- Routing strategies: cost, latency, weighted
- Per-key RPM/TPM caps stop one surface from starving another
- Every routing and failover decision captured in observability
Flash sale — peak volume
Primary model timing out
Provider degradation detected mid-spike
Gateway — automatic
Retry on backup model
Succeeded on the backup model · no code change
Storefront
Customer-visible errors: 0
Recommendations, search, and support stay online
Cost control at volume
Costs settled from the provider response — exact, not estimated. Per-surface budgets for catalog vs. search vs. support.
Observability per surface
Latency and error-rate breakdowns per model and key, exported to Langfuse, Datadog, or S3. See a degradation before a customer does.
Guardrails for shopper data
Prompt-injection detection and PII redaction on support assistants, where customers paste order and contact details.
What we can promise on reliability and data.
- 99.9% uptime SLA on every tier, backed by managed autoscaling infrastructure — failover is automatic, not a runbook.
- Shopper data is protected by PII-redaction guardrails and a configurable data policy — useful for support assistants where customers paste order and contact details.
- SOC 2 Type II is in progress (status as of August 2026); payments run through Stripe (PCI DSS Level 1) so NemoRouter never touches card data.
Planning for a known peak event? sales@nemorouter.ai will scope dedicated capacity and a budget plan with your team ahead of time.
Retail questions, answered
What happens to our storefront if a model provider goes down?
Fallback chains retry the request on a backup model automatically. Because NemoRouter routes across the whole catalog, a single provider incident does not take down recommendations, search, or support — the request fails over and the customer never sees it.
How do we keep AI spend predictable during peak season?
Costs are settled from the provider-reported response, so spend is exact rather than estimated. Give each storefront surface (recommendations, search, support) its own budgeted team. Soft caps fire Slack alerts at 70%, 90%, and 100%; a hard cap returns 402 so a runaway loop during a sale cannot blow the budget.
Can the gateway handle Black Friday traffic?
The gateway runs on managed autoscaling infrastructure and adds a low, predictable p50 overhead. Per-key RPM/TPM caps protect a shared provider pool from one surface starving another. For very large committed volume, a dedicated deployment adds reserved capacity and a residency pin.
Do we have to manage provider API keys?
No. NemoRouter is a fully managed gateway. Your services authenticate with NemoRouter virtual keys only, and we manage every provider relationship. One key, one bill, and the platform fee is a flat 4% pay-as-you-go (0% on Pro) — lower than OpenRouter's own published 5.5% credit-purchase fee (openrouter.ai/pricing, verified July 2026; see /vs/openrouter for the sourced comparison).
Ship storefront AI that survives the spike.
Start in minutes with smart routing, automatic failover, and per-surface budgets, or talk to us about dedicated capacity ahead of a known peak event.
99.9% uptime SLA · automatic failover · platform fee 4 → 0% (OpenRouter's published rate is 5.5%, verified July 2026)