Response Caching
Exact-match response caching on nRouter: how hits are matched and metered, the x-nr-response-cache header, and how to bypass or disable the cache.
nRouter caches responses by exact match. When a new non-streaming request is byte-identical to one already served — same model, same messages, same parameters — the stored response is returned instead of a new provider call. A cache hit is metered at the cache-read input rate, never free, and every response reports its cache outcome in the x-nr-response-cache header: hit, miss, or bypass.
How matching works
Matching is byte-identical, not semantic. Two requests that differ anywhere — a changed word in a message, a different temperature, an extra tool — are different cache entries, even if they mean the same thing. This keeps a hit safe to serve: the response you get back is the response that exact request produced before.
When caching applies
Caching is on by default for non-streaming requests. Streaming requests are not served from the cache under the default behaviour. The first request for a given payload is a miss and is stored; an identical request after it is a hit.
What a hit costs
A hit settles at the cache-read input rate for the model, the same rate a provider charges for reading cached input. It is never free, and the settled amount appears in x-nr-request-cost like any other request, so cached traffic stays visible in Analytics & Reports and the request log.
Turn it off
- For one request: send
nrouter_cache: falsein the request body. The response is served live and reportsx-nr-response-cache: bypass. - For the organization: Router Settings, Direct tab, the Response Caching toggle.