Vision-capable models, by tag
Tag-filtered routing pins document requests to vision-capable models. A mismatched tag set returns 400, never a wrong model. Combine vision with long-context for big PDFs.
Use Case · Document Processing
Extract and classify documents at volume.
Document workloads run vision-capable models over scans and PDFs in large batches. NemoRouter keeps every request on a vision model, paces the batch against provider quotas, and tracks cost per job.
batch · job-id invoices-0418
A document batch in flight
Documents4,200 pages
Capabilityvision
Modelgemini-2.5-pro
Cache hits312 repeats
Fallback used2 pages
Job costtracked
vision-onlybatch-pacedcost-tracked
- Vision models
- Tag-filtered
- Batch jobs
- Paced
- Cost per job
- Tracked
- Catalog
- 240+
Requests stay on vision-capable models
Per-key RPM/TPM limits throttle the loop
Spend attributed by job-id metadata
models on Alibaba US, OpenAI, AWS Bedrock, Azure Foundry, Google Vertex AI & Anthropic
Why Nemo for documents
The right model, at volume, with a cost number
Document processing has three constants: it needs vision, it runs in bulk, and it has a budget. NemoRouter handles all three behind one key.
Built for batch workloads
Each page is a standard chat.completions call, so a batch is a loop. Per-key RPM/TPM limits pace it against provider quotas; caching skips identical pages.
Cost & usage tracking per job
Tag the batch with a job id and the dashboard attributes total spend to it. Cost-per-document becomes a measured number, capped by per-team and per-key budgets.
Failover for long batches
A provider degradation halfway through hours of pages shouldn’t mean restarting. The fallback chain retries the next link transparently and the run continues.
How it works
A document batch, end to end
A batch is a loop of standard requests. Each document routes to a vision-capable model, the loop is rate-paced and failover-protected, and total spend rolls up by job id.
Document batch flow
Batch job
job-id in metadata
Thousands of documents — scans, PDFs, images.
Per document
one /chat/completions
Each page is a standard vision request.
Vision model route
catalog · vision-capable
Call a vision-capable model directly, or a weighted alias across them.
Rate-paced + failover
RPM/TPM · fallback
Limits pace the loop; failover survives a provider blip.
Cost per job
spend analytics
Total spend rolled up by the job-id tag.
Tag-filtered routing is the safety rail — a request tagged vision can never silently fall back to a text-only model.
The code
One request per document, in a loop
A document batch is a loop of standard chat.completions calls — image content in, structured fields out. These snippets are generated from the SDK examples the playground uses; add a job-id to metadata to attribute the batch.
Install
pip install openai| 1 | # Cache: enabled (org default). Pass nemo_cache: false to skip. |
| 2 | from openai import OpenAI |
| 3 | import os |
| 4 | |
| 5 | client = OpenAI( |
| 6 | api_key=os.environ["NEMOROUTER_API_KEY"], |
| 7 | base_url="https://api.nemorouter.ai/v1", |
| 8 | ) |
| 9 | |
| 10 | response = client.chat.completions.create( |
| 11 | model="gemini-2.5-flash-lite", |
| 12 | temperature=1, |
| 13 | max_tokens=1024, |
| 14 | messages=[ |
| 15 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 16 | ], |
| 17 | extra_body={ |
| 18 | # "nemo_cache": False, # Uncomment to skip cache |
| 19 | }, |
| 20 | ) |
| 21 | |
| 22 | print(response.choices[0].message.content) |
Tag each call with a job id in request metadata — spend analytics rolls the whole batch up to that job.
FAQ
Common document-processing questions
Does NemoRouter support vision-capable models for document work?
Yes. The model catalog tags vision-capable models, and tag-filtered routing keeps a document request on a vision model. A request that needs vision can never fall back to a text-only model by accident.
Can I run large batches of documents through the gateway?
Yes. Each document is a standard request through the OpenAI-compatible endpoint, so a batch job is a loop of those calls. Per-key RPM/TPM limits pace the batch, fallback chains absorb a provider blip mid-run, and caching skips identical pages.
How do I track the cost of a document-processing job?
NemoRouter reports the real cost of every call. Tag a batch with a job id in request metadata and the dashboard attributes total spend to that job. Cost-per-document is a measured number.
What happens if a provider degrades in the middle of a batch?
The fallback chain retries the next provider transparently. A 5xx or timeout on the primary triggers the next link. The batch keeps running from where it was, and the failover is logged.
Vision, at volume, with a cost number
Process documents without minding provider quotas
Tag-filtered vision routing, batch-friendly rate limits, failover, and per-job cost & usage tracking — all unlocked on every plan.