Notice: Applications for this position are closed.
We are no longer accepting submissions for Backend Engineer — LLM Infrastructure. Historical role requirements are preserved below for reference.
Backend Engineer — LLM Infrastructure
Build high-throughput LLM routing, live cost tracking, and proxy infrastructure using Python, asyncio, and Postgres.
- Location
- Remote — US / EU
- Employment
- Full-time
- Base salary
- $150k – $210k /yr
- Remote policy
- EOR hiring in 11+ countries
Offer depends on experience and location. Equity offered on top of base.
About the role
You will work on the always-in-path FastAPI proxy that sits between every customer request and every provider. That means guardrails, prompt management, A/B testing, cost attribution, rate limits, and streaming.
You will own the parts of the request lifecycle that have to stay correct under real production load.
What you'll do
- Extend the nRouter Backend FastAPI proxy with new guardrails and features
- Own streaming, retries, and provider failover correctness
- Build and maintain cost attribution from x-nr-request-cost
- Profile and tune hot paths — every millisecond is in the user-facing latency budget
- Harden multi-tenancy isolation at the request layer
Required experience
- 5+ years of backend Python in production
- Deep experience with asyncio and high-concurrency services
- Comfortable with Postgres, connection pooling, and query optimization
- Production experience with streaming APIs or proxies
Nice to have (3)
- Prior work on LLM APIs, model gateways, or SSE streaming
- Experience with LLM routing engines or model gateways
- Performance profiling and flame graph literacy
How We Work & Engineering Culture
At nRouter, we build mission-critical AI routing and governance infrastructure designed for high-concurrency production deployments. Our gateway sits directly in the inference request path of modern AI products, where sub-millisecond routing efficiency, strict tenant budget enforcement, and zero data leakage are non-negotiable.
To tackle these demanding distributed systems challenges, we have established an engineering and operational culture rooted in autonomy, craft, and mutual trust.
Async-First Remote Philosophy & Deep Work
We believe great engineers and operators perform their most innovative work in sustained blocks of uninterrupted flow. We intentionally structure our team workflows to minimize context switching, recurring status calls, and synchronous interruptions.
We default to thoughtful, long-form written communication: well-scoped RFCs, detailed pull request descriptions, and transparent issue tracking. With team members distributed across the Americas, Europe, and Asia-Pacific, everyone has the autonomy to design their schedule for peak focus.
Rapid Shipping & Continuous Production Delivery
We prioritize momentum, rapid iteration, and compact, test-driven pull requests over protracted waterfall planning. Our CI/CD pipeline validates every change against unit, integration, and black-box verification suites before automated deployment to staging and production.
Every team member has direct access to production telemetry, distributed request traces, and real-time observability dashboards, empowering you to deploy changes frequently and verify real customer impact with empirical data.
Zero Bureaucracy & Direct Code Ownership
We operate with a flat organization where technical authority rests directly with the engineers writing the code and building the product. We have eliminated bureaucratic approval chains and management overhead.
When you spot a performance bottleneck in our Rust inference gateway, an ergonomic gap in our client SDKs, or an opportunity to streamline model catalog synchronization, you have full ownership to investigate, propose the architecture, build consensus asynchronously, and execute.
Uncompromising Craft & Systems Rigor
Reliability and deterministic behavior are the core pillars of our platform. Enterprises rely on nRouter for multi-cloud failover, tokenomics accounting, and mTLS security, demanding absolute stability under extreme load.
We take immense pride in writing idiomatic, memory-safe Rust and rigorously typed TypeScript. We pair every bugfix with behavioral regression tests and maintain end-to-end black-box certification suites to guarantee enterprise-grade durability.
High Talent Density & Continuous Learning
We keep our team small, agile, and composed of exceptional craftspeople. Every team member raises the collective standard of the organization.
You will collaborate with peers who have deep expertise in systems programming, low-latency networking, cloud security, and product design. We actively invest in your professional growth with generous annual education stipends, technical conference budgets, and dedicated research time for exploring frontier AI models and open-source tooling.
Interview & Hiring Process
We respect your time and believe technical hiring should be transparent, respectful, and reflective of real everyday engineering. We do not conduct artificial trivia puzzles, whiteboard coding hazing, or adversarial gauntlets.
Every stage of our interview loop is designed to emulate how we actually work: async communication, collaborative system design, and practical problem solving. Our entire hiring loop typically concludes in 5 to 10 business days.
- 1
Step 1: Application Review & Work Sample Evaluation
Our engineering and hiring team personally reviews your application, resume, and any public work artifacts you share (such as GitHub repositories, technical blog posts, open-source contributions, or design case studies). We look for demonstrated passion for systems architecture, developer tools, or AI infrastructure, as well as clear technical communication. We provide feedback on every application within 3 to 5 business days.
- 2
Step 2: 30-Minute Intro & Recruiter / Engineering Conversation
A focused, informal 30-minute video conversation with our hiring manager or founding team member. We will discuss your technical background, past systems or projects you are most proud of, and your career motivations. Equally important, this is your dedicated forum to ask detailed questions about nRouter's business trajectory, engineering roadmap, team culture, and architectural decisions.
- 3
Step 3: Technical & Architecture Deep Dive
Depending on the role, we conduct an architecture walkthrough of a real-world system you have previously designed or offer a paid, take-home practical exercise strictly capped at ~3 to 4 hours of effort. We evaluate code clarity, error handling, performance considerations, and test coverage on a realistic scenario closely modeled after our daily challenges. We never ask candidates for uncompensated production labor.
- 4
Step 4: System Design & Cross-Functional Collaboration
A 60-minute collaborative session with future teammates. For engineering candidates, we work through an end-to-end system design challenge—such as architecting a resilient multi-provider failover engine, distributed rate limiting with token buckets, or transactional credit reservation. For non-engineering candidates, we explore real customer scenarios, growth strategy, or product design critique. We look for structured reasoning, thoughtful tradeoffs, and collegial communication.
- 5
Step 5: Fast Offer with Transparent Equity & Compensation
When there is mutual alignment, we move swiftly to extend a competitive offer. Our offers feature top-market base salary, meaningful equity ownership through stock options, and comprehensive health and wellness benefits. We maintain complete transparency regarding our cap table, current company valuation, and option pool terms, ensuring you have all the essential data to make an informed, confident career choice.
Benefits, Compensation & Total Rewards FAQ
We offer comprehensive, transparent total compensation packages designed to take exceptional care of you and your family, regardless of where you live and work around the globe.
How is cash compensation and equity structured at nRouter?
We provide top-tier cash compensation benchmarked to the 90th percentile of high-growth technology companies in your geographic region. In addition to a competitive base salary, every permanent team member receives meaningful equity ownership in the form of stock options with standard four-year vesting and a one-year cliff. We treat every employee as an owner of the company, ensuring that our collective long-term success translates into substantial financial upside for the builders.
How does international health coverage and remote employment work?
We hire globally through our trusted Employer of Record (EOR) partners in over 30 countries across North America, Europe, the United Kingdom, India, Australia, and Singapore. You receive a fully compliant local employment contract with all statutory retirement, pension, and disability contributions, accompanied by premium private health, dental, and vision insurance covering both you and your eligible dependents.
What equipment, home office, and workspace stipends do you provide?
Every new hire receives a top-tier Apple Silicon MacBook Pro (or custom Linux workstation upon request), paired with a $2,500 USD tax-free home office setup stipend to purchase ergonomic chairs, standing desks, 4K displays, and audio/video peripherals of your choice. To support your ongoing productivity, we also provide a $150 USD monthly stipend covering high-speed fiber internet, local co-working space memberships, and home office supplies.
What is your flexible PTO, vacation, and parental leave policy?
We recognize that sustained high-level output demands deliberate rest and recovery. We provide a flexible, self-managed paid time off (PTO) policy with a mandatory minimum of 20 paid vacation days per calendar year, plus all recognized local public holidays. For growing families, we provide 16 weeks of 100% fully paid parental leave for all primary and secondary caregivers following the birth, adoption, or foster placement of a child.
Is there an annual learning, conference, and development budget?
Yes. We are deeply committed to your continuous professional growth. Every team member receives an annual $2,000 USD professional development stipend to attend premier technical conferences, enroll in advanced courses, purchase technical books, or attain industry certifications. In addition, we host internal architecture seminars, guest lectures, and dedicated hackathons to explore emerging AI infrastructure patterns.
How are performance reviews and compensation refreshes handled?
We conduct biannual performance and compensation evaluations to ensure your rewards keep pace with your expanding scope, ownership, and market dynamics. Reviews evaluate technical contributions, collaborative leadership, peer feedback, and regional cost-of-living adjustments. High-impact contributors are proactively recognized with merit-based base salary increases and supplemental equity refresh grants.
Equal Opportunity & Accommodations Commitment
nRouter is proud to be an equal opportunity employer committed to cultivating a diverse, inclusive, and psychologically safe workplace where extraordinary people can do their career-defining work. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, genetic information, disability status, veteran status, or any other characteristic protected by applicable local, national, or international employment laws.
We believe that solving the frontier challenges of AI infrastructure requires perspectives from diverse backgrounds, non-traditional career paths, and varied life experiences. If you have non-traditional credentials, a self-directed engineering background, or domain expertise acquired outside traditional tech hubs, we enthusiastically encourage your application.
We are also deeply dedicated to providing reasonable accommodations for qualified applicants with physical, sensory, medical, or cognitive disabilities, as well as neurodivergent candidates. If you require any reasonable accommodation at any point during our application, interviewing, or onboarding process, please reach out directly to careers@nrouter.ai. Accommodation requests are handled with strict confidentiality by our people operations team and have zero adverse impact on your application or evaluation.
Applications are closed for Backend Engineer — LLM Infrastructure
We are no longer accepting submissions for this position. Please explore our active openings or submit a general inquiry.