Backend Engineer

The role

You will bootstrap Jan Router, an enterprise LLM gateway, from scratch. You will own the architecture, production deployment, and components ranging from the request proxy to the billing ledger. You will also contribute to the Jan Agent by wiring agent and tool traffic through the routing and metering systems.

What you will do

• Build the core gateway as a unified, OpenAI-compatible API.
• Own provider routing and reliability including load balancing and failover.
• Build billing and metering for token accounting and cost attribution.
• Ship organization controls like API key management, quotas, and RBAC.
• Handle streaming, performance, low-overhead proxying, and caching.
• Contribute to the Jan Agent and connect it to the router.
• Make deliberate speed-versus-correctness calls.

Required qualifications

• Work agent-natively using open harnesses like Pi, Hermes Agent, and OpenCode.
• Acquainted with LLM API systems including providers, streaming, and token accounting.
• Experience with LLM infra such as inference proxies, provider SDKs, or routing services.
• Acquainted with core AI concepts like context windows, inference, prompting, and evals.
• Proven ability to ship from zero to production and own infra, deploy, alerting, and CI/CD.
• Open-minded and pragmatic with ability to handle full ownership and ambiguity.

Nice to have

• A track record building production backend services.
• Experience with payments, billing, metering, or usage-based systems.
• Comfort with high-throughput proxying, gateways, and streaming.
• Solid datastore skills in PostgreSQL, Redis, and queues.
• Strong in at least one of TypeScript or Python.
• Comfort with Docker, Kubernetes, OAuth and OIDC, API keys, RBAC, tenant isolation, and secure secrets handling.
• Familiarity with the Jan.ai ecosystem or other OSS LLM tooling.
• MCP and tool-calling knowledge.
• Go or Rust for high-performance proxying.
• Contributions to open agent tooling or harnesses.

About the company

Jan is an open source AI assistant built by Menlo. It is a general purpose agent that runs open models locally or connects to frontier models.