Staff Software Engineer, AI Gateway

The role

You will set technical direction and build core systems for the Docker AI Gateway. This includes developing model routers, MCP gateways, and control planes to support agent-based development.

What you will do

• Set technical direction for core systems powering the AI Gateway
• Identify gaps in infrastructure, process, or tooling and drive fixes to production
• Define, deliver, and implement the Gateway's technical strategy
• Own high-ambiguity problems like dynamic model selection and cost-based routing
• Act as an expert for secure credential storage, session summarization, and rate limiting
• Build infrastructure to evaluate, replay, and backtest agent and LLM behavior
• Mentor senior and mid-level engineers
• Partner with product, design, and leadership to shape product direction

Required qualifications

• 8+ years of backend engineering experience
• Deep specialization in distributed systems, cloud-native infrastructure, and API design
• Experience building secure, high-throughput services
• Fluency in Go and/or Rust
• Track record of driving fixes across team boundaries
• Strong product mindset
• Ability to operate with wide latitude and significant autonomy
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience

Nice to have

• Prior experience with OpenAI, Anthropic, or similar LLM APIs
• Familiarity with RAG architectures, vector databases, or agent frameworks (e.g., LangChain, AutoGPT, CrewAI)
• Experience with policy engines (e.g., OPA), observability frameworks (e.g., OpenTelemetry), or API gateways
• Understanding of OAuth 2.1, secret management, and cloud IAM systems
• Experience with Kubernetes, Docker, and microservices architecture
• A track record of setting technical standards that other teams or engineers went on to adopt

Benefits

• Remote-first by design
• Flexibility to manage your schedule
• Generous PTO
• Designated quarterly Whaleness Days
• Designated end-of-year Whaleness break
• Home office support
• Technology stipend equivalent to US$100 net per month
• Annual stipend for conferences, courses, certifications, and continued learning
• 16 weeks of paid parental leave after six months of employment
• Equity for all full-time employees
• Medical, retirement, and paid holidays vary by country
• Docker swag

Before you apply

• Docker considers visa sponsorship on a case-by-case basis based on business needs
• Docker may use AI-assisted tools during our recruiting process
• Candidates will be invited to opt in to interview recordings

About the company

Docker is a globally distributed, remote-first team building tools that define how software gets built and delivered. Their products include Docker Desktop, Docker Hub, and Docker Scout.