Staff AI Platform Engineer

The role

You will architect and build the AI enablement stack, including GPU inference serving, model gateways, agent harnesses, and context engineering. You will design, build, and operate production systems while working closely with the AI Research team.

What you will do

• Set technical direction and architecture for the AI platform and team
• Own design docs and RFCs and help with build-vs-buy decisions
• Deploy, benchmark, and tune elastic production inference for open-weight models
• Own the model gateway with auth, routing, failover, quotas, and cost attribution
• Design reusable agent harnesses and orchestration primitives
• Build context-engineering services for memory, retrieval, and data discovery
• Instrument the stack with OpenTelemetry traces and service metrics
• Build experiment-tracking and artifact layers
• Design for productization including versioned APIs, security, and deployment
• Communicate with business and product teams about the roadmap

Required qualifications

• 8+ years of software engineering experience
• 4+ years building and operating ML/AI or LLM systems in production
• Staff-level scope and impact
• Production-grade Python
• API and service design
• Distributed systems
• Containers and Kubernetes
• CI/CD
• Testing
• Shipped production agentic systems
• Experience with context engineering: retrieval-augmented generation, embeddings, vector / hybrid search, and memory for agents
• Experience instrumenting services with OpenTelemetry tracing and metrics and operating AI services against SLOs

Nice to have

• Production experience running an LLM gateway or proxy such as SMG or Bifrost, or equivalent experience building an API gateway
• Hands-on experience deploying and tuning LLM inference engines such as vLLM, SGLang, or TensorRT-LLM on GPU infrastructure
• Familiarity with inference optimization: speculative decoding, prefix caching, tensor/pipeline/expert parallelism, disaggregated prefill and decode, GPU profiling
• Experience building evaluation harnesses for LLMs and agents, and experiment-tracking or artifact management systems such as MLflow or Weights & Biases
• Experience taking an internal platform to an external product
• Experience deploying AI systems on-prem or in regulated domains such as defense or aerospace

Benefits

• Pay depends on experience, but we strive to be at the upper end of the salary range
• Health care plan with 100% premium coverage, including medical, dental, and vision
• 401k with 5% matching
• Paid Time Off (uncapped vacation, plus sick and public holidays)
• Flexible hybrid or remote work arrangement
• Relocation assistance for qualifying employees

Before you apply

• US Citizenship may be required for certain project assignments involving security clearance

About the company

Code Metal is the leader in automated software engineering. They use formal methods to provide provably correct code for customers including the U.S. Air Force, L3Harris, RTX, and Toshiba.