Associate Director/Director, Inference Engineering

The role

You will lead the team responsible for the model serving platform, including inference APIs, serving frameworks, and deployment systems. You will define how models are served at scale and act as the technical authority connecting research to production.

What you will do

• Own the inference service end to end including API contracts, latency SLAs, throughput guarantees, and rate limits
• Drive serving architecture decisions regarding batching, KV cache, speculative decoding, quantization, and multi-node deployment
• Establish SLOs for latency, throughput, and availability, plus on-call and postmortem practices
• Partner with ML, research, and product engineering to make new models production-ready
• Build systems for model onboarding, deployment safety, evaluation gates, and observability
• Build, mentor, and retain a high-performing engineering team
• Own the serving roadmap

Required qualifications

• Deep understanding of the inference stack for large language models, including serving frameworks, batching strategies, memory constraints, and latency/throughput tradeoffs
• Experience owning a customer- or product-facing API service end to end, including SLAs, incident response, and service reliability
• Track record hiring, mentoring, and retaining senior technical talent, and scaling engineering organizations
• Strong technical judgment across latency, throughput, reliability, cost, and delivery speed
• Demonstrated ability to set technical strategy and execute as priorities change
• Ability to align ML, research, product engineering, and executive stakeholders with competing priorities

Nice to have

• Hands-on experience with speculative decoding, quantization (AWQ, GPTQ, FP8), or model parallelism strategies
• Experience with serving evaluation: output quality regression testing, A/B routing, and shadow deployments
• Familiarity with GPU capacity planning, instance provisioning, utilization, and cloud cost modeling
• Prior work in AI/ML startups or research-heavy organizations

Benefits

• medical, dental, and vision coverage
• employer-paid life and disability insurance
• flexible time off with generous company wide holidays
• paid parental leave
• educational assistance program
• commuter benefits, including bike share memberships for office based employees
• company subsidized lunch program

About the company

Lila Sciences is building Scientific Superintelligence™ to solve humankind's greatest challenges. The company combines advanced AI models with proprietary AI Science Factory™ instruments into an operating system for science.