Senior Director of Engineering - Inference Serving

The role

You will lead a software engineering team while remaining a hands-on technical contributor. You will own team growth, technical quality, and the roadmap for the inference serving platform.

What you will do

• Lead, grow, and inspire a software team
• Own hiring for the team
• Drive career development of engineers through 1:1s and feedback
• Set goals, track progress, and remove blockers
• Partner with Product, ML, QA, and DevOps leadership
• Design systems and write code on critical platform initiatives
• Participate in and lead design reviews
• Establish and uphold engineering standards and coding quality
• Identify and resolve performance bottlenecks, reliability gaps, and scaling constraints
• Stay current on emerging technologies and AI inference trends

Required qualifications

• 12+ years of software engineering experience
• 2+ years leading and managing engineering teams
• Ability to manage, develop, and hire engineers while remaining a hands-on technical contributor
• Strong technical depth in backend systems built with Python, Go, or Rust
• Experience with AI inference serving, orchestration, and infrastructure challenges of LLM workloads at production scale
• Proven ability to drive engineering excellence across an engineering team
• Excellent communication skills
• Experience running iterative engineering processes in a fast-paced environment

Nice to have

• Experience with Kubernetes ecosystem tooling, including Operators and Helm Charts
• Background in ML application optimization and productionization
• Experience building or managing teams that deliver cloud-native platforms for enterprise customers
• Familiarity with AI coding assistants and the developer tooling ecosystem
• Experience at a high-growth AI infrastructure company

Benefits

• Base salary plus equity
• 95% premium coverage for employee medical insurance
• 77% premium coverage for dependents
• Health Savings Account (HSA) with employer contribution
• Dental, Vision, Short/Long term Disability, Basic Life, Voluntary Life, and AD&D insurance plans
• Flexible Spending Account (FSA) options like Health Care, Limited Purpose, and Dependent Care
• Headspace subscription
• Gympass+ membership
• One Medical membership
• Counseling services with an Employee Assistance Program

Before you apply

• You must submit an application form for each position for which you believe you are qualified

About the company

SambaNova is a leader in next-generation AI infrastructure, delivering a full-stack inference platform for customers worldwide. The SambaStack team builds the inference serving platform used to host foundation models on RDU hardware.