Senior ML Infrastructure Engineer

The role

The Senior ML Infrastructure Engineer will design, build, and operate production-grade inference infrastructure on RDU architecture. This role involves turning inference techniques into reliable services and owning systems from request scheduling to API surfaces.

What you will do

• Design and productionize advanced inference techniques on RDU to optimize for performance and cost.
• Own SambaNova's integration with vLLM and adjacent serving frameworks.
• Own the public inference API surface exposed through SambaStack and SambaCloud.
• Build and maintain the accuracy verification and regression infrastructure.
• Partner with ML, compiler, runtime, and product teams to take inference features from prototype to production.
• Contribute to technical design discussions, code reviews, and architectural decisions.

Required qualifications

• B.S. in Computer Science, Electrical Engineering, or related field
• 5+ years of industry experience building and operating large-scale distributed systems
• Strong software engineering fundamentals: algorithms, data structures, concurrency, and systems design
• Experience designing and maintaining production services with strict latency, throughput, and availability requirements
• Working knowledge of modern LLM inference techniques and familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang
• Proficiency in Python
• Experience collaborating across teams to deliver complex, system-level engineering solutions

Nice to have

• ideally in ML serving

Benefits

• Base salary
• Equity
• 95% premium coverage for employee medical insurance
• 77% premium coverage for dependents
• Health Savings Account (HSA) with employer contribution
• Dental insurance
• Vision insurance
• Short/Long term Disability
• Basic Life insurance
• Voluntary Life insurance
• AD&D insurance plans
• Flexible Spending Account (FSA) options

Before you apply

• You must submit an application form for each position for which you believe you are qualified.

About the company

SambaNova is a leader in next-generation AI infrastructure, delivering a full-stack inference platform for customers worldwide.