Machine Learning Engineer, Ops

The role

You will own the production inference stack for text-to-speech and speech recognition models. You will productionize research model checkpoints as low-latency, real-time streaming services running on GPUs.

What you will do

• Build and optimize streaming inference engines for TTS and ASR models.
• Optimize latency and throughput using continuous batching, CUDA Graphs, torch.compile and mixed precision.
• Maintain numerical parity with the reference implementation through automated accuracy regression tests.
• Deploy and operate GPU inference services on Kubernetes.
• Build CI/CD for model artifacts with model versioning and reproducible releases.
• Own observability and performance benchmarking.
• Partner with research to productionize new models.

Required qualifications

• Hands-on experience deploying, scaling and optimizing ML models in production.
• Working knowledge of the full inference stack, from CUDA kernels to serving frameworks.
• GPU performance optimization: profiling (Nsight Systems, PyTorch Profiler), memory bandwidth, CPU-GPU synchronization.
• Solid understanding of model serving: continuous batching, KV cache management, request scheduling.
• Production experience with Kubernetes, CI/CD and cloud infrastructure.
• Strong Python and PyTorch.
• Rigorous approach to performance benchmarking.
• Experience with speech or audio models (TTS, ASR, voice conversion).
• Experience with inference frameworks such as vLLM (vLLM-Omni), SGLang, TensorRT-LLM, TensorRT or Triton Inference Server.

Benefits

• Competitive salary
• Generous company equity
• Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina
• 42 days of paid time off, including: 15 PTO days, 10 sick days, 15 company holidays, 2 floating holidays
• Generous parental leave & fertility support
• 401(k) retirement savings plan
• Lifestyle spending account – $500/month
• Complimentary lunch and snacks for in-office employees
• One Medical membership

About the company

Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. Their flagship social AI platform is Cantina.