Software Engineer - Voice AI (Inference Runtime)

The role

You will own the Baseten Voice AI inference stack from product roadmap to engineering implementation. You will design and operate real-time, large-scale model serving systems for STT, TTS, and voice agent workloads.

What you will do

• Own and lead Voice AI product areas end-to-end from architecture to production operations
• Design, build, and operate real-time, large-scale, high-performance model serving systems
• Drive cross-team collaboration to solve full-stack technical problems
• Mentor teammates through code reviews, design docs, and technical leadership

Required qualifications

• Bachelor's degree or higher in Computer Science or related field
• Proven track record owning production-grade real-time, large-scale systems where tail latency (p99) matters
• Proficient coding abilities in one or more popular programming or scripting languages
• Good taste in product, particularly developer-oriented tools
• Interest in ML/AI infrastructure and willingness to learn
• Strong collaboration and communication skills
• Comfortable using AI coding assistants (e.g., Claude Code, Codex, Cursor) as a daily productivity multiplier

Nice to have

• Experience implementing pipeline-level model runtime optimizations such as dynamic batching, async scheduling, or decode-side throughput improvements
• Experience building developer platforms: SDKs, CLIs, APIs, and self-serve workflows for ML or infrastructure products
• Experience with containerization and orchestration technologies (Docker, Kubernetes), service meshes, or distributed scheduling
• Familiarity with speech/audio ML models (STT, TTS, speech-to-speech)
• Familiarity with model-serving runtimes (vLLM, TensorRT, ONNX)
• Familiarity with systems-level performance profiling across host-device boundaries (e.g. PyTorch Profiler), diagnosing GPU utilization issues
• Exposure to customer-facing engineering: pre-sales prototyping, technical discovery, or working directly with customers to ship solutions
• Python proficiency

Benefits

• Competitive compensation, including meaningful equity
• 100% coverage of medical, dental, and vision insurance for employee and dependents (U.S. only)
• Flexible PTO policy including company wide Winter Break
• Paid parental leave
• Fertility and family-building stipend through Carrot
• Company-facilitated 401(k) (U.S. only)
• Exposure to a variety of ML startups

About the company

Baseten powers mission-critical inference for the world's most dynamic AI companies. They provide a platform that enables companies to bring cutting-edge models into production by uniting applied AI research, flexible infrastructure, and seamless developer tooling.