Software Engineer - ML Infrastructure

The role

You will design and build core systems for scalable and efficient training of large models. You will own distributed training, reinforcement learning infrastructure, and inference and evaluation systems.

What you will do

• Partner with researchers to understand and design for their changing workflows
• Build distributed training infrastructure for foundation models on large-scale medical imaging
• Build high-throughput data loading and preprocessing for volumetric and multimodal datasets
• Prototype new ideas and translate them into production-ready code
• Contribute to production serving and deployment pipelines
• Build the reinforcement learning training stack

Required qualifications

• 6+ years of experience designing, building, and operating large-scale distributed systems or infrastructure in production
• 2+ years of experience building ML infrastructure or systems in production
• Strong Python skills
• Expertise in PyTorch or JAX
• Experience and familiarity with the compute, tooling, and workflow needs of large-scale machine learning research
• Experience building infrastructure or platforms specifically for research or machine learning workflows
• Deep experience building and operating Kubernetes and cloud infrastructure at scale
• Experience with distributed training at scale (FSDP, DeepSpeed, or Megatron-style parallelism)
• Prior experience as a technical lead or mentor for other engineers

Nice to have

• Experience operating in a startup or startup-like environment
• Experience building reinforcement learning training infrastructure
• Experience with high-performance inference and serving (vLLM, SGLang, TensorRT, or Triton)
• Experience optimizing inference and serving for large models
• Experience optimizing training performance
• Experience building internal training or experimentation platforms
• Familiarity with vision-language models (VLMs) or multimodal architectures

About the company

Epsilon Health is transforming medical imaging and diagnostics by bringing frontier AI into clinical practice. The company is backed by Jack Altman, Uncork Capital, Renegade Partners, and AlleyCorp.