ML Infra Engineer, Modeling

The role

You will scale and optimize training systems and core model code. You will own infrastructure for large-scale training, including GPU/TPU compute management, job orchestration, and building JAX training pipelines.

What you will do

• Design, implement, and maintain systems for large-scale model training
• Scale JAX-based training across TPU and GPU clusters
• Profile and improve memory usage, device utilization, throughput, and distributed synchronization
• Build abstractions for launching, monitoring, debugging, and reproducing experiments
• Translate research needs into infra capabilities
• Evolve JAX model and training code to support new architectures, modalities, and evaluation metrics

Required qualifications

• Strong software engineering fundamentals
• Experience building ML training infrastructure or internal platforms
• Hands-on large-scale training experience in PyTorch
• Familiarity with distributed training, multi-host setups, data loaders, and evaluation pipelines
• Experience managing training workloads on cloud platforms (e.g., SLURM, Kubernetes, GCP TPU/GKE, AWS)
• Ability to debug and optimize performance bottlenecks across the training stack
• Strong cross-functional communication and ownership mindset

Nice to have

• Hands-on large-scale training experience in JAX
• Deep ML systems background (e.g., training compilers, runtime optimization, custom kernels)
• Experience operating close to hardware (GPU/TPU performance tuning)
• Background in robotics, multimodal models, or large-scale foundation models
• Experience designing abstractions that balance researcher flexibility with system reliability

About the company

Physical Intelligence is developing foundation models and learning algorithms to power robots and physically-actuated devices. The company consists of engineers, scientists, roboticists, and company builders.