Member of Technical Staff — Training Infrastructure

The role

This person will design, implement, and optimize distributed training systems that scale across thousands of GPUs. The role involves making large-scale training fast, efficient, and reliable for a Large Physics foundation Model.

What you will do

• Design, implement, and optimize distributed training systems that scale across thousands of GPUs
• Research and test parallelization strategies and numerical precision trade-offs across model scales
• Analyze, profile, and debug low-level GPU operations to maximize throughput and hardware utilization
• Build reusable frameworks for checkpointing, fault tolerance, and reproducibility
• Collaborate with researchers to bring novel model architectures from prototype to full scale
• Stay up-to-date on research to bring new ideas to work

Required qualifications

• Demonstrated proficiency with distributed training frameworks and techniques (e.g. FSDP, DeepSpeed, Megatron, Pytorch, JAX/XLA) to train large foundation models
• Strong grasp of state-of-the-art techniques for optimizing training workloads: parallelism strategies, memory optimization, mixed precision, communication overlap
• Ability to profile and debug performance in complex codebases, from framework internals down to kernels and collectives
• Deep understanding of deep learning frameworks (e.g. PyTorch, JAX) and their underlying system architectures

Nice to have

• contributions to open-source ML infrastructure (e.g. PyTorch, Megatron-LM, DeepSpeed, XLA)

About the company

Causal's mission is general causal intelligence. The founding team has built and deployed AI against the physical world at institutions like DeepMind, Waymo, Cruise, Insitro, Nabla Bio, and CERN.