Research Scientist / Engineer – Training Infrastructure

The role

You will build distributed systems to train large-scale multimodal models across thousands of GPUs. You will design, implement, and optimize efficient training systems and advanced parallelization.

What you will do

• Design, implement, and optimize efficient distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Optimize training stability, convergence, and resource utilization across massive clusters.

Required qualifications

• Extensive distributed PyTorch training and parallelisms in foundation-model training.
• Deep understanding of GPU clusters, networking, and storage systems.
• Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.

Nice to have

• Strong Linux systems administration and scripting.
• Experience managing training runs across 100+ GPUs.
• Experience with containerization, orchestration, and cloud infrastructure.

About the company

Luma's mission is to build unified general intelligence that can generate, understand, and operate in the physical world. They believe multimodality is critical for intelligence.