Research Member of Technical Staff- Training Systems

The role

You will own training systems performance end-to-end. You will define how models train at scale to drive efficiency, scalability, and correctness across large-scale multimodal training.

What you will do

• Own training performance end-to-end
• Diagnose and improve performance of large-scale multimodal training
• Build systematic performance attribution
• Drive measurable gains in distributed, compute, and memory efficiency
• Define and evolve parallelism strategies
• Improve execution efficiency through communication scheduling and runtime-level improvements
• Contribute to and extend training frameworks
• Establish source-of-truth performance metrics
• Build tools to identify bottlenecks and track performance
• Develop regression detection

Required qualifications

• Proven track record improving large-scale distributed training performance
• Deep hands-on experience with modern ML stacks
• PyTorch required
• Strong understanding of data / tensor / pipeline parallelism, sharded training (FSDP / ZeRO-style), communication patterns and overlap strategies, and scaling behavior across large GPU clusters
• Strong systems intuition
• Exceptional debugging and measurement ability
• High ownership mindset and comfort in a fast-moving environment

Nice to have

• JAX a plus
• GPU kernel or compiler-level experience (CUDA, Triton, graph capture, operator fusion)
• Experience with multimodal or video training (variable-length sequences, packing/bucketing)
• Experience working on large-scale training frameworks or distributed runtimes
• Familiarity with cluster topology, networking, and large-scale scheduling effects

About the company

Rhoda AI is building the next generation of generalist intelligent robots. They own the full robotics stack from hardware to foundation world models.