ML Engineer (Training Infra), Foundational Models

The role

You will own the infrastructure used to train the next family of foundational models. This involves distributed training at scale, parallelism strategies, GPU kernel work, and throughput and stability optimization.

What you will do

• Build, maintain, and push the limits of the distributed training stack across large GPU clusters.
• Design and implement parallelism strategies including data, tensor, pipeline, sequence, and expert.
• Profile and optimize end-to-end training throughput.
• Write and tune custom GPU kernels using CUDA or Triton.
• Own the reliability of long-running training jobs including fault tolerance and checkpoint integrity.
• Partner with researchers to ensure architectural ideas can be trained efficiently.
• Partner with the data team to prevent pipeline bottlenecks.

Required qualifications

• BS or MS in Computer Science or a closely related technical field (or equivalent demonstrated experience).
• 3+ years of experience building ML training infrastructure or large-scale distributed systems.
• Hands-on experience training large models with distributed training frameworks such as Megatron-LM, DeepSpeed, FSDP, NeMo, or equivalent.
• Experience being on-call for a real pretraining run.
• Deep working knowledge of GPU architecture and the CUDA programming model.
• Knowledge of standard tools for profiling GPU workloads like Nsight or PyTorch profiler.
• Strong PyTorch internals knowledge.
• Ability to read and modify low-level training code.

Nice to have

• Custom CUDA or Triton kernel development with measurable wins on real workloads.
• Direct experience training models at 10B+ parameters or on 1000+ GPU clusters.
• Familiarity with cluster orchestration and job scheduling (Slurm, Kubernetes) at scale.
• Experience with mixed precision (BF16, FP8), quantization-aware training, or other numerical work in production training runs.
• First-author papers or technical reports on training systems, scaling, or model efficiency.

About the company

Sarvam is building the bedrock of Sovereign AI for India by developing a full-stack sovereign AI platform across research, models, infrastructure and applications. The company is backed by Lightspeed, Peak XV, and Khosla Ventures.