Research Engineer, ML Infrastructure

The role

You will own core systems including distributed training infrastructure, experiment orchestration, data pipelines, and research tooling. You will build systems to support training jobs running across thousands of GPUs.

What you will do

• Build and own systems that run large-scale training jobs reliably across GPU clusters.
• Own infrastructure that runs concurrent coding agent rollouts in VM sandboxes.
• Profile and improve training throughput end to end.
• Design and maintain systems to launch, track, and analyze experiments.
• Build high-throughput, reliable data pipelines for training and evaluation.
• Diagnose and resolve training failures across GPUs, networking, numerics, and data.
• Implement and optimize parallelism strategies including data, tensor, pipeline, and sequence parallelism.
• Anticipate research team needs and build infrastructure before it becomes a constraint.

Required qualifications

• Deep experience building and operating distributed training systems for large models.
• Strong systems engineering fundamentals in distributed systems, networking, and storage.
• Ability to reason about performance across the full hardware-software stack.
• Proficiency in Python and C++.
• Experience with PyTorch or equivalent deep learning frameworks at a systems level.
• Hands-on experience with GPU performance profiling, memory optimization, and compute efficiency.
• Experience implementing or optimizing parallelism strategies for large model training.
• Track record of building tooling and abstractions that accelerate research workflows.
• Strong debugging instincts across complex, distributed systems.
• ML knowledge to engage substantively with researchers.

Nice to have

• A PhD

About the company

Cognition is an applied AI lab building end-to-end software agents and the makers of Devin, the first AI software engineer.