Machine Learning Infrastructure Tech Lead

The role

You will own the systems for high-performance model training and inference. You will spend 80% of your time building, debugging, and optimizing infrastructure and 20% setting technical direction.

What you will do

• Own the technical direction and roadmap for Reducto's ML infrastructure.
• Build and maintain the training and inference stack.
• Optimize model serving including kernels, runtimes, batching, scheduling, and distributed inference.
• Design systems for reliable multi-node, multi-GPU training and inference.
• Improve GPU utilization, latency, throughput, reliability, observability, and cost efficiency.
• Develop benchmarks to identify bottlenecks and guide infrastructure investments.
• Evaluate state-of-the-art advances in training and inference.
• Build tooling and abstractions to help ML engineers move from experiments to production.
• Partner with ML and Platform engineers on architecture, capacity planning, and technical prioritization.
• Raise the engineering bar through design reviews, mentorship, and hands-on technical leadership.

Required qualifications

• 5+ years of experience building production infrastructure, including significant ML systems experience.
• Led complex technical projects from an ambiguous problem through production deployment.
• Strong Python and systems-engineering skills.
• Understand the performance characteristics of modern GPU training or inference workloads.
• Comfortable with Kubernetes and distributed training or serving frameworks.
• Can reason across low-level model performance and higher-level platform architecture.
• Hold yourself to a high bar for quality, precision, and operational reliability.
• Operate well in a fast-changing, high-growth environment.
• Take full ownership from strategy through execution.

Nice to have

• Optimized or implemented CUDA, Triton, or custom model-serving kernels.
• Contributed meaningfully to frameworks such as vLLM, SGLang, PyTorch, TensorRT-LLM, Ray, or related open-source systems.
• Operated distributed inference or training across hundreds or thousands of GPUs.
• Built observability, scheduling, or capacity-management systems for GPU workloads.
• Experience at an early-stage or high-growth startup.
• Care deeply about connecting technical excellence to measurable business impact.

Benefits

• Unlimited PTO
• Daily Lunch
• Commuter Reimbursement
• Comprehensive Insurance, medical, dental, and vision
• Health and Wellness Budget, up to $150 per month
• Parental Leave

About the company

Reducto is the agentic document platform for leading AI teams. It provides a toolkit for working with documents by combining custom in-house and leading frontier models.