Senior / Staff ML Training Optimization Engineer

The role

You will build standardized distributed training frameworks for research and production and drive training toward greater stability and efficiency. You will profile model runtime and memory, find performance bottlenecks, and evaluate emerging technologies for Waabi's training and inference frameworks.

What you will do

• Build standardized distributed training frameworks for research and production.
• Drive training toward new levels of stability and efficiency.
• Profile model runtime and memory to pinpoint performance bottlenecks.
• Identify and evaluate emerging technologies for training and inference frameworks.
• Design new CUDA kernels, quantization-aware training and inference, and compilation/deployment techniques.
• Work with researchers and ML engineers on best practices for optimal resource usage.
• Create and improve tooling and dashboards to ensure broad adoption of your work.

Required qualifications

• MS/PhD, or Bachelor's degree with at least 4 years of industry experience, in Computer Science, Robotics, and/or a similar technical field.
• Solid coding proficiency in Python, C++ or Rust, among other languages.
• Experience with deep learning frameworks such as PyTorch or JAX.
• Skilled in profiling CPU and GPU code with tools such as PyTorch Profiler and NVIDIA Nsight.
• Open-minded and collaborative team player willing to help others.
• Passionate about self-driving technologies, solving hard problems, and creating innovative solutions.

Nice to have

• Experience identifying when custom CUDA kernels are needed, and implementing them.
• Experience with Bazel in a monorepo environment, and integrating third-party packages into dev environments.
• Experience with Kubernetes-based training platforms.

Benefits

• Competitive compensation and equity awards.
• Medical, Dental and Vision coverage.
• Unlimited vacation.
• Flexible hours and work-from-home support.
• Daily drinks, snacks and catered meals when in office.
• Regularly scheduled team building activities and social events, on-site, off-site and virtual.
• World-class facility with a gym, games room, collaborative workspaces and a patio, when in office.

Before you apply

• Waabi asks for reasonable accommodation requests to be raised with the recruiting team during the application or interview process.

About the company

Waabi, founded by Raquel Urtasun, works on Physical AI and autonomous transportation, powering commercial autonomous trucks and robotaxis. It is backed by and partners with leaders in AI, automotive, logistics, and deep tech.