Research Engineer - AI Performance & Kernel Optimization

The role

You will improve and optimize the performance of large-scale language model training and inference stacks. You will work with pretraining and inference teams to design optimized kernels and push limits of throughput, latency, and hardware utilization.

What you will do

• Kernel development and optimization for large-scale ML workloads
• Performance tuning for training and inference stacks across GPUs and other accelerators
• Profiling and eliminating bottlenecks in memory movement, communication, scheduling, and compute utilization
• Optimizing distributed training and inference systems for large MoE models
• Portability and optimization across non-NVIDIA hardware
• Collaboration with research and infrastructure teams

Required qualifications

• Strong engineering aptitude for building reliable, high-performance systems
• Excellent low-level performance intuition and the ability to reason about hardware-software interactions
• Ability to rapidly learn new systems, tools, and hardware environments
• Excellent communication and collaboration skills
• Experience writing highly performant GPU kernels at any level of abstraction–PTX, CUDA, HIP, Triton, or other kernel DSLs
• Experience optimizing ML workloads for large-scale training
• Experience with non-NVIDIA accelerator hardware, such as AMD, AWS Trainium, Google TPU, Qualcomm, ARM, Intel, and custom ASICs
• Strong understanding of distributed training systems and parallelism schemes
• Experience with performance engineering in other demanding parallel computing environments such as HPC, quantitative finance, scientific computing, graphics, compilers, or numerical simulation
• Strong systems intuition around memory hierarchy, bandwidth constraints, kernel fusion, launch overhead, communication overhead, and hardware utilization
• Experience using profiling and debugging tools to drive performance improvements
• Familiarity with infrastructure underlying large-scale training and inference

Nice to have

• Any HPC experience is a strong plus

Benefits

• Comprehensive medical, dental, vision, and FSA plans
• Competitive compensation and 401(k) plan
• Relocation and immigration support on a case-by-case basis
• In-office snacks and meals provided
• Unlimited PTO and company holidays

About the company

Zyphra is an artificial intelligence company based in San Francisco, California.