Research Engineer, Training & Inference

The role

You will maintain and optimize proprietary reinforcement learning training and serving infrastructure. You will work on maximizing throughput for reinforcement learning and production workflows across the entire stack.

What you will do

• Maintain and optimize proprietary RL training and serving infrastructure.
• Maximize throughput of reinforcement learning system from data generation to model training.
• Optimize inference stack for high-throughput reinforcement learning and low-latency LLM production traffic.
• Tune the inference engine, router, and scheduler.
• Identify and resolve performance bottlenecks within distributed clusters.
• Balance memory constraints with compute-heavy training cycles for multi-billion parameter models.

Required qualifications

• BS in Computer Science or a related technical field, or equivalent industry experience
• 2+ years of relevant, hands-on industry experience
• Proficiency in Python
• Experience building or maintaining components within ML frameworks (e.g., PyTorch, JAX, or TensorFlow)
• Understanding of distributed training concepts and collective communication primitives (e.g., NCCL) OR practical experience deploying and profiling models on GPU-accelerated cloud infrastructure

Nice to have

• MS or PhD in Computer Science, Mathematics, or a related field
• 5+ years of relevant, hands-on industry experience
• Proficiency in C++
• Experience writing or improving kernels (Triton, CuTeDSL, TileLang, CUDA, CUTLASS, ThunderKittens)
• Proven success deploying performant inference at scale using open-source or custom inference engines, routers, etc.
• Direct experience scaling models via FSDP, Tensor Parallelism, or related sharding techniques on multi-node GPU clusters
• Experience designing reinforcement learning systems for high-throughput training and asynchronous data sampling

Benefits

• Unlimited PTO
• 401(k) matching
• 100% employer-paid health, vision, and dental benefits for employees
• 50% coverage for dependents for health, vision, and dental
• Health Savings Account (HSA) available for qualifying health plans

About the company

Harmonic is building a verification engine called Aristotle that uses formal methods and reinforcement learning to check reasoning and results. They are scaling an elite technical team to support hardware and software where failure is expensive.