ML Systems, LLM Post-Training

The role

This ML Systems Engineer will optimize the performance and efficiency of large language model inference for an agentic AI platform. You will implement performance optimizations, build evaluation harnesses, and architect multi-node clusters for training and inference.

What you will do

• Design, deploy, and optimize LLM inference systems across multi-node clusters.
• Implement and benchmark concrete inference optimizations.
• Profile and analyze inference bottlenecks at the systems level.
• Build evaluation harnesses and benchmarking frameworks.
• Collaborate with research scientists to integrate new model architectures and optimizations.
• Investigate and apply emerging techniques from research papers and open-source projects.

Required qualifications

• B.S., M.S., or PhD in Computer Science, Electrical Engineering, or a related field (or equivalent experience).
• Experience with large-scale ML systems, GPU computing, or high-performance inference optimization.
• Strong proficiency in Python and C++/CUDA.
• Hands-on experience with SGLang, vLLM, PyTorch, or similar inference frameworks.
• Deep understanding of GPU architecture, memory hierarchies, and parallel computing paradigms.
• Experience deploying and optimizing LLMs in production: model serving, batching strategies, distributed inference, or quantization.
• Strong systems-level debugging and profiling skills.
• Comfort working at multiple layers of the stack from CUDA kernels to application logic.
• Self-directed problem solver.

Nice to have

• Familiarity with distributed computing frameworks (Ray, multi-node training/inference)

Benefits

• Unlimited PTO
• Full benefits (medical, vision, dental, 401k)
• Two engineering-centric offices with free parking, private gym, and free lunch, drinks and snacks

About the company

alpha-design-ai-inc develops an agentic AI platform used by leading semiconductor companies to design chips.