Staff Software Engineer, GPU Inference

The role

This person will productionize and optimize the GPU serving stack. They will work across custom inference APIs, the vLLM serving runtime, the AMD ROCm software stack, and rack-scale AMD GPU infrastructure.

What you will do

• Productionize the GPU inference stack.
• Own GPU operational readiness.
• Drive reliability in production.
• Improve inference performance.
• Optimize model-serving behavior.
• Debug across system layers.
• Ensure numerical correctness.
• Build performance and correctness infrastructure.

Required qualifications

• 8+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
• Strong programming ability in C++ and Python, including experience with multithreading, concurrency, memory management, and performance-sensitive software.
• Hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent internally developed system.
• Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling methodology.
• Experience debugging distributed systems across multiple layers.
• Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency-sensitive services in production.
• Ability to design rigorous benchmarks, interpret noisy performance results, identify bottlenecks, and translate findings into production improvements.
• Strong communication and technical leadership skills.
• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

Nice to have

• Experience with AMD Instinct accelerators and the ROCm ecosystem, including HIP, RCCL, rocprofiler, AMD SMI, AITER, hipBLASLt, Composable Kernel, or related libraries and tools.
• Deep CUDA experience that demonstrates an ability to transfer GPU systems knowledge across accelerator platforms.
• Experience modifying or contributing to vLLM, SGLang, PyTorch, Triton, TensorRT-LLM, or another open-source ML systems project.
• Experience optimizing prefill-heavy or disaggregated prefill/decode inference architectures.
• Understanding of KV-cache transfer, prefix caching, continuous batching, chunked prefill, request scheduling, and memory-aware admission control.
• Experience with multi-GPU and multi-node inference, including tensor parallelism, pipeline parallelism, expert parallelism, RDMA, collective communication, and failure handling.
• Experience optimizing Mixture-of-Experts or multimodal models.
• Knowledge of GPU kernel optimization, operator fusion, graph capture, attention kernels, GEMM tuning, and communication/computation overlap.
• Experience with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4, including validation of their numerical and model-quality effects.
• Experience building numerical-comparison, determinism, model-validation, or performance-regression test systems.

Benefits

• Build a breakthrough AI platform beyond the constraints of the GPU.
• Publish and open source their cutting-edge AI research.
• Work on one of the fastest AI supercomputers in the world.
• Enjoy job stability with startup vitality.
• Simple, non-corporate work culture that respects individual beliefs.

About the company

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. The architecture delivers industry-leading training and inference speeds.