ML Systems Engineer — Inference Acceleration

The role

Optimize the execution and serving of modern AI models on Arago's custom accelerator. Work across kernels, model execution, multi-device distribution, runtime, and inference serving to shape the software stack.

What you will do

• Analyze modern AI workloads and identify kernel-, runtime-, memory-, and system-level bottlenecks on Arago's accelerator.
• Develop and optimize custom kernels, fused operators, and execution strategies to maximize device utilization.
• Design efficient mappings of models and operators across multiple Arago devices, including communication and synchronization strategies.
• Develop inference-serving techniques such as continuous batching, paged KV caches, prefix/context caching, chunked prefill, and prefill/decode interleaving or disaggregation.
• Build profiling, benchmarking, and performance-analysis infrastructure spanning kernels, full models, and serving workloads.
• Work closely with Arago's hardware, compiler, and runtime teams to co-design software abstractions and influence future hardware features based on real model workloads.

Required qualifications

• Strong experience in high-performance ML inference, GPU/accelerator programming, or ML systems engineering.
• Deep understanding of computer architecture, accelerator/GPU execution models, memory hierarchies, parallelism, and performance bottlenecks.
• Experience developing and optimizing custom kernels using CUDA, Triton, ROCm/HIP, or equivalent low-level programming environments.
• Experience with operator fusion, tiling, scheduling, data movement optimization, graph execution, and profiling of compute- and memory-bound workloads.
• Strong understanding of distributed model execution, including tensor, pipeline, sequence, and/or expert parallelism and communication/computation overlap.
• Hands-on experience with modern inference-serving systems such as vLLM, SGLang, TensorRT-LLM, or equivalent, including KV-cache management, continuous batching, paged attention, and prefill/decode sch
• Strong C++ and Python skills.
• Comfort working on a custom accelerator stack where compiler, runtime, kernels, and abstractions are actively being developed.
• English at a proficient level.

Nice to have

• Exposure to or experience with MLIR and MLIR dialects

Benefits

• Competitive cash compensation
• Meaningful stock option plan
• Healthcare coverage (including family-friendly options)
• Pension contributions
• Professional development support
• 25 days of PTO
• Public holidays

About the company

Arago's mission is to re-engineer the foundations of computing from first principles. The company uses proprietary technology that fuses optical and CMOS technologies to deliver an order-of-magnitude increase in performance.