Machine Learning Engineer — Inference Optimization

The role

You will optimize inference latency, throughput, and cost for large-scale ML models in production. You will profile GPU/CPU inference pipelines and implement techniques like quantization and KV-cache optimization.

What you will do

• Optimize inference latency, throughput, and cost for large-scale ML models in production
• Profile and bottleneck GPU/CPU inference pipelines
• Implement and tune quantization, KV-cache optimization, speculative decoding, and model pruning
• Collaborate with research engineers to productionize new model architectures
• Build and maintain inference-serving systems
• Benchmark performance across hardware and cloud setups
• Improve system reliability, observability, and cost efficiency

Required qualifications

• Strong experience in ML inference optimization or high-performance ML systems
• Solid understanding of deep learning internals
• Hands-on experience with PyTorch (or similar) and model deployment
• Familiarity with GPU performance tuning (CUDA, ROCm, Triton, or kernel-level optimizations)
• Experience scaling inference for real users
• Comfortable working in fast-moving startup environments with ownership and ambiguity

Nice to have

• Experience with LLM or long-context model inference
• Knowledge of inference frameworks (TensorRT, ONNX Runtime, vLLM, Triton)
• Experience optimizing across different hardware vendors
• Open-source contributions in ML systems or inference tooling
• Background in distributed systems or low-latency services

Benefits

• Real ownership over performance-critical systems
• Direct impact on product reliability and unit economics
• Close collaboration with research, infra, and product
• Competitive compensation
• Meaningful equity at Series A

About the company

featherlessai is a team that cares about engineering quality, not hype.