ML Engineer - Inference & Model Deployment

The role

A founding ML engineer will turn AI and ML models into fast, reliable production systems. You will deploy models, optimize inference latency and throughput, and scale serving systems.

What you will do

• Deploy and integrate researcher-trained model checkpoints into cloud infrastructure and production pipelines.
• Profile and benchmark model performance to identify latency, throughput, memory, and compute bottlenecks.
• Implement optimization techniques such as quantization, pruning, batching, caching, efficient attention, and precision trade-offs.
• Build scalable multi-GPU inference systems for search, ranking, recommendations, agents, and other AI-powered product experiences.
• Design reliable model-serving architecture that can support millions of users.
• Develop efficient training and fine-tuning workflows including distributed training, mixed precision, and parallelism strategies.
• Work closely with search and engineering teams to make model deployment a smooth part of the development workflow.

Required qualifications

• Have deployed and optimized deep learning models in production environments.
• Have experience with large-scale model serving, multi-GPU inference, or high-throughput inference systems.
• Understand inference optimization techniques such as quantization, pruning, compilation, batching, caching, and memory optimization.
• Have strong instincts for profiling, benchmarking, and debugging model performance.
• Are familiar with efficient attention mechanisms, transformer optimization, or modern LLM/embedding/ranking model infrastructure.
• Have worked with inference frameworks or serving stacks such as SGLang, vLLM, TensorRT, or equivelant.
• Can write clean, production-quality code and integrate ML systems into backend infrastructure.
• Are comfortable with cloud platforms, distributed systems, storage systems, and modern ML training or serving workflows.

Benefits

• Health coverage
• Dental coverage
• Vision coverage
• Paid parental leave
• Relocation support

About the company

HiringCafe.com is building a job search engine that indexes millions of jobs to help people find real opportunities.