Staff / Senior Staff Machine Learning Engineer, Inference - USA

The role

This person will optimize and serve models for high-volume, realtime workloads. They will take models from research, containerize them, and ensure reliable production serving.

What you will do

• Take a model from the research team, containerize it, and optimize its serving
• Ensure models run reliably in production
• Profile code to maximize performance out of NVIDIA GPUs
• Handle thousands of concurrent connections
• Design benchmarks or prototypes to solve unclear problems

Required qualifications

• PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems
• Proficiency in C++, CUDA, Rust, or highly optimized Python

Nice to have

• Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM
• Model Acceleration. Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding
• Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections
• Public work. Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups

Benefits

• Relocation assistance

About the company

Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. They build first-party speech models, serve LLMs, and run inference for high-volume, realtime workloads.