Member of Technical Staff — Model Optimization and Inference (Experienced)

The role

You will own end-to-end inference optimization across LLMs, audio models, and diffusion-based components. You will implement KV cache strategies, evaluate inference serving frameworks, and build internal optimization tooling.

What you will do

• Own end-to-end inference optimization across the model stack
• Implement and tune KV cache strategies
• Evaluate, deploy, and extend inference serving frameworks
• Profile and benchmark end-to-end latency and throughput
• Build internal tooling for optimization and profiling
• Accelerate diffusion model inference
• Apply and develop quantization techniques
• Work with research and infrastructure to ensure optimized serving

Required qualifications

• Significant hands-on experience with LLM inference optimization
• Proven proficiency with inference serving frameworks like vLLM, SGLang, or TensorRT-LLM
• Experience optimizing diffusion model inference
• Strong Python and PyTorch skills
• A systematic approach to profiling and optimization
• Familiarity with speculative decoding or other inference-time acceleration techniques

Nice to have

• Comfort reading and writing CUDA or Triton kernels
• Hands-on experience with post-training quantization (GPTQ, AWQ, or similar)
• Familiarity with multimodal or streaming inference architectures
• Experience deploying real-time AI systems with hard latency SLAs
• Prior work at an AI lab, inference startup, or on a high-traffic model serving platform
• Contributions to open-source inference frameworks

Benefits

• Health plans including an HDHP with ~$2,000 annual HSA contributions
• $2,000/yr for gym memberships, fitness equipment, training, or classes
• One-time $500 for tech or accessories
• 15 days of PTO
• 10 public holidays
• Office closed for a full week at year-end
• Lunch, drinks, and snacks provided every workday
• Pre-tax commuter benefits up to $340/month
• 401(k) with 4% match
• Unlimited tokens

Before you apply

• We sponsor visas (O-1, H-1B, green card, etc.) from day one

About the company

Nuance Labs is a research company building photorealistic, real-time AI avatars with emotional intelligence. The team consists of PhDs from various universities and industry experience from companies like Apple, Meta, and Amazon AGI.