Member of Technical Staff — Model Optimization and Inference (New Grad)

The role

This person will perform end-to-end inference optimization across LLMs, audio models, and diffusion-based components. They will work on KV cache strategies, inference serving frameworks, and latency profiling to enable real-time conversational AI.

What you will do

• Contribute to end-to-end inference optimization across the model stack
• Implement and tune KV cache strategies
• Work with and extend inference serving frameworks
• Profile and benchmark end-to-end latency and throughput
• Build internal tooling for optimization and profiling
• Accelerate diffusion model inference
• Apply quantization techniques to reduce memory footprint and increase throughput
• Work with research and infrastructure to ensure optimized serving for new models

Required qualifications

• BS, MS, or PhD in CS, ML, or a related field
• Strong fundamentals in LLM inference or ML systems
• Exposure to inference serving frameworks
• Strong Python and PyTorch skills
• A systematic approach to profiling and optimization

Nice to have

• Familiarity with CUDA or Triton
• Curiosity about diffusion inference, speculative decoding, quantization, or other inference-time acceleration techniques
• Internship or research experience with LLM inference, ML systems, or model serving
• Contributions to open-source inference frameworks
• CUDA / Triton kernel work
• Publications or research projects in MLSys, model compression, or inference optimization
• Familiarity with multimodal or streaming inference architectures
• Experience with hard latency SLAs in any real-time system

Benefits

• Health plans including an HDHP with ~$2,000 in annual HSA contributions by the company
• $2,000/yr for gym memberships, fitness equipment, training, or classes
• One-time $500 for tech or accessories
• 15 days of PTO
• 10 public holidays
• Office closed for a full week at year-end
• Lunch, drinks, and snacks provided every workday
• Commuter benefits up to $340/month
• 401(k) with 4% match
• Unlimited tokens

Before you apply

• We sponsor visas (O-1, H-1B, green card, etc.) from day one

About the company

Nuance Labs is a research company building photorealistic, real-time AI avatars with emotional intelligence. The team consists of PhDs from various institutions and industry experience from companies like Apple, Meta, and Amazon AGI.