Member of Technical Staff - Research, Inference

The role

You will do hands-on inference research to move cost per token and tail latency. You will own end-to-end research bets and work with engineering to turn techniques into products.

What you will do

• Own end-to-end inference research bets including speculative decoding, quantization, and memory management
• Train custom speculators against real production traffic
• Work with customers to deploy and tune models
• Carry and expand collaborations with outside research labs
• Work with engineering to turn frontier serving techniques into products
• Help shape the research agenda

Required qualifications

• Research-leaning or systems background in LLM inference, with work you can point to
• Fluent in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling
• Record of shipping research or systems that other people build on
• Drive to independently take a research bet from idea to result
• Able to work in person in NYC or San Francisco office

About the company

Modal is building a new infrastructure layer for AI. The company has passed $1B in annualized run rate.