Member of Technical Staff, Inference

The role

You will work at the core of vLLM to optimize how models execute across diverse hardware and architectures. You will push the boundaries of LLM and diffusion model serving.

What you will do

• Optimize how models execute across diverse hardware and architectures
• Push the boundaries of LLM and diffusion model serving
• Work at the core of vLLM

Required qualifications

• Bachelor's degree or equivalent experience in computer science, engineering, or similar
• Deep understanding of transformer architectures and their variants
• Strong programming skills in Python with experience in PyTorch internals
• Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI)
• Ability to read and implement model architectures and inference techniques from research papers
• Ability to contribute performant and maintainable code and debug in complex ML codebases

Nice to have

• Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving
• Familiarity with RL frameworks and algorithms for LLMs
• Experience with multimodal inference (audio/image/video/text)
• Contributions to open-source ML or system infrastructure projects
• Implemented core features in vLLM or other inference engine projects
• Contributed to vLLM integrations (verl, OpenRLHF, Unsloth, LlamaFactory, etc)
• Written widely-shared technical blogs or side projects on vLLM or LLM inference

Benefits

• Medical coverage
• Dental coverage
• Vision coverage

Before you apply

• We sponsor visas on a case-by-case basis

About the company

Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware.