Member of Technical Staff, Model Efficiency

The role

You will work across the inference stack to improve core performance metrics by identifying bottlenecks and developing optimizations. You will collaborate with modeling and systems teams to accelerate inference.

What you will do

• Work across the inference stack to improve core performance metrics
• Dive deep into model execution to identify bottlenecks
• Develop innovative optimizations
• Collaborate with modeling and systems teams to experiment, measure, and ship improvements
• Build expertise in advanced performance techniques including GPU/CUDA optimizations, kernel-level improvements, and model execution strategies

Required qualifications

• 5+ years of experience writing high-performance, production-quality code
• Strong programming skills in C++ or Python
• Experience working with large language models and familiarity with the LLM inference ecosystem (e.g., vLLM, SGLang, etc.)
• Ability to diagnose and resolve performance bottlenecks across the model execution stack
• A strong bias for action

Nice to have

• Rust/Go
• GPU programming, CUDA, or low-level systems optimization
• Language modeling with transformers (MoE, speculative decoding, KV-cache optimizations)
• Scaling performance-critical distributed systems (e.g., computation, search, storage)

Benefits

• Weekly lunch stipend of $75/£75 or equivalent
• Full health and dental benefits, including a separate budget for mental health
• RRSP matching, 401K, Pension Scheme
• 100% Parental Leave top-up for up to 6 months
• Annual enrichment benefits for arts & culture, fitness/wellness, quality time, and a workspace improvement credit
• Education & learning stipend for conferences, courses, and coaching
• 6 weeks of paid vacation
• Budget for traveling to other offices if remote
• Annual company offsite
• Daily lunch program, snacks, and regular community and social events for those in office
• Co-working benefit for those not near an office
• $500 home office stipend

Before you apply

• May use AI-enabled tools to screen and assess applicants

About the company

Cohere is a security-first enterprise AI company that builds foundation AI models and end-to-end products. It is a global technology company headquartered in Toronto.