Member of Technical Staff - Pre-Training Evals Infra

The role

You will build evaluation infrastructure that researchers use to understand model capabilities and guide training decisions. The role involves evaluation authoring, distributed execution, and results analysis with a focus on pre-training.

What you will do

• Design and evolve evaluation abstractions.
• Build scalable, reliable execution infrastructure.
• Optimize evaluation and inference performance.
• Establish evaluation correctness and reproducibility.
• Build researcher-facing tools.
• Operate and monitor evaluations as training progresses.
• Turn research needs into reusable systems.

Required qualifications

• Strong proficiency in Python, C++, or Rust.
• Ability to reason deeply about concurrency, memory behavior, and performance.
• Experience designing, building, and operating reliable, maintainable distributed systems.
• Working knowledge of model inference and evaluation.
• Ability to investigate complex problems across system boundaries.
• Strong product judgment and experience translating the needs of researchers or other technical users into clear APIs and usable tools.
• Comfort owning ambiguous problems end to end.
• Attention to detail and the persistence to investigate subtle discrepancies in model outputs, evaluation results, and system behavior.

Benefits

• Salary and equity
• Stock options
• Comprehensive medical, dental, vision, and life
• Annual wellness allowance
• Lunch and dinner provided in the office daily
• 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys
• Unlimited paid time off in the U.S.
• 30 days paid time off in the U.K.
• Sponsorship support for visas
• Regular off-sites, happy hours, and team celebrations

Before you apply

• This position may require access to technology or source code subject to the U.S. Export Administration Regulations.
• Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export contr

About the company

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. They build open models to help shape the future of AI.