Member of Technical Staff, Data Flywheel

The role

This person will identify and build signals, data, and feedback loops to improve model performance. They will work across evaluation, human and synthetic data, infrastructure, post-training, and live deployments.

What you will do

• Identify high-value data sources and partnership opportunities
• Bring new data sources online from scoping through integration
• Design and build evaluations, graders, and feedback loops
• Analyze model performance and failure modes to create targeted datasets
• Develop human and synthetic data strategies
• Build infrastructure and pipelines to ingest, inspect, version, and evaluate data
• Collaborate with pre-training, post-training, applied, and partnership teams

Required qualifications

• Degree (BS, MS, or PhD) in Computer Science, Machine Learning, or related discipline, or equivalent practical experience
• Deep technical understanding of LLM training and evaluation
• Hands-on experience in evaluation design, data curation, reinforcement learning, or reward design
• Strong software engineering skills
• Experience building automated data/evaluation pipelines or large-scale ML systems
• Track record of owning high-impact projects end to end
• Ability to navigate ambiguity and adapt quickly as priorities change
• Highly collaborative, action-oriented approach
• High agency and ability to thrive in a fast-paced startup environment

Benefits

• Salary and equity
• Stock options
• Comprehensive medical, dental, vision, and life
• Annual wellness allowance
• Lunch and dinner provided in the office daily
• 22 weeks paid parental leave for all new birthing and non-birthing parents
• Unlimited paid time off in the U.S.
• 30 days paid time off in the U.K.
• Visa sponsorship support
• Regular off-sites, happy hours, and team celebrations

Before you apply

• This position may require access to technology or source code subject to the U.S. Export Administration Regulations
• Offer of employment may be conditioned on the Company's ability to provide access to technology or source code in compliance with U.S. export control laws

About the company

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. They build open models to help shape the future of AI.