Member of Technical Staff - Evaluations

The role

This person will conduct comparative analysis to understand model capabilities. They will build evaluation systems and develop frameworks for reasoning, alignment, and usefulness.

What you will do

• Conduct critical comparative analysis to advance understanding of model capabilities
• Build and refine evaluation systems and processes
• Develop generalizable evaluation frameworks for reasoning, alignment, and usefulness
• Collaborate with pre-training, post-training, and applied teams
• Push boundaries of measurement from synthetic evals to human feedback and real-world interaction data

Required qualifications

• Strong statistical analysis and experimental design skills
• Familiarity with LLM evaluation methodologies: static benchmarks, human preference evals, and/or agentic tasks
• High agency
• Thrive in a fast-paced startup environment
• Collaborative
• Detail-oriented

Benefits

• Salary and equity
• Stock options
• Comprehensive medical, dental, vision, and life
• Annual wellness allowance
• Lunch and dinner provided in the office daily
• 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys
• Unlimited paid time off in the U.S.
• 30 days paid time off in the U.K.
• Visa sponsorship
• Regular off-sites, happy hours, and team celebrations

Before you apply

• Position may require access to technology or source code subject to U.S. Export Administration Regulations
• Offer may be conditioned on the ability to provide access in compliance with U.S. export control laws

About the company

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. They build open models to help shape the future of AI.