Research Lead, Open Benchmarks Grants

The role

This person will shape research bets in open benchmarks and evaluation. They will advance frontier evaluation research with partners and drive adoption with frontier labs.

What you will do

• Shape research bets in open benchmarks and evaluation.
• Identify gaps in how frontier AI is evaluated.
• Review proposals with senior researchers.
• Work with prospective collaborators to strengthen ideas.
• Help decide where to allocate Open Benchmark Grants.
• Advance frontier evaluation research with research partners.
• Work alongside grantees, Research Fellows, and research leads to define benchmark measurements.
• Design tasks and experiments.
• Turn learnings into artifacts for other researchers.
• Work directly with lab researchers to understand measurement needs and help integrate benchmarks.

Required qualifications

• A research track record in evaluation.
• Led or made substantial contributions to benchmarks, evaluation methods, or environments used by researchers or model-development teams.
• Technical judgment about quality and experimental rigor.
• Ability to assess if an evaluation measures what it claims.
• Ability to recognize alternative explanations for a result.
• Ability to design experiments that resolve uncertainty.
• Hands-on technical depth.
• Comfort moving between research questions and implementation.
• Ability to build experiments, inspect tasks and harnesses, debug verifiers, and analyze model behavior.
• The ability to lead through research collaboration.
• Clear technical communication.

About the company

Snorkel AI is the frontier AI data lab, helping teams build the data and environments behind high-performing frontier and agentic AI. Founded out of the Stanford AI Lab in 2019, Snorkel works with leading AI labs and enterprises.