Research Scientist – Frontier Evaluations

The role

Research Scientists will design and publish rigorous evaluations for frontier AI systems. The role involves agentic, coding, safety, and expert-domain evaluations.

What you will do

• Lead the end-to-end design, validation, launch, and continuous improvement of frontier AI benchmarks.
• Partner with researchers and domain experts to develop evaluations around model failures and high-priority domains.
• Analyze model capabilities and failure modes using experimental design and statistical methods.
• Build reproducible evaluation systems, including harnesses, graders, and benchmark infrastructure.
• Collaborate with researchers to post-train models and measure performance gains.
• Communicate results through benchmark reports, technical articles, and research papers.

Required qualifications

• Strong record of publishing benchmarks or research papers.
• Clear technical communication and strong scientific writing skills.
• Commitment to experimental rigor, including baselines, ablations, statistical validity, and contamination controls.
• Ability to take an ambiguous evaluation question from initial scoping through a reproducible public release.
• Depth in agentic, coding, and safety evaluations or applied machine learning in an expert domain.

Nice to have

• PhD in a related technical field.
• Research publications at leading conferences or peer-reviewed journals.
• Interest in multidisciplinary research and the creativity to combine methods and insights from AI, engineering, science, and other expert domains.

Benefits

• Medical, Vision, Dental health insurance
• 401(k) with Employer Match
• Daily UberEats Stipend
• Monthly Wellness Stipend

About the company

AfterQuery is an applied research lab curating data solutions for foundation model development. They serve frontier AI labs to deliver data to power models.