Lead Research Scientist/Engineer (Embedded Evaluations)

The role

This role leads or supports embedded evaluations of frontier AI labs to check whether alignment is on track. Work includes weeks or months on-site at a lab, or preparing and supporting those on-site evaluators from the applied research side.

What you will do

• Conduct interviews, review a lab's evidence, gather new evidence and write a report during embedded evaluations at a frontier AI lab
• Execute evaluation playbooks quickly and adapt to unforeseen circumstances
• Prepare on-site evaluators before they are embedded and support them while they are embedded
• Define what evidence would support a safety case and write the playbook for on-site evaluators
• Develop tooling for embedded evaluators, both ahead of time and on demand
• Convert research methodology into turnkey code and procedures
• Bridge fundamental research and on-site evaluation work

Required qualifications

• Strong track record of producing or leading technically rigorous research relevant to advanced AI risks
• Ability to understand modern model training well enough to interrogate a lab's methods and learn unfamiliar pipelines quickly
• Experience leading ambitious technical projects end-to-end under hard deadlines
• Experience managing or closely supervising strong individual contributors
• Ability to extract signal from large, messy datasets and identify anomalies and unexpected model behaviors
• Experience writing about qualitative and quantitative findings for different stakeholders
• Strong conceptual understanding of metagaming, evaluation awareness, deception and reward-seeking
• Ability to use AI to accelerate work and to represent the organisation independently in high-stakes collaborations
• Commitment to the mission of reducing loss-of-control risk from AI
• Right to work in the US for the on-site embedded evaluator team

Nice to have

• Strong software engineering skills (the entire stack uses Python)
• Previous embedded expertise in professional services such as auditing or consulting

Benefits

• Market-competitive salary and equity
• Unlimited vacation
• Unlimited sick leave
• Up to 6 months of paid parental leave
• Comprehensive health, dental and vision insurance
• Retirement savings with competitive employer matching (e.g. 401(k) for US employees)
• Lunch, dinner and snacks provided on workdays
• Paid work trips, including staff retreats, business trips and relevant conferences
• Yearly $1,000 (USD) professional development budget
• Relocation support and visa fees (if applicable)

Before you apply

• Visa sponsorship is offered for the UK and US, but not guaranteed for every role or candidate
• The on-site embedded evaluator team requires the right to work in the US
• Interview process: a screening interview, 3 technical interviews and a final interview with the CEO
• Resumes are screened with AI-powered tools, with a human reviewing all resumes and making final hiring decisions

About the company

Apollo Research works on risks from loss of control, with a focus on deceptive alignment and scheming in AI models. It detects and studies scheming and works on mitigations, and it works closely with multiple frontier AI companies.