Manager / Senior Manager, AI Safety Data and Evaluation Program

The role

You will build and implement a bespoke safety strategy to identify and mitigate risks from scientific superintelligence. This involves running expert-data and evaluation programs, managing contributors, and executing red-teaming and evaluation campaigns.

What you will do

• Translate scientific briefs into executable plans covering tasks, contributor needs, and milestones.
• Source and coordinate external experts with scientific colleagues.
• Manage contributor onboarding, task assignment, training and support.
• Run expert-data collection, annotation, red-teaming and evaluation programs.
• Establish quality controls covering completeness, consistency, provenance and adherence to rubrics.
• Maintain clear records, versioned guidance and usable datasets.
• Identify questions that require scientific judgment and bring them to appropriate specialists.

Required qualifications

• Experience delivering expert-data, research-operations, evaluation or similarly demanding programs involving specialist contributors.
• A substantial working understanding of AI safety, including how threat models, capability evaluations, red teaming and safeguards inform one another.
• Evidence of applying safety understanding in research, evaluation or expert-data work.
• Strong operational judgment and the ability to manage dependencies, changing requirements and several parallel workstreams.
• A track record of setting measurable quality standards, detecting recurring problems and improving processes.
• Clear communication, careful documentation and the ability to build effective relationships with researchers, experts and vendors.
• Sufficient technical and scientific fluency to understand research briefs and work with subject-matter experts.

Nice to have

• Experience contributing to scientific study design, assessing scientific validity or developing evaluation rubrics.
• Deeper scientific expertise or research experience in biology, chemistry or materials science.
• Data analysis, scripting, annotation platforms or tools for tracking research workflows and data provenance.

Benefits

• Medical, dental, and vision coverage
• Employer-paid life and disability insurance
• Flexible time off with generous company wide holidays
• Paid parental leave
• Educational assistance program
• Commuter benefits, including bike share memberships for office based employees
• Company subsidized lunch program

About the company

Lila Sciences is building Scientific Superintelligence to solve humankind's greatest challenges. The company combines advanced AI models with proprietary AI Science Factory instruments into an operating system for science.