Strategic Project Lead

The role

The Strategic Project Lead will own end-to-end project delivery for client engagements including post-training dataset creation and benchmarking programs. You will act as an operator between clients, creative experts, and internal engineering teams to deliver rigorous results.

What you will do

• Design annotation protocols, rubrics, and evaluation designs for creative and multimodal outputs.
• Manage scope, evaluator staffing, quality routing, timelines, and client relationships.
• Ensure data quality and inter-rater reliability through real-time course correction.
• Perform qualitative and quantitative analysis to produce findings, recommendations, reports, and workshops.
• Partner with engineering to build tooling and create measurable processes.
• Develop case studies and benchmark publications.

Required qualifications

• 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems, or AI and ML evaluation workflows.
• A track record managing several concurrent projects, including annotation, evaluation, or QA work.
• A real point of view on creative quality and what "good" looks like.
• Based in San Francisco, California

Nice to have

• A strong research background, qualitative and quantitative.
• A Master's or PhD in HCI, cognitive science, psychology, design research, or a related field.
• Equivalent depth from industry.
• Statistical methods for evaluation work: inter-rater reliability, sampling, regression, significance testing.
• RLHF, LLM evaluation, or benchmark design.
• Ethnography, UX research, or design-study methods.
• Startup or 0-to-1 experience.

Benefits

• Medical, dental, vision
• 401k matching
• Company laptop on your start date

Before you apply

• Recruiting team interview
• CEO and co-founder interview
• Technical interview, research and methods focus
• VP of Product interview
• Paid case study and presentation

About the company

Contra Labs is a human-centered research lab studying how creatives use AI and how tools can serve creative work. They connect a global network of creative talent with frontier AI labs and product companies.