Human Data Evals Lead (Remote/US/LATAM)

The role

You will own data initiatives and proposals to AI labs from proposal through pilot delivery. You will design sample packages and benchmarks in collaboration with subject-matter experts.

What you will do

• Turn public benchmarks and eval targets into proposals and sample packages.
• Respond to lab data requests and pilots.
• Design and build sample packages with subject-matter experts.
• Recruit, brief, calibrate, and review a pool of experts.
• Act as a direct point of contact for lab partners on Slack and calls.
• Own pilots end to end including scoping, SOW, staffing, production, QC, and delivery.

Required qualifications

• 5+ years in technical delivery, quality, or program management, with recent experience in AI/ML data, model evaluation, or benchmarking.
• Hands-on experience delivering data or evaluation work to AI labs or enterprise ML teams, scoping through delivery.
• Working fluency with how frontier models are evaluated: benchmarks, rubrics, pass rates, headroom, and what makes a task discriminate a model.
• Proven people/vendor leadership, you've recruited, calibrated, and held a team or expert pool to a quality standard.
• Fluent English.

Nice to have

• Spanish

About the company

Anyone AI is a small team where this role is the operational center of the Human Data Division.