Human Data Operations Manager

The role

You will own pipelines that turn human expertise into AI models. You will partner with researchers to source data from vendors and manage task design, contributor workflows, and data quality frameworks.

What you will do

• Translate critical model capabilities and researchers plans into data acquisition strategies.
• Assess vendors capabilities, forecast capacity, and define task instructions.
• Hold vendors accountable for quality, delivery, and cost.
• Build dashboards tracking throughput, turnaround time, acceptance rates, rework, and cost per accepted task.
• Leverage data analytics to investigate bottlenecks and guide improvements.
• Surface vulnerabilities early and turn feedback into concrete changes to subsequent sourcing.
• Work with research to track how data acquisitions improve model capabilities.

Required qualifications

• 4+ years of experience delivering a portfolio of complex programs and prioritizing across changing research needs.
• An understanding of LLM training, evaluation, and prompt design.
• Ability to learn unfamiliar research concepts quickly.
• Proficiency in SQL for querying datasets and building dashboards.
• Proficiency in Python for data analysis, API integrations, and workflow automation.
• Clear written communication.
• Strong stakeholder judgment.
• Willingness to inspect individual tasks and datasets.
• High agency, bias towards action, and the ability to drive impact in fast-paced, dynamic, and ambiguous environments.

Nice to have

• Experience with annotation, expert data collection, human feedback, or AI evaluation

Benefits

• Salary and equity
• Stock options
• Comprehensive medical, dental, vision, and life
• Annual wellness allowance
• Lunch and dinner provided in the office daily
• 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys
• Unlimited paid time off in the U.S.
• 30 days paid time off in the U.K.
• Visa sponsorship support
• Regular off-sites, happy hours, and team celebrations

Before you apply

• This position may require access to technology or source code subject to the U.S. Export Administration Regulations.
• Any offer of employment may be conditioned on the Company's ability to provide access to technology or source code in compliance with U.S. export control laws.

About the company

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. They build open models to help shape the future of AI.