Model Evaluation Program Manager

The role

You will lead the team responsible for understanding how robot policies perform and whether new model versions deliver improvements. You will manage workflows, design evaluation plans, and analyze robot behavior and logs.

What you will do

• Manage work flows and priorities for the evaluation team.
• Provide coaching, feedback, and onboarding for team members.
• Translate research questions into evaluation plans with hypotheses and success criteria.
• Partner with researchers to compare model checkpoints and assess capabilities.
• Develop task definitions, scoring rubrics, and test scenarios.
• Measure task success, output quality, cycle time, human interventions, and recovery behavior.
• Standardize setup, resets, scoring, and intervention rules.
• Review robot behavior, logs, and video to distinguish model failures from other issues.
• Maintain consistent benchmarks and track progress across model versions.
• Use data analysis and scripting to improve reporting and partner with engineering on tooling.

Required qualifications

• Experience in robotics, machine learning, applied research, or model evaluation, with direct ownership of evaluation design and analysis.
• Demonstrated experience managing and developing a technical team.
• Ability to define success, control variables, identify gaps in test coverage, and assess improvements.
• Technical fluency to work with researchers on model checkpoints, generalization, data quality, and experimental tradeoffs.
• Comfort using Python or similar tools to analyze results, inspect logs, and improve evaluation workflows.
• Ability to turn ambiguous requests into executable plans and explain findings to research and operations teams.

Nice to have

• Evaluating robot manipulation policies, embodied AI, autonomous systems, or other machine learning systems operating in the physical world.
• Building regression suites, benchmark datasets, failure taxonomies, or evaluation tools, and connecting findings to model development or deployment readiness.

About the company

Dyna Robotics builds general-purpose robots powered by a proprietary embodied AI foundation model. The team comes from Google DeepMind, Meta, and Cruise.