Member of Product Staff, AI/ML (Evals)

The role

You will lead the product strategy and execution for the evals platform. You will build shared measurement infrastructure and define operating models across product pods to measure AI quality.

What you will do

• Drive product strategy and execution for the evals platform.
• Own the roadmap across the eval lifecycle.
• Build shared measurement infrastructure for defining quality and running experiments.
• Own standards for LLM judges, rule-based evaluators, human annotation, and online monitoring.
• Make model selection fast and routine through a repeatable evaluation way.
• Own the operating model across pods and define eval gates.
• Work with Data Engineering on the de-identification pipeline.
• Work with Data Science on bootstrapping judge quality.
• Work with Clinical Science on flagged production cases.
• Work with the agent platform team as workflows go agentic.

Required qualifications

• 5+ years of product management experience with significant ownership of ML powered products or platform systems.
• Deep understanding of how to measure and improve model quality, including evaluation frameworks, annotation pipelines, and benchmark design.
• Strong technical fluency across ML, data pipelines, and distributed systems.
• Experience working closely with ML researchers and engineers to drive impact in production.
• Ability to balance long term architectural investments with near term quality improvements.
• Strong communication skills and the ability to translate complex technical concepts into clear decisions and narratives.
• A track record of delivering high quality products in domains where accuracy, reliability, and trust are paramount.

Nice to have

• Experience building evaluation platforms, ML observability systems, or quality measurement pipelines.
• Worked in clinical, healthcare, or regulated environments with a high bar for accuracy and compliance.
• Worked on specialty specific or domain specific model adaptations.
• Worked on personalization systems, context ingestion frameworks, or ambient intelligence products.
• Experience shipping large scale ML products with human in the loop workflows.

Benefits

• 14 paid holidays
• Flexible PTO for salaried employees
• Accrued time off for hourly employees
• Medical, Dental, and Vision coverage for all full-time employees and their families
• Monthly HSA contributions for High Deductible Health Plan users
• Paid parental leave
• Family forming benefits
• 401(k) matching
• Personal device allowance
• Flexible Spending Accounts (FSA) and Commuter Benefits
• Monthly contributions for fitness, professional development, coworking, and more
• Dedicated access to therapy and coaching

About the company

Abridge is an AI-native clinician intelligence platform designed to help health systems coordinate clinical, financial, and evidence-based decisions. The company has offices in San Francisco, New York, and Pittsburgh.