Sr. Staff Machine Learning Systems Engineer

The role

This Senior Staff engineer will own the end-to-end evaluation systems and data pipelines for AI/ML models in a regulated healthcare environment. You will set technical direction for evaluation methodology, design scalable data pipelines, and lead multi-team projects to ensure AI products are effective and safe.

What you will do

• Set technical direction for evaluation systems including metric, judge and scorer design
• Design and scale data pipelines for ingestion, transformation, versioning, labeling and calibration
• Define how to evaluate new AI services from scratch
• Drive multi-team initiatives to replace manual processes with automated gates
• Own adversarial and red-team evaluation as a risk-reduction program
• Work through complex, cross-team technical disagreements
• Originate new approaches and methodology that become reusable standards
• Lead major platform improvements and re-architect core systems
• Build working relationships across ML engineering, data science, platform, clinical, legal, and product
• Mentor other engineers and raise technical and statistical bars

Required qualifications

• 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems
• Hands-on depth in designing and calibrating LLM judges/scorers
• Hands-on depth in building statistically sound regression-testing methodology
• Hands-on depth in measuring agreement against human labels
• Hands-on depth in designing adversarial/red-team evaluation approaches
• Hands-on depth in dataset versioning, feature and benchmark pipelines, labeling and calibration workflows
• Hands-on depth in high-throughput ingestion and transformation systems
• Experience leading multi-team projects to completion
• Track record of mentoring other engineers, including senior ones
• Excellent communication
• Strong Python
• Statistical fluency to design and defend a testing framework

Nice to have

• Experience with Databricks, MLflow, Unity Catalog, or similar data/eval platforms
• Experience building reporting tools for people without direct engineering access
• Prior experience in a regulated industry (healthcare, fintech, life sciences)
• A track record of company-wide talks or write-ups

Benefits

• Competitive salary & equity compensation
• Unlimited PTO
• Company holidays
• Quarterly mental health days
• Comprehensive health benefits including medical, dental & vision
• Parental leave
• Employee Stock Purchase Program (ESPP)
• 401k benefits with employer matching contribution
• Offsite team retreats

About the company

Hims & Hers is a health and wellness platform that provides access to personalized, affordable, and accessible care. It is a public company traded on the NYSE.