Mercor
Tech Lead, Applied AI - Evals
San Francisco. Posted 5 days ago
The role
You will own the evals product end to end by setting architecture and roadmaps. You will lead the product used by labs and enterprises to evaluate models and power APEX launches.
What you will do
- Set the architecture and roadmap and make tradeoff calls in design reviews
- Ship the APIs, data models and UI that labs and enterprises use to run evals
- Build the systems that run every new frontier model on APEX
- Build failure analysis that links failure patterns to the exact steps that went wrong
- Calibrate graders against expert judgment
- Ensure the product stays reliable and compute costs stay in check
- Lead pilots with labs and enterprises
- Hire and mentor engineers, and manage the people on the team
Required qualifications
- Bachelor's Degree in Computer Science or a related field
- 5+ years building and running backend systems in production, with time at senior or staff level
- Strong full-stack skills in a modern backend language (Python, Go or TypeScript) and a frontend framework such as React
- A track record of owning a platform or product area end to end, including leading the team that built it
- Strong product sense that starts from what the user needs
- Experience building data-heavy products: data models, APIs and views over large volumes of results
- Excellent written and verbal communication with engineers, researchers and customers
- Comfort with ambiguity and a habit of shipping iteratively
Benefits
- Bi-annual performance bonus structure
- Generous equity grant vested over 4 years
- Up to $15k Relocation bonus
- $10K housing bonus (if you live within 0.5 miles of our office)
- $1.5K monthly stipend for meals
- Free Equinox membership
- $200 monthly laundry reimbursement
- $200 monthly personal wellness reimbursement
- Health, Dental, Vision insurance
About the company
Mercor builds infrastructure that enables frontier AI labs and enterprises to train, evaluate, and improve AI on real-world professional work. The company is profitable and recently raised a $750 million Series D at a $20 billion valuation.