Research Engineer - Evals

The role

You will design metrics, build pipelines, and generate datasets to measure output quality. You will own the feedback loop from output quality back to model and product decisions.

What you will do

• Design the metrics that define what good output actually means across millions of sites, formats, and edge cases
• Build the pipelines and harnesses that measure quality rigorously and at scale
• Generate and curate the datasets that make evaluation trustworthy
• Own the feedback loop from output quality back to model and product decisions
• Turn did that work into an answer the whole team can act on

Required qualifications

• 4+ years in ML, research engineering, or data-heavy backend, with real evaluation work
• Engineering depth to build real evaluation systems
• Ability to own ambiguous problems where the metric itself has to be invented
• Authorized to work in the US or Canada

Benefits

• Competitive equity
• 15 days mandatory PTO, anything after 24 days is available
• 12 weeks fully paid parental leave
• $100 USD/month wellness stipend
• $1,000 USD/year Learning & Development expense
• Team offsites
• 3 paid months sabbatical after 4 years
• Medical, dental, and vision coverage
• Life and Disability insurance
• Virtual care and health guide
• Mental health support
• Fertility and family building

Before you apply

• Open to candidates already authorized to work in the US or Canada
• In the US, supports H-1B transfers, F-1 OPT and STEM OPT, TN, and E-3
• In Canada, work permit sponsorship considered case by case
• Includes a 40-hour paid work trial

About the company

Firecrawl is a company that turns the web into data AI agents can use via an API. It is a small team building infrastructure for AI companies.