Research Engineer, Benchmarks

The role

You will design, build and maintain internal benchmarks that test how frontier AI agents perform on domain-specific tasks. You will work with subject-matter experts to turn real workflows into benchmark tasks and check that results match real-world use.

What you will do

• Design, implement and maintain internal benchmarks for evaluating frontier agents on domain-specific tasks.
• Partner with subject-matter experts to translate real-world workflows into benchmark tasks and evaluation criteria.
• Build reliable infrastructure to run models and agents against benchmark tasks at scale.
• Develop metrics and analyses to assess benchmark difficulty, reliability and failure modes.
• Validate whether benchmark results align with real-world evaluations and user needs.
• Write clear technical documentation and reports explaining benchmark methods and results.

Required qualifications

• Two to four years of experience in software engineering, machine learning engineering or research engineering.
• At least two years building AI benchmarks, evaluation infrastructure or agent environments.
• Technical education or strong fundamentals in a technical field.
• Proficiency with Python, Docker and Linux for research or production infrastructure.
• Hands-on experience designing and operating evaluations or environments for AI agents or large language models.
• Experience building reliable model or agent execution infrastructure, developing evaluation metrics or statistical analyses, and validating benchmark performance.
• Ability to work with subject-matter experts and understand workflows across technical or business domains.
• Strong technical writing, attention to detail, independent problem-solving and comfort in an early-stage environment.

Nice to have

• Published research or technical writing on AI benchmarks, evaluation methods or model failure modes.
• Reinforcement learning pipelines, data generation and agent evaluation experience.

Before you apply

• Visa sponsorship is available.

About the company

clera is a small, technical AI evaluation team building rigorous benchmarks for frontier AI agents. The team works with research teams and customers to show how agents perform on realistic workflows.