Callosum
Research Engineer, Evals - Member of Technical Staff
London. Posted 2 months ago
The role
You will design benchmarks and evaluation suites for agentic behavior. You will build methodologies, evaluation infrastructure, and tools to turn raw traces into structured evidence.
What you will do
- Design benchmarks and evaluation suites for agentic behaviour
- Build the methodology as well as the harness
- Red-team evaluations and design sanity checks
- Turn raw traces into structured evidence
- Infer system capabilities from partial evidence and estimate performance
- Build evaluation and observability infrastructure
Required qualifications
- Evidence that you can run research of your own
- Deep hands-on experience with LLMs in agentic settings
- Statistical discipline
- Strong engineering
- Strong communication skills
Nice to have
- Evaluations or benchmarks you built that other people went on to use
- Broad training in empirical method, potentially from a field outside machine learning
- Experience with automated grading and model-based judging
- A published track record in a relevant field
Benefits
- Competitive Salary
- Equity & Ownership
- Private healthcare
- Visa sponsorship
- Relocation benefits
Before you apply
- We offer Visa sponsorship and relocation benefits to hire the best in the world
About the company
Callosum is the Intelligent Systems Company building infrastructure that unifies heterogeneous compute across the full stack. They are engineers and scientists based in London.