Research Scientist, Benchmarks & Evals

The role

You will build benchmarks to demonstrate the capabilities of a large human data network for frontier AI models. You will design public benchmarks, build private test sets, and run data-value experiments.

What you will do

• Design and ship public benchmarks for image, video and document models
• Build private test sets from real, opt-in data that no model has trained on
• Run data-value experiments: train models with and without Kled data, measure what changes
• Design human evaluation studies and the automatic metrics that track them
• Publish leaderboards and reports that researchers at frontier labs trust
• Find where models fall short and tell our team what data to collect next
• Work directly with the founders and lab buyers to turn results into data deals

Required qualifications

• 5-7+ years ML research or research engineering experience
• A benchmark or evaluation you built that other people actually used
• Strong hands-on experience training and fine-tuning image or video models (PyTorch)
• Deep understanding of experimental design and statistics (controls, ablations, human studies)
• Experience running experiments end to end, from GPUs to final report
• Clear writing for both researchers and buyers

Nice to have

• Publications at NeurIPS, ICML, ICLR or CVPR (datasets and benchmarks especially)
• Experience on an evaluation or data team at an AI lab
• Experience selling or licensing data to AI labs
• Experience with data valuation or data attribution research

About the company

Kled is building the largest opt-in human data network in the world. They filter, standardize, and license data to frontier AI labs and enterprises.