Member of Technical Staff, Coding Agents

The role

You will help define how frontier coding agents are benchmarked and improved. You will partner with AI researchers and software engineers to develop benchmarks, reward systems, and data-quality techniques for AI for Coding.

What you will do

• Create and publish Coding benchmarks that challenge state-of-the-art frontier agents.
• Understand and shape the frontier of coding agents and publish studies of findings.
• Help the world's top AI Labs improve their models for coding.
• Publicly contribute to the field through benchmarks, open-source tools, research, and technical writing.

Required qualifications

• Extremely passionate about building software with agents and regularly tinkering with the latest coding agents.
• Comfort operating in an ambiguous, fast-moving environment with substantial ownership.
• Previous experience working on a Coding benchmark, OR
• Published research on AI for Software / Code-Generation at NeurIPS, ICLR, ICML, COLM, etc., OR
• Extensive software engineering experience or strong GitHub profile of OSS contributions, along with deep knowledge of agent harnesses.

Benefits

• Equity in a fast-growing company
• 401(k) match
• Competitive compensation
• Financial coaching
• Paid parental leave
• Fertility benefits
• Parental coaching
• Medical, dental, and vision
• Mental health support
• $500 wellness stipend
• $2,000 learning stipend
• Commuting support, free lunch, and gym in our SF office

About the company

Handshake's mission is to organize expert human knowledge to advance the AI economy. Handshake AI works with frontier labs on data, evaluation, and post-training challenges.