SWE-Bench AI Task Auditor - Freelance AI Trainer Project

The role

Technical specialists in Software Engineering will audit tasks used to train and evaluate AI systems. You will assess if software engineering tasks are technically accurate, realistic, solvable, and reproducible.

What you will do

• Assess whether software engineering tasks are technically accurate, realistic, solvable, reproducible, and supported by reliable tests and evaluation criteria.
• Provide clear, actionable feedback on identified codebase integration issues, test failures, or logic errors within the tasks.

Required qualifications

• Demonstrable professional experience and deep technical knowledge in software engineering
• Experience in complex codebase navigation, real-world application development, and SWE-Bench standards
• Strong analytical and problem-solving skills to test, troubleshoot, and evaluate complex technical scenarios
• Supply a secure computer
• Supply high-speed internet

About the company

agency is sourcing experienced technical specialists in Software Engineering to audit tasks used to train and evaluate AI systems.