Staff Software Engineer (AI Evals)

The role

You will evaluate the quality of interactions with modern coding agents. You will assess whether AI-generated responses, reasoning, and engineering judgment align with how a strong engineer thinks.

What you will do

• Evaluate AI-generated coding interactions end-to-end
• Judge whether outputs are useful and correct at a high level
• Assess the quality of explanations and reasoning
• Distinguish between different levels of response quality
• Provide clear, opinionated feedback on what worked and what did not
• Help define what great looks like when interacting with tools like Cursor

Required qualifications

• Staff / Principal-level engineer or equivalent experience
• Strong background in Go, Python, Rust, or TypeScript / JavaScript
• Hands-on experience using OpenAI Codex, Claude Code, or Cursor
• Deep familiarity with modern AI-assisted dev workflows
• Able to evaluate code without needing to fully execute or deeply review every line
• Comfortable giving direct, opinionated feedback
• High bar for what good engineering looks like

Nice to have

• Experience with tools like Cursor or similar AI-first IDEs
• Prior exposure to prompt design or evaluation workflows
• Experience mentoring senior engineers or defining engineering standards

Before you apply

• Take-home evaluation exercise
• Background Check

About the company

g2i