Data Scientist, Agent

The role

You will own how Lovable measures and improves its AI agent. You will build evaluation systems and experiments to improve success rates and reduce errors.

What you will do

• Define and own metrics for agent quality including success, completion, and error rates.
• Build evaluation systems and experiment frameworks for agent changes.
• Turn agent traces and telemetry into concrete fixes.
• Work directly with the agent engineering team.
• Build tooling and agents that produce evaluations continuously.
• Set the bar for judging agent behavior.

Required qualifications

• Strong SQL and Python.
• Applied statistics.
• Experimentation.
• Experience or strong interest in LLM evaluation and observability: building evals, scoring outputs, tracing agent behavior, and catching regressions.
• Ability to design A/B tests for agent changes where outcomes are noisy.
• Ability to build systems and agents that produce insight continuously.
• Entrepreneurial mindset.
• Ability to thrive in ambiguity.

Before you apply

• Submit your application in English.
• Fill in a short form and jump on an intro call with our recruiting team.
• A call with the hiring manager.
• A take-home case study.
• A Most Impressive Project session.

About the company

Lovable is a software creation platform that allows people to act on problems. It is a generational company building from Stockholm with teams in London, Boston, New York, and San Francisco.