Member of Technical Staff (Answer Quality & Evals)

The role

You will build and improve technical foundations for Answer Quality. This includes building shared evaluation infrastructure and a platform to replay and analyze agent traces.

What you will do

• Build shared evaluation infrastructure to run reliable evals and make product and model decisions
• Develop the platform for replaying and analyzing agent traces to reproduce production behavior and diagnose failures
• Build and operate scalable systems for processing, storing, and monitoring interaction, trace, and evaluation data
• Partner with data scientists, engineers, and product teams to turn answer-quality problems into evaluations, analyses, and product improvements
• Operate in a small, high-impact team to shape how Perplexity measures and improves Answer Quality

Required qualifications

• 4+ years of software, data, or machine learning engineering experience shipping and operating production systems
• Strong proficiency in Python and SQL
• Solid fundamentals in system design, data modeling, and distributed systems
• Experience building big-data systems, including distributed compute, large-scale storage, and high-volume pipelines
• Demonstrated ownership of ambiguous technical projects from initial design through production operation
• Ability to work effectively with data scientists, engineers, and product partners

Nice to have

• Experience building evaluation, experimentation, observability, or machine learning infrastructure
• Familiarity with LLM and agent systems, including tool use, execution traces, replay, and simulation
• Experience building on top of large-scale data processing platforms such as Databricks, Snowflake, or ClickHouse

About the company

Perplexity serves tens of millions of users daily with reliable, high-quality answers grounded in an LLM-first search engine and specialized data sources.