Perplexity
Member of Technical Staff (Software Engineer, Enterprise Agent Evals & Efficiency)
San Francisco. Posted 27 days ago
The role
This person will build enterprise evaluation environments and improve agent performance across tools, skills, and models. They will own the improvement loop by defining success, building evaluations, and shipping changes to improve task success, latency, and cost.
What you will do
- Build realistic enterprise evaluation environments with complex workflows and reproducible system state.
- Own evaluation infrastructure, task suites, and automated graders.
- Analyze agent trajectories and production failures to identify root causes and ship improvements.
- Develop reusable skills and improve tool descriptions, schemas, and result handling.
- Build and evaluate model-selection and routing strategies.
- Improve agent efficiency through context management, skill loading, caching, and execution strategies.
- Build regression gates and launch-readiness signals for model, connector, tool, skill, and harness changes.
Required qualifications
- 5+ years of software engineering or machine learning engineering experience.
- Track record of owning complex production systems from design through deployment and operation.
- Strong system design skills for building reliable, scalable infrastructure.
- Strong proficiency in a backend language such as Python, Go, or Rust.
- Hands-on experience building, evaluating, or improving LLM-based applications or agents.
- Deep expertise in at least one of the following: Evaluation infrastructure, benchmark design, and automated grading; Agent harnesses, tool interfaces, and reusable skills; Realistic test environments
- Strong experimental and statistical judgment.
- Sound technical judgment in balancing task quality, latency, reliability, and cost.
- Track record of leading ambiguous, cross-team initiatives from architecture through production adoption.
About the company
Perplexity's Enterprise Agent Evals & Efficiency team makes agents better at completing real work across enterprise connectors, tools, and data.