Software Engineer (AI Quality & Reliability)

The role

You will build automated evaluation and regression systems to measure AI quality. This role focuses on test infrastructure, independent quality evidence, and reliable releases for voice agents and application workflows.

What you will do

• Build shared evaluation frameworks, test harnesses, and versioned datasets.
• Build automated end-to-end regression coverage across conversations, tool calls, and integrations.
• Evaluate task completion, tool-use correctness, latency, and conversation quality.
• Produce release-readiness evidence and maintain quality dashboards.
• Reproduce defects and add durable regression cases.
• Integrate automated checks into CI/CD and reduce flaky tests.
• Partner with domain reviewers on healthcare-specific cases.

Required qualifications

• Strong software engineering skills in Python or TypeScript.
• Experience building maintainable test automation, reusable fixtures, and CI pipelines.
• Practical experience evaluating LLM-powered applications, agents, or voice systems.
• Experience testing APIs, asynchronous workflows, and distributed systems.
• Ability to trace a failure across service boundaries.

Nice to have

• Experience with streaming audio
• Experience with telephony
• Experience with healthcare workflows
• Experience with EHR integrations
• Experience with reliability testing in privacy-sensitive products

Benefits

• Health coverage

About the company

Taxo builds Voice AI agents that take on administrative work for healthcare teams. The company aims to lead healthcare into the agentic era.