Arbiter Ai

Senior QA Engineer, Voice AI Agents

Remote, US. Posted a month ago

The role

You will investigate call and platform issues by reviewing conversations and tracing failures through logs and transcripts. You will perform campaign QA and help build and maintain the QA framework and automation.

What you will do

  • Investigate reported and detected issues on live campaigns
  • Trace problems across call recordings, transcripts, system logs, campaign configuration, and integration data
  • Write clear, reproducible bug reports
  • Triage incoming issues and route to the right owner
  • Run pre-launch QA on new campaigns and workflow changes
  • Carry out ongoing conversation QA on live campaigns
  • Check that campaigns behave as designed
  • Help build and maintain the QA framework
  • Automate repetitive checks
  • Track quality metrics and report patterns

Required qualifications

  • 3 to 5 years in QA, support engineering, technical operations, or a similar role on a production software platform
  • A strong debugging instinct
  • Comfort reading logs, querying data with SQL, and working with APIs and JSON or YAML configuration
  • Experience writing test plans and bug reports that engineers trust
  • Clear written communication and precise documentation habits
  • Good judgment on severity and prioritization

Nice to have

  • Experience with voice AI, conversational AI, IVR, or contact center platforms
  • Scripting ability in Python or similar for automating checks and analysis
  • Exposure to LLM evaluation, prompt testing, or conversation quality scoring
  • Healthcare experience, especially payer or provider outreach, and familiarity with HIPAA and PHI handling
  • Experience with observability tools or with call analytics tools

Benefits

  • Highly Competitive Salary & Equity Package
  • Generous Paid Time Off (PTO)
  • Comprehensive Health, Vision, and Dental Insurance
  • Life and Disability Insurance
  • Simple IRA Matching
  • Professional Development Budget
  • Wellness Programs

About the company

Arbiter is the AI-powered care orchestration system that unites healthcare. They are building connected infrastructure for healthcare using Agentic AI approaches.