Senior AI QA Engineer for Sora Union

Global. Posted 21 months ago

The role

You will design and execute tests for LLM-generated responses and build AI-powered testing agents. You will work with software developers, product managers, and designers to ensure high quality for client products.

What you will do

Design and execute tests to evaluate LLM-generated responses
Build, maintain, and improve AI-powered testing agents and test harnesses
Test Model Context Protocol (MCP) tools and integrations
Use Claude, ChatGPT, Playwright MCP, and related tools for test automation
Develop and maintain test plans, test cases, and regression suites
Design, execute, and maintain automated tests for web applications, APIs, and end-to-end workflows
Investigate product behavior and identify edge cases
Test backend APIs, verify data integrity, and work with SQL queries
Reproduce, document, triage, prioritize, and track defects
Partner with engineering, product, and other teams to clarify requirements and assess risks

Required qualifications

Degree in computer science or mathematics
Minimum of 7 years of experience in a professional QA role
Hands-on experience with AI/LLM testing
Practical experience using AI tools such as Claude and ChatGPT
Experience with Playwright and MCP-based automation
Experience building or maintaining AI-powered testing agents, test harnesses, or automated evaluation workflows
Strong understanding of manual, exploratory, functional, smoke, sanity, regression, integration, and end-to-end testing
Experience testing web applications and backend APIs
Familiarity with test automation frameworks such as Playwright, Selenium, or Cypress
Experience with API testing tools such as Postman
Working knowledge of REST APIs
Working knowledge of SQL and databases

Nice to have

working knowledge of GraphQL

About the company

Sora Union is a professional services company specializing in brand design and engineering services. The team is globally-distributed and made up of diverse professionals impacted by or at risk of displacement due to conflict or climate change.