Sr. Software Development Engineer in Test, AI/ML

The role

This role involves specializing in ML/AI quality, including automation for model evaluation, LLM-assisted test generation, and validation of AI-powered workflows. You will lead quality initiatives, develop automation frameworks, and manage implementation for complex machine learning-powered products and services.

What you will do

• Define and complete quality strategies, test plans, and automation coverage for ML-powered services and platform components.
• Use LLMs and other AI-assisted techniques to generate, expand, and maintain high-value test cases.
• Design scenario-based test suites for AI features, including adversarial prompts and edge cases.
• Lead QE efforts for multi-functional projects, driving risk assessment and release readiness.
• Design, develop, and maintain scalable automation frameworks using Python and/or Java.
• Build automated validation for ML and LLM outputs.
• Debug test failures, service anomalies, model inconsistencies, and AI behavior regressions.
• Perform functional, integration, regression, API, end-to-end, performance, and reliability testing.
• Improve automation reliability and optimize execution efficiency.
• Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines.

Required qualifications

• Bachelor’s degree in Computer Science or equivalent
• 5+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
• Experience using LLMs to generate, transform, and prioritize test cases for AI-powered experiences.
• Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human-in-the-loop review workflows.
• Strong experience testing RESTful APIs, microservices, and distributed architectures.
• Proficiency in Python, Java, JS or similar languages for automation development.
• Hands-on experience with automation frameworks such as pytest, JUnit, Selenium, Playwright, Cypress, or Appium.
• Experience with CI/CD systems and test pipelines (Jenkins, GitHub Actions, etc.).
• Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker).
• Familiarity with databases, monitoring, and observability tools.
• Strong understanding of SDLC, Agile methodologies, and release processes.
• Excellent problem-solving, debugging, and communication skills.

Nice to have

• Experience validating ML outputs using statistical analysis or scenario-based testing approaches.
• Familiarity with ML infrastructure, data pipelines, or model-serving platforms (Seldon, KServe, Ray Serve, etc.).
• Prior work in content moderation ML, security, fraud detection, or adversarial ML.
• Experience testing high-scale, low-latency online services.
• Experience with Databricks or similar ML platform tooling.
• Familiarity with Node.js, React, or modern frontend technologies.
• Experience testing mobile, console, or other non-PC platforms.

Benefits

• medical
• dental
• vision
• matching 401(k)
• paid time off
• wellness program
• employee discounts for Sony products
• may be eligible for a bonus package

Before you apply

• Sony Interactive Entertainment conducts background checks at the offer stage for all new employees

About the company

Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. It is a dynamic technology company and an entertainment leader.