Testing Engineer

The role

A Testing Engineer will build and maintain quality across software features and AI/ML systems. You will write automated tests, validate AI/ML pipelines, and integrate quality practices into the delivery process.

What you will do

• Write and maintain automated test scripts for UI, API, and integration testing
• Validate ML model outputs including accuracy, consistency, and expected behavior
• Test data pipelines and AI workflows
• Execute automated test suites and track defects
• Integrate automated tests into CI/CD pipelines
• Support model evaluation processes and test AI workflows end-to-end
• Test AI/ML systems including LLM evaluation, RAG retrieval, and AI safety
• Collaborate with developers, ML engineers, and data scientists
• Participate in sprint ceremonies and quality strategy discussions
• Take ownership of testing responsibilities end-to-end

Required qualifications

• 2+ years of experience in test automation or AI/ML testing
• Proficiency in Python, JavaScript, Java, or similar
• Experience with test automation frameworks for UI, API, or ML workflows
• Understanding of AI/ML concepts and the ML lifecycle
• Experience with CI/CD pipelines and version control
• Working knowledge of API testing and data validation
• Solid understanding of SDLC/STLC and Agile methodologies
• Proficiency with bug-tracking and test management tools
• Familiarity with testing AI/ML systems
• Demonstrable, day-to-day usage of AI-forward tools such as Claude and Cursor
• Strong problem-solving skills

Nice to have

• Experience with mobile automation
• Performance testing
• MLOps pipelines
• Basic statistics

About the company

Robots & Pencils is an applied AI engineering firm building AI co-workers that integrate into enterprise operations. Founded in 2009, they have delivery centers in Canada, the United States, Eastern Europe, and Latin America.