QA Tester II
On-site · Pasig City, Metro Manila, Philippines
Job Summary
QA Tester II role focused on AI Development. Define and implement test strategies for AI-powered applications, including functional, integration, regression, and AI-specific testing (prompt evaluation, hallucination detection, guardrail validation). Build and maintain automated test frameworks using Python (Pytest) and UI automation tools (Playwright, Cypress, or Selenium) for React front-ends and API back-ends. Validate AI model outputs for accuracy, safety, and adherence to expected behaviors across agentic workflows and conversational interfaces. Integrate automated test suites into CI/CD pipelines to ensure rapid feedback on code changes. Lead defect management: identify, document, reproduce, and track bugs; collaborate with developers to verify fixes. Collaborate with AI Developers, Full Stack Developers, and DevOps Engineers to ensure quality is embedded throughout the development lifecycle. Establish QA metrics and reporting to communicate quality status and risk to stakeholders.
Required Qualifications
- Minimum 3 years of experience as a QA Engineer or SDET
- Proficient in Python for writing automated test scripts (Pytest preferred)
- Experience with UI automation tools (Playwright, Cypress, or Selenium)
- Strong understanding of REST APIs; experience validating API responses and JSON schemas
- Familiarity with testing LLM-powered applications, including non-deterministic output validation; experience with evaluation frameworks (RAGAS, LangSmith, or similar) is a plus
- Working knowledge of Microsoft Azure; ability to navigate cloud logs and debug deployed services
- Experience integrating test suites into CI/CD pipelines (Azure DevOps, GitHub Actions)
- Strong written and verbal English communication skills
- Nice to Have: experience with performance testing, security testing, or accessibility testing
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.