Staff Software Engineer, QE
RemoteNew York City, New York, United States or India
Job Summary
Lead the development of test automation frameworks across UI, API, and GraphQL layers while designing and integrating AI evaluation frameworks to assess accuracy and consistency of LLM-powered features. Drive adoption of newer testing technologies, optimize test execution pipelines for faster feedback in CI/CD environments, and implement shift-left testing practices to embed quality ownership within teams. Lead root cause analysis for critical issues, provide technical mentorship on automation best practices and AI tooling, and partner with developers to ensure testability and quality considerations are addressed early.
Required Qualifications
- 8+ years of hands-on experience in software quality engineering roles
- At least 2 years operating at a Staff/Lead level
- Deep proficiency in testing TypeScript/JavaScript and modern frontend ecosystems (React, Webpack, or similar)
- Expert-level experience with Playwright or comparable browser automation frameworks (Cypress, Selenium WebDriver) for end-to-end testing
- Strong experience with unit and component testing frameworks such as Vitest, Jest, or React Testing Library
- Proven track record of designing and building test automation frameworks from scratch or significantly evolving existing ones at scale
- Solid understanding of CI/CD systems (GitHub Actions, Jenkins, Buildkite, Gitlab or similar) and how to integrate and optimize test suites within them
- Experience with GraphQL APIs - Writing integration tests, mocking schemas, and validating query/mutation contracts
- Experience in SaaS, fintech, or market intelligence domains
Desired Qualifications
- Familiarity with performance and load testing tools (k6, Lighthouse, Artillery)
- Experience with visual regression testing (Percy, Chromatic, Playwright visual comparisons)
- Hands-on experience with Allure or similar test reporting/analytics platforms
- Understanding of AI/ML-powered products and the unique testing challenges they present (non-deterministic outputs, data quality, model evaluation)
- Track record of contributing to developer experience improvements like test tooling plugins, custom ESLint rules for test quality, documentation-as-code practices
- Familiarity with Docker, Kubernetes, and cloud infrastructure (AWS/GCP)
- Familiarity with AI/LLM evaluation frameworks
- Hands-on experience with AI-assisted development tools such as Cursor, GitHub Copilot, or Claude for accelerating test authoring, code review, and debugging
- Experience building or configuring MCP (Model Context Protocol) servers
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.