Senior Software Quality Engineer
$153,200–$221,750 year
On-siteSan Francisco, California, United States
Job Summary
Own the end-to-end quality strategy for a new AI-first web application, spanning the prompt and agent layer, the in-browser experience, and the generated output. Compose, build, and maintain modern web test automation (Playwright / Cypress, TypeScript/JavaScript) and the underlying infrastructure that keeps it fast, deterministic, and developer-friendly. Stand up evaluation harnesses for automated workflows: golden prompt suites, regression baselines, model/version comparisons, hallucination and safety checks, and human-in-the-loop review pipelines. Ensure comprehensive quality for a real-time web product — performance, resilience, accessibility (WCAG), and cross-browser/cross-device coverage. Embed quality into CI/CD: build release gates that catch real regressions, reduce flake, and turn production telemetry into pre-submit signal. Work closely with engineering, product, and design to build testability into the process early. Guide engineers on test strategy, automation patterns, and adopting a shift-left outlook.
Required Qualifications
- Bachelor's degree or equivalent experience in Computer Science or a related field
- 8+ years of experience in customer-facing software quality / SDET positions
- Automation experience with modern web frameworks (Playwright, Cypress, or similar)
- Strong coding skills in TypeScript/JavaScript
- Experience designing test strategy for AI / ML / LLM-powered features —prompt regression, evaluation datasets, model drift, timely-injection and safety testing, or equivalent
- Strong fundamentals in API testing, performance and load testing, and observability tooling (Datadog, Splunk, Sentry, OpenTelemetry, or similar)
- Excellent collaboration and communication; able to influence engineers and PMs early in the build cycle
- Embrace generative and AI-assisted development workflows, and be eager to learn and adapt as the tooling continues to evolve
- Thrive in a fast-paced, agile, founding-team environment with real ambiguity
Desired Qualifications
- comfort with Python is a plus
- Prior experience with creative tools, design-to-code, browser-based graphics (Canvas/WebGL), or real-time collaboration stacks
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.