Senior Director, Quality Engineering
$230,000–$250,000 year
Remote
Job Summary
Define and execute the enterprise Quality Engineering strategy, embedding quality across the AI-native SDLC through automation, measurable standards, and release governance. Build modern testing frameworks for AI-powered products, LLM applications, and agentic workflows, including automated evaluation, model validation, and continuous quality monitoring. Drive adoption of AI agents to automate test generation, regression testing, defect analysis, and release readiness. Establish scalable testing practices across functional, integration, regression, performance, security, accessibility, and end-to-end testing to improve release confidence. Lead and scale a high-performing, distributed Quality Engineering organization while driving cross-functional alignment. Define validation frameworks ensuring AI systems are reliable, safe, compliant, and trustworthy while partnering with Security, Privacy, Clinical, and AI teams to satisfy HIPAA, SOC 2, and applicable clinical and regulatory requirements.
Required Qualifications
- Proven experience leading and scaling quality engineering organizations (multiple teams across products) in high-growth & fast paced environments, including distributed and globally sourced teams
- Deep technical expertise in test automation architecture at scale & modern frameworks (e.g., Playwright, Cypress, pytest, Jest), CI/CD integration, contract and API testing, performance and resilience testing
- Hands-on experience building evaluation and testing systems for AI/ML or LLM-powered products: eval harnesses, golden datasets, LLM-as-judge, regression testing across model and prompt changes, red-teaming, and guardrail validation
- Demonstrated experience deploying AI agents or AI-assisted tooling into the SDLC, test generation, triage, maintenance, or code review with measurable productivity gains
- Strong track record of shift-left transformation: moving organizations from manual QA gatekeeping to engineer-owned quality with platform support
- Experience with quality in regulated or high-consequence domains (healthcare, fintech, or similar), including HIPAA/SOC 2, PHI-safe test data, and audit and traceability requirements
- Strategic mindset with the ability to balance long-term vision and hands-on execution; credible in deep technical design reviews
- Strong cross-functional leadership and communication skills, with the ability to influence at executive levels and make quality trade-offs legible to non-engineering stakeholders
- Demonstrated ability to drive measurable outcomes (e.g., escaped defect rate, change failure rate, test cycle time, automation coverage, eval pass rates, cost per release)
- Candidates must be based in the NYC metro area or able to relocate independently within 90 days of their start date
- Occasional travel will be required for team on-sites
Desired Qualifications
- Proven experience leading and scaling quality engineering organizations (multiple teams across products) in high-growth & fast paced environments, including distributed and globally sourced teams
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.