Senior AI Quality & Reliability Engineer
$102,400–$179,000 year
On-siteChicago, Illinois, United States or Irving, Texas, United States
Job Summary
Design and execute AI Quality Engineering strategies for enterprise AI platforms, including large language models, intelligent automation, and agentic systems. Build scalable practices covering AI-native testing, validation processes, runtime quality controls, and automated frameworks to support responsible deployment. Lead validation activities such as functional, prompt, and workflow testing while partnering with AI Engineering, Security, and Clinical teams. Support runtime reliability through observability, telemetry, distributed tracing, and incident response. Develop evaluation frameworks and quality scoring methodologies to improve solution reliability. Mentor Quality Engineers and guide technical adoption of modern testing practices. Collaborate with stakeholders to validate healthcare workflows and payer operations using human-in-the-loop validation.
Required Qualifications
- Bachelor's degree in Computer Science
- 5 or more years of experience in Quality Engineering, Quality Assurance, software testing, enterprise application delivery, technology operations, or related technology functions
- 2 or more years of experience providing technical leadership for testing initiatives, automation programs, or enterprise technology delivery projects
- Experience supporting Quality Engineering or Quality Assurance across enterprise platforms, APIs, healthcare applications, operational workflows, or integrated business systems
- Strong knowledge of software development life cycle (SDLC), Agile methodologies, test automation frameworks, defect management, release validation, and production support processes
- Strong analytical, problem-solving, organizational, communication, collaboration, and leadership skills
- Demonstrated ability to manage multiple priorities and deliver results within fast-paced, highly collaborative enterprise environments
Desired Qualifications
- Relevant degree
- Advanced degree in Computer Science, Engineering, or a related field
- Experience validating AI-powered applications, intelligent automation, machine learning, large language model (LLM), or AI-enabled business workflows
- Experience with AI Quality Engineering practices, AI-assisted testing, runtime observability, monitoring, telemetry, or reliability engineering
- Experience in healthcare technology, payer operations, clinical workflows, or other regulated industries supporting AI governance and responsible AI deployment
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.