AI Quality Associate
$110,000–$145,000 year
Hybrid · Chicago, Illinois, United States or New York City, New York, United States
Job Summary
AI Quality Associate focused on validating the quality of AI-generated outputs, agent behaviors, and AI-assisted workflows. Design and execute structured evaluations for AI-enabled features, assess outputs for groundedness, instruction adherence, consistency, usefulness, tone, and risk; identify hallucinations and unsafe recommendations; build and maintain golden datasets, benchmark prompts, scorecards, and rubrics; validate model and workflow changes; collaborate with Finance SMEs, product managers, and engineers to define acceptable quality in real business contexts; produce evidence to support release decisions and continuous improvement. Requires 4+ years in QA/AI evaluation or related roles, strong writing and analysis, and familiarity with finance/domain processes; hybrid US-based, with remote work 2 days per week; Bachelor’s degree preferred; salary range $110,000–$145,000 plus benefits and bonus; no immigration sponsorship.
Required Qualifications
- 4+ years of experience in QA, analytics, business process validation, AI evaluation, operations, or similar roles
- Strong writing, analysis, and pattern-recognition skills
- Experience evaluating outputs against nuanced criteria rather than only binary correctness
- Experience with finance, accounting, FP&A, transaction services, or business process design preferred
- Bachelor's degree preferred
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.