Senior Product Quality Analyst - AI Voice
$114,486–$163,552 year
On-siteLos Angeles, California, United States or New York, United States
Job Summary
Define and measure AI voice quality for Spotify's models by creating evaluation rubrics that assess naturalness, tone, pacing, and pronunciation. Plan and execute evaluation campaigns to turn qualitative and quantitative findings into actionable recommendations for Voice Production and product teams. Benchmark internal voices against third-party models and competitors to identify performance gaps and opportunities for improvement. Establish an ongoing evaluation rhythm to reassess model quality as technology evolves, while determining appropriate quality trade-offs across different languages and regions. Partner with voice experts, machine learning teams, and vendors to maintain a consistent quality bar across competing priorities.
Required Qualifications
- experience in product quality, evaluation, data curation or a closely related field where human judgment is used to assess model or product performance
- ability to independently define evaluation criteria and turn subjective qualities into clear, repeatable rubrics
- comfort designing and running both qualitative and MOS-style evaluations and synthesizing the results into actionable insights
- understanding of voice or audio quality, including qualities such as naturalness, pacing, tone, accent and pronunciation, or the ability to develop that expertise quickly
- strong judgment and can maintain a consistent quality bar while balancing different product requirements, markets and stakeholder perspectives
- comfortable working autonomously, identifying what needs to be evaluated and establishing the processes and cadence to make that evaluation useful over time
- ability to communicate clearly across creative, technical and product teams and can act as a reliable point of contact for internal partners and external vendors
- curiosity about generative AI, speech technology and the evolving AI voice landscape and can translate changes in the industry into meaningful benchmarks for Spotify
- This role is based in New York or Los Angeles
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.