Music Content Reviewer (AI Evaluation)
RemoteIndia or Colombia
Job Summary
Create advanced music-related prompts across disciplines and difficulty levels to evaluate Large Language Models. Assess AI-generated responses for factual accuracy, reasoning quality, and nuance, while identifying hallucinations, outdated information, and logical inconsistencies. Verify claims regarding music theory, artists, composers, genres, instruments, and production techniques. Develop benchmark datasets and adversarial test cases, then provide clear, evidence-based feedback supported by reliable references. Document findings to maintain high annotation standards and collaborate with researchers to improve model performance. This 8-week contractor assignment requires a Master's degree in Music or related fields, 3+ years of professional experience, and full-time commitment with PST overlap.
Required Qualifications
- Master's degree or higher in Music, Music Theory, Musicology, Composition, Music Production, Performance, or a closely related discipline
- At least 3 years of professional, research, or teaching experience in a music-related field
- Strong knowledge of music theory and at least one additional area such as music history, performance, composition, genres, instruments, or production
- Ability to evaluate complex music-related content for factual, logical, and contextual accuracy
- Experience conducting research and supporting conclusions with reliable references
- Excellent written English communication skills
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.