Labeler / Annotator – AI Response Evaluation (French)
$28,080,000–$31,928,000 year
Remote
Job Summary
Conduct side-by-side evaluations of AI-generated responses across general-purpose questions, web search results, file-based outputs, and multi-turn conversations. Assess French-language content for accuracy, relevance, clarity, instruction-following, and cultural nuance while applying detailed annotation guidelines. Document evaluation decisions with clear rationale and maintain consistent quality across high-volume tasks. Participate in training, calibration, and ongoing quality-review activities to ensure alignment with team standards. This role supports Blueprint's AI and product development teams in Chile, helping refine how systems understand and communicate in French.
Required Qualifications
- Native-level or professional fluency in French
- Deep familiarity with French linguistic conventions, tone, idioms, and cultural context
- Strong English reading comprehension, including the ability to understand detailed guidelines written in English
- Prior experience with side-by-side labeling, annotation, or comparative content evaluation
- Excellent analytical thinking and attention to detail
- Ability to identify subtle differences in quality, meaning, tone, and instruction-following
- Ability to learn and consistently apply structured evaluation frameworks
- Strong written communication and the ability to explain evaluation decisions clearly
- Ability to work independently while maintaining alignment with team quality standards
Desired Qualifications
- Background in linguistics, translation, localization, or a related field
- Experience with AI response evaluation or model-quality assessment
- Experience with data labeling or annotation
- Experience evaluating search relevance, content quality, or user-facing digital experiences
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.