Italian Data Labeling Associate((Human-in-the-Loop AI))
$70,720–$70,720 year
On-siteSan Francisco, California, United States
Job Summary
Evaluate AI model outputs and provide structured, high-quality feedback while performing audit-based reviews to identify patterns, edge cases, and failure modes. Apply guidelines thoughtfully, flagging when they do not reflect real-world scenarios, and contribute to improving evaluation frameworks rather than just executing them. Identify trends in model performance, communicate insights clearly, and participate in team discussions, calibrations, and stakeholder syncs to partner with leads and cross-functional teams on refining quality standards. Document findings in a clear, concise, and actionable way. Full-time, 40-hour onsite role in NYC, Seattle, Bellevue, Redmond, San Francisco, Sunnyvale, or Burlingame.
Required Qualifications
- Native-level language proficiency
- University degree (Bachelor's or higher)
- B2 or superior level of English
- 1–2 years of professional writing experience
- Strong, structured writing skills
- Ability to apply complex writing rules and guidelines consistently
- Strong understanding of safety considerations in GenAI data delivery
- 2+ years of relevant experience in GenAI safety
- Strong critical thinking
- Attention to detail
- Ability to make sound judgment calls in ambiguous situations
- Naturally curious about AI, technology, and how systems behave
- Comfortable speaking up, asking questions, and contributing ideas
- Strong written and verbal communication skills
- Ability to stay consistent while working with evolving guidelines
- Must be authorized to work in the U.S. (no visa sponsorship)
- Must be located in or able to commute to New York City, Seattle, Bellevue, Redmond, San Francisco, Sunnyvale, or Burlingame
- 100% onsite position
Desired Qualifications
- Experience in data quality, QA, annotation, or analysis
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.