Sr. Product Mgr, Global Focus Categories
On-site · Toronto, Ontario, Canada
Job Summary
Senior Product Manager for Global Focus Categories leading AI/ML evaluation and data-quality programs. You will coordinate cross-functional work between category managers, merchandisers, domain guides, scientists, and engineers to design and run human evaluation and annotation programs, manage distributed annotation teams across timezones, translate evaluation results into actionable requirements for model and product teams, and define end-to-end processes to ensure AI-driven experiences meet quality standards. You will build quality standards for training/evaluation data, monitor live-site quality signals, and partner with science teams and business partners to maintain data labeling schemas, coverage, and freshness across focus categories. Must understand AI model training, data curation, human judgment, and evaluation shaping model behavior; experience with LLM evaluation and annotation vendors is a plus; experience with AI observability and production-grade quality monitoring is a plus.
Required Qualifications
- 5+ years of PM experience with deep exposure to AI/ML or technical platform products
- Experience leading annotation or evaluation programs, including working with distributed annotator pools, external vendors, or crowdsourced labeling at scale
- Analytical and data science fluency — comfortable writing SQL queries to explore datasets, track program metrics, and validate quality signals independently
- Ability to work alongside Applied Scientists — understands how DS workflows operate, can scope analytical work clearly, and knows when to roll up their sleeves
- Working knowledge of how AI models are built and trained — including the role of data, human judgment, and evaluation in shaping model behavior
- Direct exposure to LLM evaluation — preference data, benchmarking, red-teaming, or human feedback collection is a plus
- Experience with annotation vendors or crowdsourced labeling programs is a plus
- Familiarity with AI observability, quality monitoring, or anomaly detection in production is a plus
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.