Product Manager, APEX
$180,000–$300,000 year
On-siteSan Francisco, California, United States
Job Summary
Own the roadmap and portfolio strategy for Mercor's APEX benchmark family, setting priorities across new benchmark development, leaderboard launches, and infrastructure investment. Run the intake for new benchmarks by evaluating proposals against real demand and company strategy while enforcing eval integrity through contamination policies, holdout strategies, and clear methodology publishing. Build the end-to-end evaluation pipeline from harness execution to published results, including grading, model onboarding, and cost reporting. Work directly with frontier AI labs and enterprise customers to understand evaluation needs, defend results, and translate insights into the next benchmark. Close the loop to the business by connecting demand signals to loss analysis investments and tracking adoption to justify ROI. Write specs and PRDs, spot-check failures, and make small PRs to continuously improve the system.
Required Qualifications
- 5+ years in product management, technical program management, or a customer-facing technical role
- Experience reasoning fluently about rubric design, inter-rater reliability, agentic harnesses, contamination and overfitting, and what a small sample can and can't support
- Ability to tell a real capability gap from measurement noise
- Track record of building a product, program, or business line from nothing
- Ability to take full accountability for outcomes, not just outputs
- Ability to self-direct in ambiguous contexts, creating clarity for others
- Ability to bring research, engineering, operations, and GTM around a shared methodology
Desired Qualifications
- Prior background in SWE, ML, or DS
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.