Member of Technical Staff — Research Engineering, Evaluation
On-siteSan Francisco, California, United States
Job Summary
Design and build a central, reusable evaluation framework that every model and team runs through. Implement evaluation pipelines, benchmark suites, and baselines to make model quality measurable and comparable across efforts. Build visualization and dashboard tools that transform raw results into shared, actionable understanding for the whole organization. Establish sound statistical methodology to distinguish real improvements from noise, partnering with research and domain teams to translate domain-specific goals into standardized, automated metrics. This role requires strong software engineering skills, a full-stack range, and a solid grasp of probability and statistics.
Required Qualifications
- Strong software engineering skills and experience building data or evaluation pipelines at scale
- Experience turning research or model outputs into metrics, benchmarks, and visualizations that teams rely on
- Solid grasp of probability and statistics, with the judgment to design evaluations that measure what they claim to
- Full-stack range: comfortable building both backend pipelines and the frontend tools people read results in
- Owns deliverables end-to-end, from collecting requirements to autonomously driving execution
Desired Qualifications
- We value a relentless approach to problem-solving, rapid execution, and the ability to quickly learn in unfamiliar domains
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.