Research Operations, Code
$130,000–$250,000 year
On-siteSan Francisco, California, United States
Job Summary
Decompose frontier model capabilities to identify capability gaps and translate research goals into concrete task designs and quality specifications. Design and drive execution of multi-million-dollar data pipelines, balancing quality, throughput, and cost while managing bottlenecks through workflow restructuring and incentive systems. Own end-to-end delivery for leading AI labs, acting as the primary point of contact to deliver high-quality reasoning data against demanding timelines. Spend approximately 30% time on customer engagement, 40–50% on data production and scaling, and 15% on operations and metrics review. Work in-person five days a week in San Francisco, supporting a profitable Series C company building the layer between human expertise and frontier models.
Required Qualifications
- Technical judgment
- Coding literacy
- ML or Model Benchmark familiarity
- Ability to evaluate model outputs
- Ability to reason about frontier-model capabilities and failure modes
- Ability to read benchmark/eval work
- Ability to translate research goals into concrete task and data designs
- Operational ownership
- Track record of running complex, high-stakes projects end to end
- Energy for large-scale execution
- Gritty process optimization under pressure
- Strong analytical skills
- Communication skills
- Comfort owning high-profile relationships with technical customers
- Location: San Francisco (in person, five days a week)
Desired Qualifications
- Pipeline building
- Designing data or automation pipelines
- Hands-on with LLMs/agents
- Building synthetic-data or model-in-the-loop systems
- Research fluency
- Connecting model/benchmark literature to what data would move a frontier model
- Having created a benchmark
- Published analysis of model behavior
- Backgrounds from ML/data/software engineers who love operating
- Technical PMs
- Research engineers
- Strong generalist operators with real technical range
- Backgrounds from consulting
- Backgrounds from finance
- Backgrounds from high-growth startups paired with real technical fluency
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.