Research Engineer, Domain Scaling
$350,000–$850,000 year
HybridNew York City, New York, United States or San Francisco, California, United States
Job Summary
Own the end-to-end data strategy for knowledge work verticals, from task sourcing through RL training. Manage technical relationships with external vendors, evaluating data quality and designing reward signals while collaborating with domain experts to build pipelines. Explore novel methods for creating RL environments for high-value tasks and develop QA frameworks to prevent reward hacking. Run generalization experiments to measure how data strategy changes improve model capabilities and partner with RL and product teams to translate goals into training environments and evaluations.
Required Qualifications
- Bachelor's degree or an equivalent combination of education, training, and/or experience
- A field relevant to the role as demonstrated through coursework, training, or professional experience
- Experience with fine-tuning large language models for specific domains or real-world use cases
- Experience with reinforcement learning, reward design, or training data curation for LLMs
- Comfort managing technical vendor relationships and iterating quickly on feedback
- Ability to read through datasets to understand them and spot issues
- Strong cross-functional collaboration skills
Desired Qualifications
- Experience training production ML systems
- Experience designing evals or benchmarks for LLMs
- Domain expertise in a vertical where we would like to make our models more useful
- Experience working with external vendors or technical partners
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.