Senior Software Engineer - AI Interaction Evaluator (Codex / Claude Code, up to $200/hr)
$208,000–$416,000 year
RemoteUnited States or Calgary, Alberta, Canada
Job Summary
Evaluate AI-generated coding interactions end-to-end, judging outputs for usefulness, correctness, and alignment with strong engineering judgment. Assess the quality of explanations and reasoning, distinguishing between response tiers based on whether the model thinks like a great engineer. Provide clear, opinionated feedback on what worked, what failed, and what felt misleading to an experienced developer. Define standards for great interactions with tools like OpenAI Codex, Claude Code, and Cursor. This role requires Staff or Principal-level engineers with deep familiarity in TypeScript, Python, or modern AI-assisted dev workflows to make rigorous subjective judgments on engineering taste rather than syntax.
Required Qualifications
- Senior Software Engineer (SR+) level
- Staff / Principal-level engineer (or equivalent experience)
- Strong background in TypeScript / JavaScript
- Strong background in Python
- Hands-on experience using OpenAI Codex
- Hands-on experience using Claude Code
- Hands-on experience using Cursor
- Deep familiarity with modern AI-assisted dev workflows
- Able to evaluate code without needing to fully execute or deeply review every line
- Comfortable giving direct, opinionated feedback
- High bar for what 'good engineering' looks like
- Contract employment type
- Rate of $100–$200/hour
- 10–20 hours/week availability
- Start date ASAP (through early May)
Desired Qualifications
- Experience with tools like Cursor or similar AI-first IDEs
- Prior exposure to prompt design or evaluation workflows
- Experience mentoring senior engineers or defining engineering standards
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.