Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Build realistic developer environments containing codebases, infrastructure, and historical context. Design challenging tasks from intermediate states, defining prompts, success criteria, and solvability. Write functional and integration tests to verify agent solutions, accepting valid approaches while rejecting incorrect ones. Iterate on tasks and tests based on QA feedback to ensure evaluations remain fair and robust. This project-based freelance role involves creating datasets to evaluate AI coding agents on real-world developer tasks. Compensation is up to $200/hr for approximately 30 hours of work, with flexible scheduling.