Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Build realistic developer environments including codebases, infrastructure, and contextual history to simulate believable development scenarios. Design intermediate tasks with specific prompts and success criteria, then write functional and integration tests that accept valid agent solutions while rejecting incorrect ones. Iterate on tasks and evaluations based on QA feedback to ensure robustness and fairness. This project-based evaluation focuses on testing AI coding agents against real-world developer challenges, requiring 8+ years of Python and JavaScript experience. Compensation reaches up to $200/hr for the estimated 30-hour effort, with flexible scheduling.