Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Build realistic developer environments with codebases, infrastructure, and context to simulate believable development histories. Design challenging tasks from intermediate states, defining prompts and success criteria that test AI coding agent capabilities. Write functional and integration tests to verify agent solutions, accepting valid approaches while rejecting incorrect ones. Iterate on tasks and tests based on QA feedback to ensure evaluations remain fair and robust. This project-based freelance role requires 8+ years of software development experience with a core stack including Python, JavaScript, Docker, and Postgres. Tasks are estimated at 30 hours and must be submitted by deadlines to meet acceptance criteria.