AI Evaluation Engineer – Terminal-Bench / Coding Agent Benchmarks

We're looking for an experienced AI evaluation specialist for a remote contract evaluating AI coding agents in terminal-based environments. What you'll do: - Evaluate AI-generated solutions to coding, debugging and sysadmin tasks in the command line - Run test suites, investigate failed executions, and tell genuine solution errors apart from environment issues - Review task instructions, expected outcomes and test reliability - Write clear, evidence-based, reproducible findings Must have: - Hands-on experience with Terminal-Bench or similar coding-agent evaluation work (Outlier, Mercor or comparable AI training platforms) - Strong Linux, Git, command-line and scripting skills - Proficiency in Python (JavaScript, Java or C++ a plus) - Experience with Docker, test harnesses and automated grading - Excellent written English To apply, please include: 1. The AI evaluation platforms and project types you've worked on (no confidential details) 2. Your specific Terminal-Bench or terminal-agent evaluation experience 3. Your programming languages and dev environment 4. Your availability and best hourly rate Shortlisted candidates may be asked to complete a short practical assessment.