STEM Domain Experts for AI Benchmarks

Project Overview: We are seeking STEM Domain Experts (both Non-Coding Specialists & Coding Contributors) to build evaluation benchmarks for Frontier AI models. Your goal is to design authentic, expert-level research workflows to challenge AI as a true scientific collaborator. Tracks Available: ** Track 1: Non-Coding Specialist** Design open research objectives with objective correctness. Collect and curate input data (PDFs, papers, datasets). Write detailed reference answers (about 500 words) describing expert reasoning. Establish comprehensive evaluation rubrics (4–15 criteria). ** Track 2: Coding Contributor** Build Dockerized execution environments (Python, R, Julia). Design 25+ step terminal workflows (data processing, simulation, parameter optimization). Write oracle solutions (solve.py/solve.sh) and deterministic automated tests (test.sh). Target Domains (STEM, Engineering preferred): Chemistry, Physics, Mathematics, Engineering (Mechanical, Electrical, Material), Data Science in STEM contexts. Requirements: Bachelor’s degree or higher in STEM (Master's, PhD, or PhD candidates strongly preferred). Strong English proficiency (reading/writing). STRICT RULE: NO AI/LLM tools (ChatGPT, Claude, etc.) allowed for task creation. Violation results in immediate termination without pay. Tasks must reflect real-world workflows, not academic homework. Compensation & Flexibility: Paid per accepted task with competitive expert-level rates. Fully remote and flexible schedule.