Senior engineer to audit and label PRs from major open source projects (hourly, paid trial)

We’re building a dataset of expert judgments on real pull requests from large open source projects: ClickHouse, PostHog, Kubernetes, CPython, LLVM, CockroachDB, Mattermost, Home Assistant, and similar. For each PR, you’ll read the diff, the linked issue, the review discussion, and what happened afterward (follow-up fixes, reverts, CI history). Then you’ll assign [labels, e.g. risk level, change type, whether it needed rework] with a short written justification. This is careful reading work, not volume work. We’d rather have 15 thoughtful labels an hour than 60 shallow ones. What we’re looking for 5+ years of professional software engineering, comfortable reading unfamiliar codebases Real experience in at least two of: C/C++, Go, Python, Rust, TypeScript, distributed systems, databases, compilers Strong preference for people who have contributed to any of the projects above. Please link your merged PRs. Fluent at navigating GitHub history: blame, linked issues, revert commits, CI runs Clear, concise written English How it works Hourly contract, tracked with the Upwork desktop app Starts with a paid trial of about 3–4 hours on a fixed set of PRs. We review it and decide on ongoing work quickly. Ongoing work is ~10–20 hours/week with flexible scheduling. Could be ongoing. Rate: Depending on experience AI tool policy (please read) Do not use ChatGPT, Claude, Copilot, or any other AI tool to analyze PRs or write justifications. We need independent human judgment, because that’s the entire point of the dataset. Justifications must cite specific files, lines, and context from outside the diff. We verify the work, and AI-assisted submissions will end the contract. If this policy doesn’t work for you, no hard feelings, but please don’t apply. We will expect to do a video call with you where you perform a sample review on a selected repo to confirm this. To apply, answer these three questions in your proposal Link one or two PRs you’ve authored or reviewed in a large codebase, and say in a sentence what was tricky about each. Pick any merged PR from one of the projects above that later needed a follow-up fix or revert. Link both, and explain in 3–4 sentences what a reviewer could have caught. Which of the listed projects are you most comfortable reading, and why? Proposals that don’t answer these won’t be reviewed.