PostHog Engineer — Build an “AI-as-a-Tester” Agent + Stratified Testing (Part-Time, Ongoing)

# PostHog Engineer — Build an "AI-as-a-Tester" Agent + Stratified Testing (Part-Time, Ongoing) ## About us We're a small product team building **GoalOasis**, a video-game mentorship app. We're setting up our testing program around **PostHog** and need an engineer to configure it and help build an AI agent that tests our product automatically — measured against real human testers. ## What you'll do (high level) - Stand up and configure **PostHog**: event taxonomy, autocapture, feature flags, experiments, session replay, and surveys. - Instrument our current **Figma Make** prototype with a clean, portable event schema. - Pair with our **in-house JavaScript engineer** to build an **AI agent that acts as an automated tester**, firing the same events a real user would. - Set up **cohorts/properties so agent traffic and human traffic are cleanly separated** (stratified testing). - Configure **experiments + feature flags** on the flows under test and onboard ~75 async human testers into the same setup. - Help build a **data export / training pipeline** so what we gather now can train the agent to work with our **live app** later. ## Required - Hands-on experience **configuring PostHog** (events, feature flags, experiments, session replay). - Comfortable building or integrating **AI agents** — we're **agent- and language-agnostic** (LangChain / LangGraph, etc. all fine). - Clean event/schema design and solid JavaScript collaboration skills. ## Nice to have - **Data analysis with PostHog experiments** (funnels, retention, significance, HogQL) so you can help interpret results. - Experience designing testing systems meant to **outlive an MVP launch**, and **fine-tuning models** on collected data. ## Our live stack (for context, later phase) React Native front end; back end on Fluent Community / Fluent Boards REST API, custom Keystone.js, and Firebase Functions. ## Who you'll work with A part-time team: two UX designers, a React Native engineer, and the founder/director (web developer) who manages the process. (A PHP / WordPress developer joins later for plugin work — not part of this role.) ## Engagement - **Part-time, work at your own pace** — the deadline is flexible. - **Ongoing**, with our MVP launch targeted **~7–13 months** out. ## To apply Share a brief note on your PostHog work and any AI-agent or test-automation projects you've shipped. A link or short walkthrough is worth more than a long cover letter.