Data Engineer for a data science company in the video game industry

Indicative duration: 100-200 days (TBD) Location: Remote WHO WE ARE Hushcrasher is a data science company based in France building decision intelligence for the video game industry. Our ambition is to become the reference for data science in the video game industry: the partner studios and publishers turn to when they want decisions grounded in evidence rather than intuition. In six months, we’ve secured a competitive European grant, our data is being used in Harvard University’s Department of Economics, and inbound demand from studios, publishers and investors is coming in faster than we can take on. Our edge lies in the models we build, grounded in state-of-the-art research in Economics and Machine Learning, and shaped by technical leadership trained in a Nobel Prize-winning research environment. All of it runs on fast-growing amount of data pulled from many heterogeneous external sources and consolidated into an analytical lake. The rigor of our models is only ever as good as the foundation underneath them. Building and industrializing that foundation is the data engineering challenge we are facing. THE CONTEXT We currently pull data from 6 different sources, relying on two ingestion modes: APIs and web scraping. It started as a scattered set of scripts, and we’ve begun industrializing it into a structured, orchestrated monorepo. Today it’s still an MVP covering two sources and a couple of endpoints, not yet in production. The pattern is there, it needs to be hardened and extended. We’re looking for a freelance data engineer to provide the following services: auditing the current codebase, challenging the existing architecture, taking the pipeline to production, and extending it across our sources. One clarification on intent: the goal is a clean analytical lake of files, refreshed on a regular cadence and directly usable by our data science team. It is not a transactional database, and not an API behind an application. The deliverable is a reliable, fresh, observable data pipeline, ready to be modeled. SCOPE OF WORK Before getting into the specifics: we are looking for an architect’s view before an implementer’s. The new system is not yet in production, so there is still room to break things, rethink assumptions, and build something clean, robust, and genuinely fit for our data sources and our use cases. This applies across all three components of the scope below. 1. Reworking and industrializing legacy sources (core scope) Today we pull a lot of this data through legacy scripts. Each source should be brought into the established pattern and made production-grade: entity discovery, retrieval (via API, scraping or native-protocol clients), parsing, materialization into Parquet, orchestration, and deployment; built as a proper discovery-and-refresh pipeline, with resilient retrieval, observability, and sane failure handling. It’s repeatable work, done source by source. We will provide the list of currently used endpoints so you can assess and price the scope accurately. 2. New endpoints from existing sources We don’t yet pull every endpoint of every source we use. This part extends coverage to the ones we’re missing, bringing them into the same pattern. 3. Building the analytical layer There’s currently a minimal analytical pipeline: We have a monolithic script that joins sources into one unified file. This component has two steps: • Design the analytical layer based on requirements from the data science team: its architecture, the transformations (cross-source joins, aggregations, historicization), the conventions, and the interface through which our data science team exports its results back into the lake. Statistical modeling is data science’s job, but you define the framework it works within. • Build the analytical tables pipelines used by the data science team, once the design is settled. TECH STACK Python (fully typed), uv, Prefect 3, PostgreSQL, Cloudflare R2, Parquet / PyArrow, duckdb, deployment is manual on our VPS, CI (type check + test suite) through Github actions. Note on infrastructure: DevOps support will be handled separately. The scope of this engagement is strictly focused on the data engineering architecture and pipelines, not sysadmin work. CONTRACTOR PROFILE • A confident, autonomous data engineer. • Expert-level Python, fluency with an orchestrator (Prefect), PostgreSQL, object storage, and TDD. • Hands-on analytical modeling to design the analytics layer. • An architect’s eye: able to audit what exists and propose well-argued evolutions. • Proven experience with resilient scraping. • A knowledge of data science’s needs (quality, freshness, usability of the data produced) is a real plus. • Prior work with small teams or startups is a plus. ENGAGEMENT STRUCTURE This is a contractor engagement for a defined scope of services, structured around deliverables and milestones, and expected to span several months. This engagement will be structured with a modular, fixed-price approach. Parts 1 & 2 will be quoted on a per-data-source (or per-endpoint) basis, allowing you to budget each integration individually based on its specific complexity. The analytical layer (Part 3) will be scoped as its own distinct milestone. You’ll receive a detailed technical brief so you can accurately assess and price each source. The engagement is fully remote, and open to both English- and French-speaking contractors. Depending on where you are based and if it makes sense to start in person, we may suggest an on-site meeting, with travel and accommodation covered.