Monthly Data Pipeline Operator — Run Our Production Data Refresh (12-Month Recurring)
python, postgresql, linux
Preferred qualifications
- Experience: Intermediate
We are a property-data company. Once a month we rebuild the core dataset we sell, and we are looking for one reliable contractor to own that process for the next 12 months.
This is an operations role. The pipeline already exists and is documented in detail. We need someone to run it well, on schedule, at controlled cost — not to rebuild it.
WHAT THE WORK IS
Each month you run a 12-stage pipeline end to end. In outline:
- Register the run and export roughly 100 million property records to S3
- Rent CPU machines and scrape updated street-level imagery
- Download several million images and pack them for distribution
- Stand up multi-GPU inference servers and score the imagery with our own vision model
- Import the results into Aurora PostgreSQL
- Run a scheduled maintenance window to swap production tables and rebuild downstream views
- Complete a 14-point QA checklist before the run is declared good
It is roughly 15 to 25 hours of work per month, concentrated into a 5 to 7 day stretch. You set your own hours, with one exception: the maintenance window is agreed in advance and has historically landed on a weekend morning or early morning US time. That part is real, and it is priced into the rate.
Everything runs on our cloud accounts. You never front costs.
WHAT WE NEED YOU TO HAVE
Required:
- Experience running production data pipelines where a mistake was visible to customers
- Strong Linux over SSH, and comfort living in tmux for multi-day jobs
- Python
- PostgreSQL at scale: 100M+ row tables, materialized views, partitions, and the ability to find and cancel a stuck query in pg_stat_activity without panicking
- AWS, particularly S3 and Aurora
- Multi-GPU inference serving (SGLang, vLLM or equivalent) and enough CUDA to stand up an 8-GPU box from scratch
- Comfort renting from GPU and CPU marketplaces, with real attention to bandwidth and instance pricing
Nice to have:
- Hugging Face Hub used as infrastructure rather than as a model zoo
- Large-scale image acquisition experience
- A track record of taking an undocumented process and making it boring
WHAT MATTERS MOST
We have a detailed SOP and we will walk you through a full run before you take it over. What we want is someone who reads the runbook, follows it, checks the throughput number before walking away from a job, and raises a hand early when something looks wrong.
There are a few known rough edges with documented workarounds. We will show you all of them. This role rewards discipline over improvisation.
Part of the job is improving the documentation as you go and training a backup operator by month six, so that the process stops depending on any one person.
NOT IN SCOPE
Re-architecting the pipeline, new product features, model training, customer support. We do have a redesign in mind and may scope it separately later, but it is not this contract.
ENGAGEMENT
- 12-month contract, one delivered cycle per month
- Fixed monthly fee per completed cycle, plus a one-time onboarding fee for your first supervised run
- Independent contractor
- Fully remote, anywhere, provided you can commit to the maintenance window
TO APPLY
Please skip the generic cover letter. Answer the four questions instead — we read those first and almost nothing else.
Open job
AI proposal draft
Generate a short cover letter to copy into the offer. Says you are interested and ready to work.
Sign in to generate an AI proposal draft.
Log in