Reddit Data Collection: r/adjusters Complete Thread and Comment Scrape
Бюджет: $400.0
FIXED /
⭐ 5.00 (1)
USA
data-extraction, data-scraping, python, api, json, data-cleansing
I need a Python developer to collect complete thread and comment data from Reddit's r/adjusters subreddit for a research project.
SCOPE
All threads and all comments — including nested replies at every depth — from the trailing 24 months. Nested replies are essential. I need the full conversation structure, not just top-level posts.
OUTPUT
JSON or CSV — tell me which you prefer and why. Each record should include: thread title, thread URL, author, timestamp, comment text, comment depth, parent ID, upvote count, and any metadata you think adds value. Deleted and removed comments should be handled explicitly rather than silently dropped.
DELIVERABLES
1. A paid test run on one week of data so I can validate the format before full collection
2. The complete dataset once the format is approved
3. A short note on your method — library used, rate-limit handling, deduplication approach
PLEASE ANSWER THIS IN YOUR PROPOSAL:
Reddit's API caps listing retrieval at roughly 1,000 items per sort order. How will you get complete coverage of a 24-month window despite that cap?
TERMS
Reddit's official API only. Compliant collection — no HTML crawling. If you see a better approach than what I've described, propose it rather than following my spec silently.
WORKING STYLE
I respond quickly and I'd rather hear about a problem early than receive a clean-looking file that's quietly incomplete. Ask questions.
Отвори в Upwork