Node.js + Puppeteer Automation Engineer: Automated Web Scraper for E-Commerce Registries

We require a senior-level software engineer with deep expertise in browser-evasion strategies to build a lightweight, high-performance web-scraping script. This script will track dynamic transaction and inventory parameters from primary registration engines in the endurance sports sector. You do not need to touch a website frontend, design user interfaces, or configure front-facing web layouts. Your sole responsibility is to write a bulletproof backend script that runs automatically on a nightly schedule, extracts targeted metrics from a provided list of primary target URLs, and exports them cleanly to a private cloud folder. A signed NDA is a strict requirement prior to contract award and code sharing. Core Technical Requirements • The Required Stack: Strictly Node.js utilizing Playwright or Puppeteer (Playwright preferred for WebKit context shifting). Standard Python/BeautifulSoup scripts will not be accepted due to heavy dynamic JavaScript rendering, interactive element containers, and aggressive anti-scraping firewalls present on the target websites. • Target Environments: The script must navigate and extract data from 3 to 4 major consolidated industry registration engines (specifically RunSignUp, Active.com, and Race Roster) [1.3]. • Data Extraction Points: For our queue of target URLs, the script must parse the pages and extract: 1. Current Inventory / Registration Capacity (e.g., parsing text strings indicating "85% Full", "50 Slots Left", or registration tiers closed). 2. Price Increase Timelines & Dates (e.g., parsing upcoming price-jump dates or registration cutoff deadlines). 3. Temporal Status Recognition: Look at the year in the event date string. If the event date is in the past, or if the page text explicitly reads 'Registration is Closed,' assign a specific flag in our output (e.g., Status: Expired/Closed) [1.5]. • Scheduling & Integrity: Optimize the script as a lightweight batch-processing routine configured to run on a nightly cron-job scheduler at 2:00 AM EST. The script must incorporate rotating premium residential/mobile proxies, randomized mouse path behavior (e.g., Bézier curves/ghost-cursor), and deep fingerprint randomization to prevent request throttling or IP blocking by Akamai and Cloudflare. • Error Handling: Implement robust error-logging. If an individual target URL modifies its HTML layout, the script must safely log the issue, skip that single record, mark that specific attribute as "Layout_Modified", and continue processing the rest of the queue without crashing the nightly synchronization loop. • Output Deliverables: Generate a unified, structured data file (CSV or JSON format) where each row is keyed by a unique identifier. This file must automatically overwrite and upload to a secure, private cloud folder (Google Drive or Dropbox API) on a daily basis. 🕒 Timezone & Availability Requirements (Strict) The automated script executes nightly at 2:00 AM US Eastern Standard Time (EST). • You must have availability to review log outputs, debug firewall blocks, and optimize proxy performance shortly following the 2:00 AM EST run. • Candidates in Latin America (aligned hours) or Europe (morning hours during the run) are highly preferred. • If you are in an Asian/Oceanian timezone, you must explicitly confirm your willingness to adjust your shift to monitor the live 2:00 AM EST deployment. Project Scope & Structure (Fixed Budget: $2,500) This is a discrete project with an expected timeline of 2 to 3 weeks. Payments are split into 3 strict, quality-verified milestones: • Milestone 1 (25% - $625): Core Node.js environment initialized, premium rotating proxy framework integrated, and anti-fingerprint verification perfectly passing stealth tests. • Milestone 2 (50% - $1,250): All 3-4 target engines successfully mapping registration capacities, pricing cutoff dates, and handling historical date-checking flags gracefully. • Milestone 3 (25% - $625): Fault-tolerant try-catch logic active, daily CSV/JSON file overwrites automated, cloud drive API integration finalized, and the script completes 3 consecutive nights of flawless execution at 2:00 AM EST in a staging sandbox. Screening Questions (Please answer when applying) 1. Walk us through your preferred method for handling proxy rotation, custom headers, and simulating human interaction within Node.js + Playwright/Puppeteer to avoid getting throttled by heavily protected e-commerce sites utilizing Akamai Bot Manager. 2. How will your script handle an individual target site changing its HTML layout without crashing the entire nightly database synchronization routine? 3. Please share a brief example or description of a dynamic, headless browser web scraper you have previously built that bypassed an aggressive enterprise firewall.