Experienced web scraping specialist for large eBay store

We are seeking an experienced web scraping specialist for a single eBay Shop with circa 2 million products (UK site only). Please note that we scrape sites and we are looking for someone to create the script for us and help to install on our infrastructure and then work with us moving forward. We are NOT just looking for the data in a one-off exercise, though we will expect you to help to gather a full set of data with the script in order to sign it off (to show that blocking has been overcome) There is ongoing work for the person who we choose as we currently run over 2000 crawlers daily. The reason for coming onto Upwork is because we are very busy currently and don't have the time and this is a good way to find people to work with ongoing :) PLEASE DO NOT BID IF: 1. YOU ARE NOT EXPERIENCED IN SCRAPING AND OVERCOMING BLOCKING, ESPECIALLY RELATING TO EBAY 2. YOU ARE PART OF AN AGENCY - WE ARE ONLY LOOKING FOR INDIVIDUALS TO WORK WITH 3. YOU USE COPY+PASTE FOR ALL OF YOUR BIDS OR USE AN AI BOT (IRONIC AS YOU WILL NOT READ THIS IF YOU ARE A BOT!). WE WILL AUTOMATICALLY REJECT ANY OF THESE The shop is on eBay UK only, so there are no regional complications - the challenge here is the sheer size of the shop (around 2 million products) and eBay's blocking at this scale. We already have a crawler built but have been having issues with blocking. We may need to go to the product detail pages in order to get all of the information that we require, depending on whether it is available at the listing view - we will confirm this once we start working together. We need a full crawl of the entire shop monthly. In between the full crawls, we will need re-crawls of only the products where we have a match to our own range, so a much smaller subset. Please state what you will use to do the scraping. Currently we use Python + Scrapy for most of our crawlers, but we are flexible and use some headless and also Node.js We also need to know what other resources you may require (e.g. proxies). Please note that we are looking for an hourly rate, but will probably ask for a fixed rate once we fully understand what you would need for the project As you know we will get a lot of bids and what can you tell us that will help us to choose you to work with? Anyone saying "let's jump on a call" or similar will be deleted. Happy to do a call once we have narrowed down Looking forward to hearing from you