Python Developer for Ethical Web Data Extraction

Job Description: We are looking for an experienced Python Developer to build reliable and ethical tools for extracting publicly available data from websites and web applications. The goal is to create automated solutions that collect, clean, and structure data from sources where data extraction is permitted. The work will focus on building maintainable and efficient data pipelines while strictly following ethical practices and respecting website terms of service and robots.txt files. Responsibilities: Build and maintain web data extraction tools using Python Extract data from static and dynamic (JavaScript-rendered) websites where permitted Develop clean, well-structured, and reusable code for data collection Clean, validate, and organize extracted data into usable formats (CSV, JSON, Excel, or database) Implement proper error handling, logging, and data validation Document the code and processes clearly for future maintenance Required Skills: Strong proficiency in Python Experience with BeautifulSoup, Requests, and Scrapy Experience with Playwright or Selenium for handling dynamic websites Solid understanding of HTML, CSS Selectors, and XPath Proficiency in Pandas for data cleaning and processing Experience storing data in databases (such as PostgreSQL or MongoDB) Strong skills in writing clean, maintainable code with proper error handling and logging Good understanding of ethical data extraction practices and respect for website terms of service Preferred Skills: Experience with the Scrapy framework Knowledge of working with public APIs (preferred over scraping when available) Experience building scheduled or automated data pipelines Familiarity with best practices for responsible and ethical web data collection Important Notes: The work will only involve websites and data sources where extraction is permitted. We expect the developer to follow ethical practices and respect robots.txt and website terms of service. Preference will be given to solutions that use public APIs when available. What to Include in Your Proposal: Your experience with web data extraction or automation projects (please share 1-2 examples) Which tools and frameworks you prefer to use (Scrapy, Playwright, etc.) Your approach to building ethical and maintainable data extraction solutions Your hourly rate or fixed price for similar projects Your availability We are looking for a professional and detail-oriented developer who can deliver clean, reliable, and well-documented solutions.