Python Developer (Advanced Web Scraping)

🎯 Role purpose We are looking for an experienced Python Developer with strong web scraping expertise to join our engineering team and take ownership of production scraping projects We are expanding the team to increase our capacity to build new scraping solutions while maintaining existing systems reliably. The goal is to have engineers who can own individual projects, investigate production problems, resolve issues quickly, and continuously improve scraping reliability Within the first few months, we expect the developer to take ownership of existing scraping projects and contribute to the development of new ones 📋 Responsibilities - Build new production scraping services and scraping functionality - Maintain and improve existing production scraping systems - Investigate scraping failures and production defects and follow them through to resolution - Troubleshoot browser automation, HTTP/network, proxy, session, browser compatibility, and source-related issues - Work with advanced browser automation, including Playwright, Selenium Driverless/NoDriver, CloakBrowser, and custom browser environments - Adapt scraping implementations when target websites, APIs, or browser behavior change - Improve scraper reliability, success rates, performance, and maintainability - Work with proxies and specialized browser configurations in production environments - Integrate scraping services with APIs, PostgreSQL, Redis, and surrounding backend components - Monitor production behavior and investigate issues using logs, metrics, and monitoring tools - Validate changes before handing them over to QA - Work with QA during validation and defect resolution - Participate in code review and deliver production-ready changes 🔴 Must-have skills - Strong practical Python experience. This is not an entry-level Python position - Strong commercial experience building and maintaining web scraping systems - Strong experience with Playwright. - Strong experience with Selenium, Selenium Driverless, NoDriver, or comparable advanced browser automation approaches - Experience with specialized or customized browser environments. Direct CloakBrowser experience is valuable, but comparable advanced tooling is acceptable - Advanced understanding of browser behavior, compatibility, environment consistency, and automation reliability - Advanced experience with browser automation in complex web environments - Experience working with proxies, proxy rotation, and related networking challenges - Strong understanding of HTTP and networking fundamentals, including requests, headers, cookies, sessions, redirects, and common failure modes - Experience with advanced browser automation and custom browser tooling - Experience with asynchronous and concurrent Python - Experience with PostgreSQL and Redis - Experience with Docker and Linux environments - Strong production debugging and troubleshooting skills - Experience working with and integrating external APIs - Ability to test and validate your own implementation before QA - Experience maintaining systems that are already running in production 🟢 Nice-to-have skills - Direct experience with CloakBrowser - Mobile or Android-based scraping experience - Experience building or operating high-volume or distributed scraping systems - Experience with Prometheus, Grafana, Kibana, or Elasticsearch - Experience with gRPC 🏆 Expected ownership This role requires ownership beyond implementing isolated development tasks. You will be expected to: - Independently investigate production defects within the projects you own - Take responsibility for maintaining assigned scraping services and following problems through to resolution - Deliver production-ready changes and validate your implementation before QA - Work within a mandatory code-review process before production deployment - Use technical supervision and code review effectively without depending on constant direction - Communicate blockers, technical uncertainty, risks, and delays clearly and early - Provide realistic estimates and maintain reasonable delivery predictability - Take increasing ownership of existing projects and contribute to the design and development of new scraping systems 💼 Project context You will work across multiple production scraping projects rather than on a conventional CRUD/backend application. Our systems combine Python services, browser automation, HTTP/API integrations, proxies, databases, specialized browser automation tooling, and specialized browser environments A significant part of the technical challenge comes from working with complex and frequently changing web environments. The environment includes technologies such as Python, Playwright, Selenium Driverless/NoDriver, CloakBrowser, custom browser tooling, PostgreSQL, Redis, Docker, Prometheus, Grafana, Kibana, and Elasticsearch Some projects also involve mobile/Android-based scraping and device/browser environments. These are live production systems, so the role combines new development with continuous monitoring, debugging, maintenance, and reliability improvements ⚠️ This position is probably not a good fit if your experience is primarily general Python backend or CRUD development without substantial web scraping experience. Likewise, basic scraping experience limited to simple requests, HTML parsing, or conventional Scrapy projects will not be sufficient for the technical challenges of this role