Python Scraping Engineer

About Smart Working

At Smart Working, we believe your job should not only look right on paper but also feel right every day. This isn’t just another remote opportunity - it’s about finding where you truly belong, no matter where you are. From day one, you’re welcomed into a genuine community that values your growth and well-being.

Our mission is simple: to break down geographic barriers and connect skilled professionals with outstanding global teams and products for full-time, long-term roles. We help you discover meaningful work with teams that invest in your success, where you’re empowered to grow personally and professionally.Join one of the highest-rated workplaces on Glassdoor and experience what it means to thrive in a truly remote-first world.

About the Role

As a Python Scraping Engineer, you will design, build, and maintain large-scale web scraping systems that collect critical search intelligence data from Google and other complex web environments.

This role sits at the intersection of data engineering, reverse engineering, and distributed systems reliability, requiring deep technical expertise to handle scraping challenges such as dynamic rendering, anti-bot defences, frequent markup changes, and large-scale data collection.

You will develop resilient scraping infrastructure capable of operating reliably against highly protected platforms while ensuring data accuracy, performance, and observability. Working closely with data engineers, data scientists, and product teams, you will help ensure that collected data remains trustworthy and actionable.This role is ideal for engineers who enjoy solving technically challenging problems, reverse engineering complex systems, and building highly reliable distributed data pipelines.

Responsibilities

• Design, develop, and maintain large-scale Python-based web scraping systems
• Build scrapers capable of extracting data from Google and other highly protected websites
• Develop scraping pipelines for dynamic, JavaScript-heavy websites using browser automation
• Continuously adapt scraping systems to changes in page structure, request flows, and anti-bot defences
• Engineer robust data extraction and processing pipelines to ensure high data quality and reliability
• Implement advanced scraping strategies including proxy rotation, fingerprinting, and request routing
• Monitor scraping systems and quickly identify and debug failures in production
• Optimise scraping infrastructure for performance, reliability, cost efficiency, and low latency
• Collaborate with data engineers, data scientists, and product teams to ensure data is usable and trusted
• Maintain clear technical documentation and operational runbooks for scraping systems
• Contribute to the adoption of AI-assisted development tools and agentic coding workflows

Requirements

• Strong professional experience with Python development
• Proven experience building production-grade web scraping systems
• Hands-on experience scraping Google or similarly protected platforms
• Strong understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behaviour
• Experience using browser automation frameworks such as Playwright, Selenium, Puppeteer, or similar
• Strong knowledge of HTML parsing, DOM traversal, and structured data extraction
• Experience handling rate limiting, CAPTCHA systems, IP rotation, and bot detection mechanisms
• Experience building asynchronous or concurrent scraping architectures
• Experience operating scraping systems at scale in cloud environments
• Strong debugging and troubleshooting skills for complex distributed systems
• Experience working with headless browser fingerprinting and anti-bot evasion techniques

Nice to Have

• Experience with Docker or Kubernetes
• Experience working with distributed task systems or job queues
• Experience implementing data quality monitoring or anomaly detection systems
• Experience working with search, advertising, or competitive intelligence datasets

Benefits

• Fixed Shifts: 12:00 PM - 9:30 PM IST (Summer) | 1:00 PM - 10:30 PM IST (Winter)
• No Weekend Work: Real work-life balance, not just words
• Day 1 Benefits: Laptop and full medical insurance provided
• Support That Matters:Mentorship, community, and forums where ideas are shared
• True Belonging: A long-term career where your contributions are valued

At Smart Working, you’ll never be just another remote hire.

Be a Smart Worker - valued, empowered, and part of a culture that celebrates integrity, excellence, and ambition.

If that sounds like your kind of place, we’d love to hear your story.