Adthena

Senior Anti-Bot / Scraping Engineer

London I UK Remote. Posted a month ago

The role

You will design, build, and operate large-scale, resilient web scraping systems focused on Google and Google-like search environments. The role involves working on complex scraping problems including dynamic rendering, anti-bot defences, and high-volume data collection.

What you will do

  • Design and maintain large-scale Python scraping systems
  • Build scrapers for dynamic, JavaScript-heavy pages using browser automation
  • Adapt to changes in page structure, request flows, and anti-automation mechanisms
  • Engineer robust extraction pipelines
  • Implement proxy, fingerprinting, and request-routing strategies
  • Monitor scrape health and debug production issues
  • Optimise performance, cost, and latency across scraping infrastructure
  • Work with data engineers, data scientists, and product teams
  • Document scraping logic and operational runbooks

Required qualifications

  • Strong professional experience with Python (or another main language, i.e. GoLang, Rust, JavaScript) including production-grade scraping systems
  • Deep hands-on experience scraping Google or similarly protected platforms
  • Solid understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behaviour
  • Experience with tools such as Playwright, Selenium, Puppeteer, or equivalent browser automation frameworks
  • Strong knowledge of HTML parsing, DOM traversal, and data extraction techniques
  • Experience handling rate limiting, CAPTCHA, IP rotation, and bot detection systems
  • Comfortable working with asynchronous and concurrent scraping architectures
  • Experience running scrapers at scale in cloud environments
  • Strong debugging skills and ability to reason about complex failure modes
  • Experience with headless browser fingerprinting and anti-bot evasion techniques

Nice to have

  • Familiarity with containerisation and orchestration, such as Docker and Kubernetes
  • Experience with distributed task systems and job queues
  • Background in data quality monitoring and anomaly detection
  • Experience working with search, advertising, or competitive intelligence data

Benefits

  • Stock Options
  • Trust-Based Vacation
  • Remote-First or Hybrid Work
  • Flexible Work
  • Career Growth via Sherpa plans
  • Monthly Hackdays & Training Days
  • Up to 6 months fully paid maternity leave
  • 2 months paternity leave
  • Home Office Stipend: $200
  • Swag Welcome Gift: $70 credit
  • Birthday Day Off
  • Regular Socials: Summer & Christmas parties, annual and quarterly offsites, monthly meet-ups

Before you apply

  • 2 Stages of interviews

About the company

Adthena is a search intelligence platform powered by patented AI technology. It helps brands, marketers, and agencies with insights into the paid search market.