Research Engineer - Web Crawlers

The role

You will own crawling systems that source data from the open web for frontier AI models. You will build and operate large-scale distributed web crawlers and design targeted crawling pipelines.

What you will do

• Build and operate large-scale, distributed web crawlers to discover, fetch, and extract data.
• Solve crawling problems like content extraction from messy HTML, deduplication, freshness, and rate-limit handling.
• Design targeted crawling pipelines for audio, video, and multilingual content.
• Turn crawled data into clean training-ready datasets.
• Create tooling and infrastructure for researchers to request, monitor, and explore web data.

Required qualifications

• Hands-on experience building and scaling web crawlers or scraping systems.
• Strong engineering skills in distributed systems at scale such as Kubernetes, queue-based architectures, or custom pipelines.
• Capacity to autonomously evaluate the quality, coverage, and compliance of crawled data.
• Ability to build tooling to measure data quality, coverage, and compliance.
• Showcase solving hard problems with artifacts such as past projects, designs, or GitHub contributions.

Nice to have

• Experience in support of machine learning training data

Benefits

• Annual discretionary stipend for professional development
• Annual discretionary stipend to meet up with colleagues
• Annual team offsite
• Monthly co-working stipend if not located near a main hub

About the company

ElevenLabs is an AI research and product company transforming how we interact with technology. They serve millions of users and thousands of businesses through platforms like ElevenAgents, ElevenCreative, and ElevenAPI.