Reflectionai

Member of Technical Staff - Engineering Lead, Data Acquisition

San Francisco, CA. Posted 24 days ago

The role

You will lead a team of data ingestion engineers building the ingestion layer for training corpora. You will manage web crawling, data ingestion pipelines, and data lakes while making technical and architectural decisions.

What you will do

  • Build, mentor, and grow a high-performing team of data ingestion engineers.
  • Provide front-line leadership across the full ingestion stack.
  • Make targeted individual contributor contributions to the technical stack.
  • Manage day-to-day execution and prioritize team work.
  • Guide technical and architectural decisions regarding scalability, reliability, and cost.
  • Partner with pre-training and data quality teams to evaluate ingestion decisions.
  • Work with data partnerships, digitization operations, and external vendors to onboard new sources.
  • Raise the bar for technical judgment, prioritization, communication, and execution.

Required qualifications

  • Experience building, mentoring, and growing data or infrastructure engineering teams while staying technically hands-on.
  • Deep experience building web-scale data acquisition or ingestion systems with real ownership of production-grade pipelines at multi-TB to PB scale.
  • Strong coding ability.
  • Deep expertise in at least one of: web crawling & acquisition, large-scale extraction & ingestion pipelines, or data lakes / corpus storage & delivery.
  • Working knowledge across the others.
  • Fluency with the modern large-scale data toolkit: distributed compute (Ray, Beam, Spark), orchestration (Airflow, Prefect), formats (Parquet, JSONL, WARC), and object-store / data-lake architectures.
  • Familiarity with how LLMs are trained and evaluated.
  • Ability to guide strategy and drive execution across concurrent data campaigns.
  • Ability to partner effectively with research teams, external data vendors, and partnerships/legal on acquisition.

Nice to have

  • Comfortable growing into leading a larger team quickly if you haven't managed at that scale before.

Benefits

  • Salary and equity
  • Stock options
  • Comprehensive medical, dental, vision, and life
  • Annual wellness allowance
  • Lunch and dinner provided in the office daily
  • 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys
  • Unlimited paid time off in the U.S.
  • 30 days paid time off in the U.K.
  • Visa sponsorship support
  • Regular off-sites, happy hours, and team celebrations

Before you apply

  • This position may require access to technology or source code subject to the U.S. Export Administration Regulations.
  • Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export contr

About the company

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. They build open models that let anyone control their intelligence and help shape the future of AI.

Member of Technical Staff - Engineering Lead, Data Acquisition at Reflectionai | JobZoey