Member of Technical Staff — Data Infrastructure

The role

You will build the data platform including storage, compute, and loading systems. This involves designing petabyte-scale storage and owning the shared compute and orchestration platform.

What you will do

• Design and operate petabyte-scale storage
• Own the shared compute and orchestration platform
• Optimize data strategy end to end from storage to loading
• Build systems for cataloging, deduplication, lineage, search, and reproducibility
• Implement platform-level quality and monitoring tooling
• Scale infrastructure to improve engineering velocity and ensure reliability
• Build and operate ingestion pipelines for critical data sources

Required qualifications

• Demonstrated experience building large-scale data pipelines and distributed compute systems (e.g. Spark, Ray, Beam)
• Knowledge of state-of-the-art methods and tools for data ingestion, storage, and loading including file formats and storage systems (e.g. Parquet, Zarr, Delta Lake)
• Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines
• Understanding of how data loading throughput affects large-scale training, and experience optimizing it
• Owns deliverables end-to-end, from collecting and translating requirements to autonomously driving execution

About the company

Causal is building a Large Physics foundation Model (LPM) to achieve general causal intelligence. The founding team has built and deployed AI in robotics, drug discovery, and particle physics at institutions like DeepMind, Waymo, Cruise, Insitro, Nabla Bio, and CERN.