Lead Data Engineer

The role

This Lead Data Engineer will lead complex initiatives to modernize data infrastructure, platforms, and pipelines. You will implement core platform systems and set directions and standards for data infrastructure. You will work with various engineering and scientific teams to build secure, reliable, and scalable data engineering solutions.

What you will do

• Serve as technical lead and subject matter expert on complex data engineering projects.
• Lead the implementation of data infrastructure, platforms, tools, and data products.
• Lead the optimization and reliability of data storage solutions.
• Implement complex backend and data integration pipelines.
• Establish and reinforce engineering standards for testing, documentation, release management, and monitoring.
• Lead implementations for platform observability, management, resilience, capacity, and performance.
• Diagnose and resolve complex failures spanning infrastructure and data pipelines.
• Collaborate across I&O, Lab IT, PI Tech, data science, and bioinformatics teams.
• Mentor junior engineers and establish engineering standards.
• Leverage generative AI tools to accelerate pipeline development and data transformation.

Required qualifications

• 6+ years of hands-on experience in data engineering.
• Proven expertise in designing and operating cloud-native data architectures, particularly within AWS.
• Advanced proficiency in Python and SQL.
• Extensive experience with modern frameworks and tools such as Dagster, dbt, Spark, Iceberg, Lake Formation, Athena, Glue, etc.
• Strong experience with Git, CI/CD, CDK, containerized workloads, Docker, and ECS.
• Experience supporting both structured and unstructured data (CSV, JSON, Parquet, imaging, etc.).
• Demonstrated success in leading complex, cross-functional projects involving technical and scientific stakeholders.
• Practical experience using generative AI tools (e.g., Claude Code, GPT-based tools, etc.).
• A track record of leading complex technical initiatives under ambiguity.

Nice to have

• Experience supporting bioinformatics, cheminformatics, or clinical data workflows.
• Familiarity with scientific software and ELN’s, particularly Benchling and CDD.
• Exposure to Agile or Scrum-based development methodologies.
• Relevant certifications (e.g., AWS Solutions Architect Associate, Data Analytics Specialty).

Benefits

• Healthcare coverage
• Annual incentive program
• Retirement benefits
• A broad range of other benefits

About the company

Flagship Pioneering is a life sciences innovation enterprise that invents and builds transformative companies. The Scientific Cloud team provides the data and technology infrastructure across Flagship's portfolio of companies.