Lead Data Engineer, Investment Data Platform

The role

You will design and build a modern investment data platform for an asset allocator client. This includes designing warehouse and lakehouse architecture, building production-grade pipelines, and leading a small engineering team.

What you will do

• Design and build the warehouse and lakehouse, data models, orchestration, and semantic layers.
• Build production-grade Python, SQL, dbt, and orchestration work including batch and near-real-time pipelines.
• Bring software engineering discipline to data including code review, testing, CI/CD, documentation, observability, and data quality controls.
• Lead and mentor engineers through design reviews and pairing.
• Set technical direction and own outcomes end to end.
• Work with investment, operations, finance, and reporting stakeholders to create data products.
• Architect and build core investment data domains such as security and entity reference data, positions and transactions, performance, risk, and private-markets fund data.
• Design data models that are correct, reconcilable, and point-in-time aware.
• Own reliability, performance, and cost of the platform.
• Evaluate tooling and vendors across Snowflake, dbt, Dagster/Airflow, and Azure.

Required qualifications

• 8+ years of hands-on experience in data engineering, analytics engineering, or software engineering focused on data platforms.
• 3+ years leading technical work, as a tech lead, principal or staff engineer, or engineering manager who still builds.
• Expert SQL and Python.
• Deep data modeling and architecture judgment.
• Production experience with modern warehouse and lakehouse platforms (Snowflake, Databricks, or similar), dbt, and orchestration tools (Dagster, Airflow, or similar).
• Strong software engineering practice: version control, code review, automated testing, and CI/CD applied to data.
• Track record of shipping and operating systems you built.

Nice to have

• Experience with investment management systems and data: security master, IBOR/ABOR, OMS, risk, performance, fund accounting, or client reporting.
• Private markets data, including entity mapping across sources without universal identifiers.
• Azure data services (Azure Data Factory, Azure DevOps pipelines, Synapse) and Power BI.
• Experience in asset allocators, asset managers, hedge funds, private markets firms, or financial data providers.

About the company

Castleton Tower is a boutique consulting firm founded by executives from top-tier hedge funds and asset managers. They work with investment management firms to modernize data infrastructure and build AI-ready foundations.