Senior Data Engineer (Data Platform)

The role

You will design and build a data platform that connects fragmented data sources to power AI agents and Navigator. This involves building containerized services, managing an open lakehouse, and developing fullstack tools for logistics experts.

What you will do

• Design, build, and operate containerized services for external data connections
• Own the open lakehouse including object storage and Apache Iceberg tables
• Build batch and streaming ingestion for disconnected and intermittent networks
• Run the platform on Kubernetes across commercial, government, and air-gapped environments
• Build transformation pipelines to turn raw data into semantic entity models
• Implement entity resolution and data fusion
• Structure unstructured inputs into typed, relational records
• Build lineage, provenance, data quality checks, and confidence labeling
• Build fullstack applications for SMEs to connect sources and map fields
• Partner with Product, AI/ML, Frontend, and Infrastructure teams

Required qualifications

• 8+ years of professional data engineering experience
• Hands-on production experience with an open table format (Apache Iceberg, Delta, or Hudi)
• Experience with a distributed query engine (Trino, Presto, or Spark SQL)
• Demonstrated experience with streaming and batch data pipeline technologies (Kafka, NiFi, Debezium, Spark, or similar)
• Strong proficiency in Python and SQL
• Strong data modeling skills for analytical and operational workloads
• Production orchestration experience (Airflow, Dagster, Argo Workflows, or similar)
• Production experience writing services in Python and/or TypeScript
• Front-end skill (Vue, React, or similar) to ship internal tools
• Experience designing and documenting RESTful APIs
• Strong Kubernetes and Docker experience
• Experience with AWS data services or Azure equivalents

Nice to have

• Active SECRET or TOP SECRET clearance
• Data catalog, lineage, and governance experience
• Data versioning and branching
• Apache NiFi or other low-code/no-code pipeline tools
• Entity resolution, semantic data fusion, knowledge graphs, or ontology modeling
• Shipping data workloads into air-gapped or classified environments
• Palantir Foundry or ontology experience
• Data infrastructure for AI
• Operations research, optimization, or constraint-solver background
• Background in defense logistics, sustainment, or supply chain systems

Benefits

• Generous options grant
• 100% employer-paid health insurance premiums
• 401k
• Opportunities for rapid career growth
• Unlimited PTO
• Free lunches and snacks in the office

Before you apply

• Ability to obtain and maintain relevant security clearances
• Must be able to work in a classified environment when necessary

About the company

Gallatin is rebuilding logistics infrastructure for the national security missions of the United States and allied partners. They build AI systems that determine how logistics decisions are made.