Senior Staff Engineer, Data Platform

The role

You will set the technical direction for the Data Platform and build it hands-on. You will own the Databricks lakehouse and the AI pipelines that run alongside it, using Temporal for durable orchestration.

What you will do

• Architect and evolve the Databricks lakehouse, including medallion layering, Delta Lake, Unity Catalog governance, and workspace and compute strategy.
• Design, build, and optimize Spark and SQL pipelines on Databricks, including performance tuning and cost management.
• Define data contracts, quality checks, access controls, and PHI-aware governance with security and compliance.
• Build data foundations for AI products, including LLM-ready datasets, retrieval and vector-store pipelines, and evaluation data.
• Build durable AI pipelines on Temporal, including pulls from Databricks gold tables and long-running LLM extraction jobs.
• Define how the team does agentic development, including tooling, guardrails, code review, and test strategy for agent-written code.
• Write and review technical designs, set platform-wide standards, and make long-term build-versus-buy decisions.
• Mentor engineers across teams, unblock other pods, and communicate trade-offs to technical and non-technical stakeholders.

Required qualifications

• Bachelor's degree in Computer Science, Electrical Engineering, Mathematics or a related field
• At least 12 years of software or data engineering experience, including designing and operating production data platforms at scale and setting architecture across multiple teams
• Experience in a highly regulated domain such as Healthcare, Insurance or Finance
• Deep, hands-on expertise with Databricks; expertise in Snowflake or Microsoft Fabric is also considered aligned
• Experience with Spark, Delta Lake, Unity Catalog, Workflows or equivalent, and performance and cost tuning at scale
• Experience building pipelines for ML or LLM applications, such as LLM-based extraction, embeddings and vector search, feature pipelines, or evaluation datasets
• Fluency in Python and SQL, with testing, CI/CD, infrastructure as code, and code review practices
• Demonstrated practical use of agentic coding tools (such as Claude Code or similar) in real delivery work
• Experience owning ambiguous, cross-team problems end to end and influencing without direct authority
• Clear written and verbal communication, including authoring design docs

Nice to have

• Experience with healthcare data such as claims, EHR, FHIR or HL7, and HIPAA-compliant handling of PHI
• Databricks certifications or experience with Delta Live Tables, Lakeflow, or Databricks Asset Bundles
• Experience with other orchestrators (Airflow, Dagster, Prefect)
• Experience with cloud infrastructure (GCP, AWS, or Azure), Terraform, and Kubernetes
• Contributions to open-source data, workflow, or AI infrastructure projects

Benefits

• Competitive salaries with equity packages
• Medical, dental and vision insurance
• Flexible working hours
• Hybrid work options
• Inclusive environment that fosters creativity and innovation

Before you apply

• Experience in a highly regulated domain (Healthcare, Insurance, Finance) is required

About the company

Qualified Health is redefining what is possible with Generative AI in healthcare, providing guardrails for safe AI governance, healthcare-specific agent creation, and real-time algorithm monitoring. It works with leading health systems and is backed by premier investors.