Software Engineer – Query Engines

The role

You will design and build query planning, optimization and execution features for the data processing engines behind Palantir Foundry. The work centers on Apache Spark, DataFusion and Iceberg and on the performance and correctness of large data systems.

What you will do

• Design and implement query planning and optimization that turns complex computations into efficient execution plans
• Extend execution engines, including query operators, parallelism, memory management and data movement
• Develop Foundry's Iceberg catalog and engine integrations, including table metadata, transactions, and efficient reads and writes
• Build shared transformation semantics and execution interfaces for workloads across Foundry
• Improve incremental processing so pipelines reuse previous results while preserving correctness
• Evaluate and integrate advances in open-source data systems against real-world workloads
• Investigate correctness and performance issues and build tests and benchmarks that prevent regressions
• Work with product teams and customers to turn operational needs into engine capabilities

Required qualifications

• 4+ years of professional software engineering experience developing database engines, distributed data processing systems, storage systems, or comparable infrastructure
• Depth in query planning, execution, or performance optimization
• Operating on top of these technologies alone does not qualify
• Experience diagnosing correctness and performance problems in complex systems
• Strong written and verbal communication skills
• Ability to work effectively across teams, incorporate feedback, and hold a high bar for quality

Nice to have

• Experience building or extending systems such as Spark, DataFusion, Iceberg, or comparable technologies
• Experience contributing to open-source projects

Benefits

• Life at Palantir programs for health and well-being, with offerings that may vary by region

Before you apply

• Palantir will never ask you to submit a payment or share financial information during the interview process
• Palantir will provide a reasonable accommodation for applicants living with a disability on request

About the company

Palantir builds software for data-driven decisions and operations used by partners to develop drugs, forecast supply chain disruptions and locate missing children. Its platforms include Palantir Foundry.