Senior Software Engineer

The role

You will expand the reach of Lance and LanceDB within the data infrastructure ecosystem. You will work on high-performance computing, big data, and open-source systems.

What you will do

• Designing and maintaining efficient distributed Lance dataset operations
• Building efficient indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino
• Working on table formats, data encodings, and various aspects of the Lance format in Rust
• Driving open-source community efforts to integrate the Lance format with Spark, Hive Metastore, Presto, Trino, Ray, and other data infrastructure systems
• Operating and improving internal data processing infrastructure
• Promoting the Lance format in open-source communities and at Big Data conferences

Required qualifications

• 10+ years of experience building high-performance databases, big data systems, or large-scale data services
• Deep understanding of internals of open-source Big Data or AI training systems (e.g., Hadoop, Spark, Flink, Ray, Iceberg, Delta Lake, Hudi, ClickHouse, Trino, Presto, PyTorch, or JAX)
• Strong experience with high-performance computing in C++, Java, and/or Scala
• Experience with Rust (or willingness to learn it)
• Proven ability to move fast, work independently, and collaborate with a high-caliber team

Nice to have

• Contributor, committer, or PMC member in Apache or other large open-source projects
• Experience with Apache Arrow, DataFusion, Parquet, Iceberg, or Delta Lake
• Track record of driving large features or integrations in distributed systems
• Strong community presence and passion for open-source collaboration

About the company

LanceDB is the AI-native Multimodal Lakehouse. Customers like Runway, Midjourney, and Netflix build the future of AI on LanceDB.