Cylake Inc
Data Pipeline Engineer
Sunnyvale. Posted 8 months ago
The role
You will design, build, and maintain a scalable, open-source data lakehouse architecture. You will architect end-to-end data pipelines from ingestion through transformation to consumption.
What you will do
- Design, build, and maintain a scalable, open-source data lakehouse architecture supporting petabyte-scale analytics workloads
- Architect end-to-end data pipelines from ingestion through transformation to consumption
- Ensure high performance, reliability, and data quality
Required qualifications
- Proven track record of success architecting, building, and running large-scale data systems (PB scale)
- Experience with both batch and real-time processing architectures
- Experience with open source Data Lakehouse components, including Apache Iceberg, PostgreSQL, Neo4j, Apache Parquet, etc.
- Experience with tools for stream processing and data analytics - Apache Kafka, Spark, Flink, etc.
- Understanding and experience with data transformation solutions
- Excellent programming experience with Python
- Strong communication and documentation skills
Nice to have
- Experience with CSP data platforms
- Understanding of data lineage, quality, and governance tooling
Benefits
- Competitive compensation
- Comprehensive benefits package
Before you apply
- Requires access to information protected under U.S. export control laws and regulations, including EAR and ITAR
- Employment is conditioned on authorization to receive software or technology controlled under these laws without sponsorship for an export license
About the company
cylake-inc is a small team building the next generation of cybersecurity products from the ground up.