Senior AI Data Pipeline Engineer (Autonomous Driving)

The role

AI Data Pipeline Engineers build core data processing pipelines and datasets for autonomous driving algorithms. They develop scalable distributed systems for large-scale datasets and high-performance data serving SDKs for ML model training and evaluation.

What you will do

• Develop high scale, reliable data extraction pipeline to extract millions of raw data from data collection fleet and convert to high-value scene data
• Develop data labeling pipelines to perform the auto labeling inferences for autonomous driving algorithms
• Develop advanced autonomous driving data SDK, including scene data search, datasets preparation, dataset loading, etc.
• Build up the data lakehouse for autonomous driving scene dataset, including the sensor data, calibration data, as well as annotation data
• Dig into performance bottlenecks all along the data processing pipelines
• Bootstrap and maintain infrastructure for data platform components
• Collaborate with cross-functional teams to align data pipelines with overall autonomous driving system architecture

Required qualifications

• Bachelor's degree or higher in Computer Science, Engineering, Robotics, or a similar technical field
• Minimum of 7 years of experience in Data Engineering, DataOps or ML Platform roles
• Proficient in Python and solid experience in Python SDK development
• Solid hands-on experience with data pipeline job orchestration with Databricks Workflows or Apache Airflow
• Solid working experience in Databases (e.g., MongoDB, PostgreSQL, etc)
• Extensive experience with data technologies and architectures such as Data Warehouse (e.g., Hive) or Lakehouse (e.g., Delta Lake)
• Experience with Apache Spark or other big data computing engines
• Excellent leadership and communication skills

Nice to have

• Experience with autonomous vehicle sensor data (e.g., LiDAR, camera, radar)
• Experience with ML model training lifecycle (e.g., data preparation, model training / validation / deployment, etc)
• Understanding of modern AI frameworks (e.g., PyTorch, TensorFlow etc.)
• Understanding data governance principles, data privacy regulations, and experience implementing security measures to protect data

Before you apply

• Application Review - Coding Test - 1st interview - 2nd interview - Offer Negotiation - Hiring
• All documents must be submitted in PDF format and under 30MB in size

About the company

42dot is a mobility AI company and the Global Software Center of Hyundai Motor Group. They develop software-defined vehicle technologies and aim to create a self-managing urban transportation operating system.