Data Engineer

The role

You will design, build, and manage data pipelines and core tables. You will develop canonical datasets to track product metrics and collaborate with various teams to provide data solutions.

What you will do

• Design, build and manage data pipelines.
• Integrate user event data into the data warehouse.
• Develop canonical datasets to track user growth, engagement, and revenue.
• Work with Infrastructure, Data Science, Product, Marketing, Finance, and Research teams.
• Implement robust and fault-tolerant systems for data ingestion and processing.
• Participate in data architecture and engineering decisions.
• Ensure security, integrity, and compliance of data.

Required qualifications

• 3+ years of experience as a data engineer.
• 8+ years of any software engineering experience (including data engineering).
• Proficiency in at least one programming language such as Python, Scala, or Java.
• Experience with distributed processing technologies and frameworks, such as Hadoop, Flink and distributed storage systems (e.g., HDFS, S3).
• Expertise with any of ETL schedulers such as Airflow, Dagster, Prefect or similar frameworks.
• Solid understanding of Spark and ability to write, debug and optimize Spark code.

Benefits

• Relocation assistance

Before you apply

• Background checks for applicants will be administered.

About the company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.