Software Engineer - Data Infrastructure

The role

You will design and build data pipelines to ingest, process, and transform user vote data. You will work with researchers and product leadership to scale data infrastructure and generate result dashboards.

What you will do

• Design and build robust data pipelines to ingest, process, and transform user vote data.
• Collaborate with researchers and product leadership to understand product goals and necessary data.
• Design and implement solutions to generate result dashboards and reports.
• Ensure the integrity, data quality, and reliability of the pipelines.
• Scale data infrastructure to accommodate increasing data volumes and evolving analytical needs.

Required qualifications

• 5+ years of experience in software engineering, with a dedicated focus on data engineering and big data technologies
• Proficiency in SQL
• Proficiency in at least one programming language commonly used for data analysis (Python, Scala, R)
• Hands-on experience with data processing and pipeline frameworks (Apache Spark, Ray Data, etc.)
• Hands-on experience with at least one popular big data analytics platform (Databricks, Snowflake)
• Demonstrated experience in designing, implementing, optimizing, and debugging production data pipelines

Nice to have

• Prior work in data analytics or datalake platforms
• Experience in advanced data analysis tools, such as Delta lake, streaming tables
• Exposure to machine learning

Benefits

• Competitive compensation aligned to the markets where team members are based
• Medical benefits
• Dental benefits
• Vision benefits
• Additional support programs

About the company

Arena is the platform for evaluating how AI models perform in the real world. Founded by researchers from UC Berkeley's SkyLab, the company builds the foundation for understanding, shaping, and benefiting from AI.