Senior Data Engineer

The role

The Senior Data Engineer will design, build, and operate large-scale, data-intensive platforms in the cloud. This role involves developing reliable data pipelines, scalable data processing systems, and high-availability architectures for analytics, machine learning, and data science.

What you will do

• Designs and implements sophisticated software systems using object-oriented or functional programming in Python, Scala or Java.
• Participate in and adhere to professional software engineering practices using Agile, TDD, CI, git, Jira, and Stash.
• Maintain an advanced level of proficiency with Computer Science/Software Engineering knowledge.
• Work independently and as part of a team.
• Mentor junior engineers in routine, less-complex tasks.
• Lead small projects.
• Other duties as assigned.

Required qualifications

• Bachelor's or Master's in a related field, or commensurate experience
• 5+ years in Computer Science, with a proven track record of successful projects
• 5+ years of experience operating and deploying solutions using AWS Service including S3, EMR, SQS, SNS, Lambdas, Athena, Glue, Kinesis, and MSK
• 5+ years of experience with one or more of the following programming languages: Python, Scala, or Java
• 5+ years of experience with SQL
• 5+ years of experience with big data technologies such as Cassandra, Spark, Hadoop, Parquet, Redash, etc.
• Advanced proficiency in building data integrations using both API and file-based protocols
• Advanced proficiency with test-driven development and unit testing
• Experience with Git Version Control or similar
• Experience with CI/CD pipeline patterns and best practices
• Proficiency with alerting and monitoring tools like Grafana
• Demonstrated ability to learn new technologies and languages

Benefits

• Flexible Time Off program
• Hybrid work model
• Variety of medical insurance options, including a $0 cost premium employee coverage
• Competitive 401K match with no vesting requirement
• National health, dental, and vision plans
• Progressive family plan benefits

About the company

The Weather Company is the world’s leading weather provider, helping people and businesses make more informed decisions and take action in the face of weather. The company reaches hundreds of enterprise clients and more than 360 million monthly active users via its digital properties.