Senior Data Engineer

The role

You will design and build scalable data platforms and pipelines to power identity verification products and analytics. You will also automate data operations and optimize the cost, performance, and reliability of data workloads.

What you will do

• Design and build batch and streaming data pipelines.
• Own end-to-end delivery of complex data initiatives including architecture, implementation, testing, deployment, monitoring, and documentation.
• Develop and evolve the data platform to support large-scale data processing.
• Automate data operations like validation, quality checks, alerting, backfills, and recovery workflows.
• Optimize cost, performance, and reliability of data workloads.
• Partner with Data Science, Product, and Engineering teams to translate requirements into technical solutions.
• Evaluate and adopt new technologies including processing engines, storage formats, orchestration tools, and GenAI-assisted ingestion.

Required qualifications

• 5+ years of hands-on data engineering experience building and maintaining production-grade data platforms and pipelines.
• Strong programming skills in Python or Scala for data processing.
• SQL for data analytics.
• Deep experience with distributed data processing frameworks such as Apache Spark, including performance tuning and optimization.
• Proven experience building data solutions using services on AWS (EMR, Lambda, s3, etc).
• Strong understanding of data modeling and data warehousing concepts, including partitioning and schema design for large-scale datasets.
• Experience operating and supporting production pipelines, including monitoring, alerting, incident response, and improving reliability.
• Solid foundation in software engineering practices, including version control, CI/CD, testing strategies, and code review.
• Strong communication and collaboration skills.

Nice to have

• Experience with streaming or near-real-time data processing (Kafka, Kinesis, etc).
• Hands-on experience with data orchestration tools (Airflow, Step Functions, etc).
• Familiarity with modern data platform patterns such as Data Lakehouse, Data Mesh, and large-scale data sharing across teams.
• Experience with prompt engineering using modern GenAI, Large Language Models (LLM).
• Experience mentoring other engineers and contributing to engineering-wide standards, best practices.

Before you apply

• Socure cannot provide sponsorship now or in the future for this role.

About the company

Socure is building the identity trust infrastructure for the digital economy by verifying identities in real time and stopping fraud.