Senior Data Engineer

The role

You will design, build, and operate high-throughput distributed data systems for quantitative research and production trading workflows. You will develop Python services and pipelines to process large volumes of market and reference data across on-premises and AWS infrastructure.

What you will do

• Design, build, and operate high-throughput distributed data systems
• Develop production-grade Python services and pipelines
• Build systems for large-scale historical data processing and time-critical overnight pipelines
• Design scalable processing architectures across on-premises compute and AWS
• Build observability and operational tooling to monitor data health
• Collaborate with quantitative researchers, trading teams, software engineers, and infrastructure teams
• Take ownership of systems from architecture through deployment and incident resolution

Required qualifications

• 5+ years of experience building and operating production-grade data platforms, distributed systems, or large-scale data-processing applications
• Strong Python software engineering experience
• Strong understanding of distributed processing concepts such as partitioning, parallelism, concurrency, fault tolerance, idempotency, and failure recovery
• Experience designing systems that process large data volumes with demanding throughput or completion-time requirements
• Hands-on experience with workflow orchestration technologies such as Apache Airflow
• Experience with AWS services including S3, EC2, AWS Batch, IAM, CloudWatch, and cloud SDKs
• Experience operating business-critical pipelines and diagnosing data, performance, and reliability issues in production
• Excellent problem-solving skills

Nice to have

• Experience building market-data platforms or other high-throughput financial data systems
• Knowledge of non-equity asset classes such as futures, fixed income, commodities, or FX
• Experience with time-sensitive overnight processing or systems operating on the path to production trading
• Familiarity with Apache Parquet, Apache Arrow, or distributed processing frameworks such as Spark, Ray, or Dask
• Experience with Docker, Kubernetes, and Infrastructure-as-Code tools such as Terraform or CloudFormation
• C++ development experience
• Experience working in high-performance computing or hybrid cloud environments

Benefits

• Initiatives and programs to enable employees achieve a healthy work-life balance

About the company

Qube Research & Technologies (QRT) is a global quantitative and systematic investment manager operating in all liquid asset classes. It is a technology and data driven group implementing a scientific approach to investing.