Backend Engineer — Data Pipeline

The role

You will design and ship backend systems for multi-stage, high-volume data processing. You will own the backend systems and contracts that make data de-identification pipelines correct, replayable, and safe.

What you will do

• Design and ship backend systems for multi-stage, high-volume data processing
• Define authoritative, versioned contracts for manifests, artifacts, lineage, and state transitions
• Make retries, checkpoints, partial failures, replay, backfills, migrations, and rollbacks safe and understandable
• Build independent reconciliation and verification
• Turn escaped and recurring failures into fixtures, regression coverage, release gates, and durable recovery paths
• Expose trustworthy run state and safe controls to products
• Diagnose production behavior across code, queues, stores, artifacts, data formats, and deployed versions
• Improve correctness, throughput, and operating leverage
• Use AI deeply in development and in bounded verification systems
• Partner with machine learning, applied science, full-stack product, platform, security, and data engineering teammates

Required qualifications

• At least three years of professional software engineering experience, including personal ownership of production backend systems
• Strong in asynchronous or distributed systems and can reason precisely about queues, concurrency, state, storage, idempotency, partial failure, and recovery
• Worked on systems where output could be materially wrong even when every service looked healthy
• Can define invariants and use reconciliation, control totals, diffs, replay, goldens, shadow paths, or independent sources to verify correctness
• Can design versioned data and artifact contracts and migrate them safely in a live system
• Can debug from evidence across system boundaries and turn incidents into durable system improvements
• Can choose technical work based on operator and customer consequences
• Uses modern AI engineering tools fluently, verifies their output, and knows when model-backed checks need deterministic guardrails and human review
• Communicates clearly across product, ML, data, platform, security, and customer-facing teams

Nice to have

• Experience with large-scale batch processing, workflow orchestration, event-driven systems, or data movement
• Experience with schema evolution, manifests, lineage, CDC, migrations, reindexing, or backfills
• Experience in payments, ledgers, reconciliation, claims, fraud, identity, search quality, observability, or another domain with delayed or weak ground truth
• Experience with sensitive or multi-tenant data, least-privilege systems, auditability, quarantine, and fail-closed release paths
• Experience combining deterministic checks, synthetic fixtures, offline replay, model-based judges, and human review
• Experience with Python, AWS, Airflow, Batch, SQS, S3, DynamoDB, PostgreSQL, or comparable systems

About the company

Replay is building the first-ever bank for enterprise data. They help businesses monetize historical operating data by licensing it to frontier AI and data labs for model training.