Data Engineer

The role

A data engineer will build ingestion and search systems for AI products. You will own systems from source acquisition through to serving queries using global patent literature, case law, and other data sources.

What you will do

• Build and operate high-throughput, resumable pipelines for large datasets.
• Extract useful content from complex documents and other formats.
• Handle malformed records and changing schemas.
• Validate outputs while preserving structure and metadata.
• Build keyword, vector and structured search.
• Design schemas, indexes and partitioning for fast queries.
• Link patents, scientific records and supporting documents.
• Profile parsing, ingestion, database builds and queries.
• Diagnose CPU, memory and storage I/O bottlenecks.
• Tune jobs and infrastructure for throughput, latency and cost.

Required qualifications

• Strong Python and SQL, with experience designing and operating production databases.
• Solid experience building and operating production data pipelines over large, messy datasets.
• Expertise with running search systems over large document collections.
• End-to-end ownership from raw data to user-facing functionality.
• A good understanding of schema design, indexing and query optimisation.
• A track record of diagnosing and fixing performance bottlenecks in live systems through profiling and measurement.

Nice to have

• Experience with PostgreSQL/pgvector
• OpenSearch (or Elasticsearch)
• Spark/Delta Lake
• AWS
• NoSQL databases
• Rust/C++

Benefits

• Competitive Salary
• Significant Equity
• Full visa sponsorship
• Private medical insurance
• Free meals

About the company

Solve Intelligence is a fast-growing startup transforming the IP industry. The company has received $55M in total funding from investors including Y Combinator, 20VC, Visionaries, Microsoft and Thomson Reuters.