Machine Learning Engineer

The role

You will improve how the system understands and protects sensitive enterprise data. You will work on de-identification pipelines including named-entity recognition, entity resolution, and structured extraction.

What you will do

• Own and improve NER, entity resolution, structured or tabular detection, document understanding, semantic review, or related de-identification systems
• Transform model failures and capability ceilings into a prioritized improvement roadmap
• Design active-learning loops to identify examples for hand-labeling
• Build representative datasets and benchmarks
• Choose and combine deterministic rules, classical ML, fine-tuning, embeddings, multimodal models, and LLM-based approaches
• Design experiments, tune thresholds, and analyze precision-recall and utility tradeoffs
• Productionize improvements with reproducible artifacts and safe rollout
• Optimize inference cost, latency, and throughput
• Build high-fidelity evaluation environments
• Build reliable model- or agent-based harnesses

Required qualifications

• 3+ years of professional machine learning or software engineering experience, including improving models in production
• Startup experience
• Fluent use of modern AI tools
• Experience moving model quality through error analysis, data work, experimentation, implementation, deployment, and iteration
• Strong grasp of precision, recall, F1, calibration, thresholding, class imbalance, imperfect labels, distribution shift, and representative evaluation
• Strong Python and software engineering skills to work inside data pipelines and production systems

Nice to have

• Experience with NER, entity resolution, information extraction, document understanding, multimodal systems, or privacy-preserving ML
• Experience with hyperparameter tuning, data augmentation, model merging, ensembles, knowledge distillation, or multimodal model training
• Experience fine-tuning or adapting transformer, GLiNER, embedding, vision-language, or small specialized models
• Experience with active learning, uncertainty sampling, weak supervision, human-in-the-loop review, or LLM-assisted evaluation pipelines
• Experience building goldens, adversarial corpora, replay systems, model bakeoffs, agentic harnesses, or programmatic evaluation environments
• Experience with difficult ML or labeling problems
• Experience with ONNX Runtime, TensorRT, model pruning, quantization, or other CPU/GPU inference optimization
• Experience with sensitive enterprise data or other high-trust production systems
• Experience with synthetic data generation and managing the synth-to-real gap

About the company

Replay is building the first-ever bank for enterprise data to help businesses monetize historical operating data by licensing it to AI and data labs.