Senior / Staff Software Engineer — AI / ML

The role

You will study how frontier-grade data is generated and evaluated to make the process faster, cheaper, and more rigorous. You will work on agentic evals, AI model routing, fine-tuned small models, and predictive difficulty.

What you will do

• Cut the cost of long-horizon agent evaluation with adaptive sampling, statistically grounded early stopping, model cascades, caching, and cheap-first gating.
• Route every eval and judge call to the cheapest model that clears the quality bar, with fallback, monitoring, and cost attribution.
• Fine-tune and serve open-weight models where they match frontier quality.
• Build models that estimate how hard a task is for frontier systems before running a single rollout.
• Build golden datasets and quantify the accuracy and calibration of LLM-as-judge systems.
• Turn research prototypes into reusable, configurable components for engineers and researchers.

Required qualifications

• 5+ years building production ML or software systems, with end-to-end ownership from prototype to production
• Hands-on experience running LLM or ML workloads in production
• Comfort reasoning about non-deterministic systems
• Deep grounding in statistics and experimentation: experiment design, hypothesis testing, sampling, and confidence intervals
• Strong Python and software engineering fundamentals, including testing, code review, and system design
• Experience designing evaluations and interpreting results rigorously
• A habit of finding high-impact problems before they are assigned
• Clear communication with researchers, engineers, and business partners

Nice to have

• Fine-tuning and serving open-weight models, and judging when a smaller model meets the quality bar
• Building LLM evaluation or experimentation platforms, model gateways, or routing systems
• Experience with agentic workloads, benchmarks, or RL environments
• A record of taking research into production: publications, open-source work, or shipped research-driven features
• MS or PhD in Computer Science, Machine Learning, Statistics, or a related field

About the company

Snorkel AI is the frontier AI data lab, helping teams build the data and environments behind high-performing frontier and agentic AI. Founded out of the Stanford AI Lab in 2019, Snorkel works with leading AI labs and enterprises.