Member of Technical Staff (Evals & Post-Training)

The role

You will work across research, infrastructure, and production systems to build an environment factory and design graders. You will develop methods for mining tasks and creating eval sets to improve model performance through reinforcement learning and post-training techniques.

What you will do

• Building an environment factory that converts recorded enterprise data and task definitions into runnable environments
• Designing graders that turn ambiguous business objectives into verifiable rewards
• Developing methods for mining useful tasks, trajectories, and evaluation cases from historical workflows
• Creating eval sets that are representative, reproducible, and resistant to overfitting
• Finding the right combinations of models, tools, context, and policies to maximize performance while reducing inference cost
• Training and evaluating agents that operate over long horizons, incomplete information, and large tool spaces
• Building replay and observability systems that make agent behavior explainable and measurable
• Scaling from individual environments to thousands of concurrent training and evaluation runs

Required qualifications

• 1-7 years of experience building production software or machine-learning systems
• Understand how environment design, reward design, context, tooling, and policy behavior interact
• Comfortable turning fuzzy business objectives into tasks and signals that can be evaluated reliably
• Can diagnose whether a model’s limitations come from the model itself, its context, its tools, its harness, or its training
• Can move between research questions and production implementation without treating them as separate jobs
• Write strong software and can build systems that process large, messy datasets at scale
• Care about reproducibility, observability, and understanding why a model behaves the way it does

Nice to have

• Working on reinforcement-learning environments
• LLM post-training
• Evaluation infrastructure
• Agent harnesses
• Closely related systems

Benefits

• Significant equity and ownership
• Equinox membership
• Free meals, coffee, and snacks
• Health insurance
• Unlimited PTO

About the company

Ambral Labs helps enterprises own the intelligence behind their most important workflows. They graduated from YC S2025 and are growing the founding team.