Magic.Dev
Member of Technical Staff, RL Research & Environments
San Francisco. Posted 24 months ago
The role
You will design and operate data, evaluation, and environment systems to improve model capabilities after pretraining. This role focuses on post-training tasks like building datasets, designing reward signals, and running iterative training loops.
What you will do
- Design and build post-training datasets using synthetic generation, targeted data collection, and self-play
- Implement filtering, scoring, and mixture strategies for RL and post-training corpora
- Build and maintain evaluation frameworks that surface long-context failure modes
- Design reward signals and training environments for targeted capability improvements
- Run ablations across data sources, reward designs, and long-horizon task structures
- Improve reliability and observability of post-training data and environment pipelines
- Collaborate closely with Product and Research to translate capability goals into measurable iteration cycles
Required qualifications
- Strong software engineering fundamentals
- Experience building or operating large-scale data or ML systems
- Ability to design and interpret experiments that measure model behavior changes
- Comfort working at the intersection of ML, data systems, and infrastructure
- Strong attention to data quality and evaluation rigor
- Track record of owning experimental or production systems end-to-end
Benefits
- 401(k) plan with 6% salary matching
- Generous health, dental and vision insurance for you and your dependents
- Unlimited paid time off
- Visa sponsorship and relocation stipend to bring you to SF, if possible
About the company
Magic's mission is to build safe AGI that accelerates humanity's progress on the world's most important problems. The company believes the most promising path to safe AGI lies in automating research and code generation.