Machine Learning Scientist - ML Research

The role

Machine learning scientists will design and analyze experiments to evaluate AI model behavior using human preference signals. You will develop new metrics and methodologies to understand model reasoning, style, and robustness.

What you will do

• Design and conduct experiments to evaluate AI model behavior
• Develop new metrics, methodologies, and evaluation protocols
• Analyze large-scale human voting and interaction data
• Collaborate with engineers to implement and scale research findings
• Prototype and test research ideas rapidly
• Author internal reports and external publications
• Partner with model providers to shape evaluation questions
• Contribute to the scientific integrity of the Arena Intelligence leaderboard and tools

Required qualifications

• PhD or equivalent research experience in Machine Learning, Natural Language Processing, Statistics, or a related field
• Strong understanding of LLMs and modern deep learning architectures
• Proficiency in Python and ML research libraries such as PyTorch, JAX, or TensorFlow
• Demonstrated ability to design and analyze experiments with statistical rigor
• Experience publishing research or working on open-source projects in ML, NLP, or AI evaluation
• Comfortable working with real-world usage data and designing metrics beyond standard benchmarks
• Ability to translate research questions into practical systems and collaborate across engineering and product teams

Benefits

• Competitive compensation aligned to the markets where team members are based
• Medical, dental, and vision
• Additional health and wellness support programs
• Opportunity to work on cutting-edge AI with a small, mission-driven team
• A culture that values transparency, trust, and community impact

About the company

Arena is a platform for evaluating how AI models perform in the real world. Founded by researchers from UC Berkeley's SkyLab, the company builds foundations for understanding and shaping AI.