Member of Technical Staff - Research

The role

Researchers and research engineers will design and build next generation AI benchmarks. You will create evaluations to assess the real-world capabilities of LLMs.

What you will do

• Design and develop novel, high-impact benchmarks that assess challenging real-world capabilities
• Conduct research to ensure our benchmarks are valid, reliable, and meaningful
• Collaborate with foundation model labs and enterprises to understand evolving evaluation needs
• Analyze model performance across benchmarks and articulate results
• Publish research findings and contribute to the broader evaluation research community
• Work closely with the infrastructure team to implement your benchmark designs at scale
• Stay current with the latest developments in LLM capabilities and evaluation methodologies

Required qualifications

• Master's degree or PhD in Computer Science, NLP, Machine Learning, or related field
• Undergrads with very strong research backgrounds may also be considered

Nice to have

• Experience specifically in LLM evaluation or benchmarking research
• Familiarity with foundation model architectures and capabilities
• Experience working with industry partners or in applied research settings
• Background in areas like human-computer interaction, psychology, or domain-specific evaluation
• Experience at early-stage startups or research labs
• Contributions to open-source evaluation tools or datasets

Benefits

• Highly competitive salary and meaningful ownership
• Relocation and transportation support
• Health/dental insurance coverage
• Lunch and dinner provided
• Free snacks/coffee/drinks
• Unlimited PTO
• $1,500 housing stipend (within one-mile radius)

About the company

The core methodology behind this platform comes from NLP evaluation research conducted at Stanford. The team has prior work experience at NVIDIA, Meta, Microsoft, Palantir and HRT.