Domain Advisor Consultant (Subject Matter Expert)

The role

Subject Matter Experts will partner on AI evaluation and training projects. You will review quality criteria, curate benchmark data, and analyze model failures to shape Large Language Models.

What you will do

• Review and advise on quality criteria used to evaluate AI outputs
• Create, edit, or certify Golden Tasks as benchmark examples
• Analyze model failures and provide technical breakdowns
• Consult on the difficulty and nuance of specific prompts

Required qualifications

• Advanced degrees (PhD, MD, JD) or equivalent industry-recognized status
• Currently operate as an independent consultant or freelancer
• Have own workflows, equipment, and professional methodology
• Ability to articulate reasoning process clearly in writing
• Must be properly registered to operate as independent contractors in your country of residence

Nice to have

• Prior experience in AI evaluation

About the company

agency is looking to engage elite Subject Matter Experts to partner on high-impact AI evaluation and training projects.