Data Annotation Specialist, Safety

The role

You will label, rank, audit, and refine human- and model-generated text to improve model safety and quality. This includes creating prompts and safety test cases to evaluate model behavior.

What you will do

• Label, rank, audit, and refine human- and model-generated text
• Assess model outputs against detailed safety guidelines, rubrics, and style standards
• Write realistic prompts, user scenarios, and adversarial examples
• Identify annotation inconsistencies or unclear guidelines
• Provide actionable feedback on recurring edge cases and model failures
• Complete annotation tasks with strong attention to detail

Required qualifications

• 1+ years of experience in Content Moderation, Trust and Safety, AI data annotation, LLM evaluation, or a related analytical role
• Experience applying detailed guidelines to complex and sensitive content
• Strong contextual and sociocultural judgment
• Ability to recognize and manage personal bias
• Emotional resilience to work with unsafe, explicit, and/or toxic content including sexual, violent, or psychologically disturbing nature
• Excellent command of written English
• Ability to clearly justify content evaluations
• Strong attention to detail and commitment to accuracy
• Ability to work independently in a global, asynchronous team
• Ability to use a personal laptop (BYOD)
• Must be located in Canada or the United States

Nice to have

• exposure to quality assurance, red teaming, and/or prompt engineering
• fluency in another language

Before you apply

• Initial Screening of resume and writing samples
• Virtual Annotation Test including writing sample and interacting with a chat bot
• Video Screen with Operations Team
• Exposure to explicit, offensive, or inappropriate NSFW text content

About the company

Cohere is a team of researchers, engineers, and designers training and deploying frontier models for developers and enterprises. Their mission is to scale intelligence to serve humanity.