Member of Technical Staff, Pre-Training Data

The role

You will develop the data pipeline that underpins advanced language models. You will conduct data ablations to evaluate data quality and construct pre-training data mixtures to enhance model performance.

What you will do

• Conduct data ablations to assess data quality and experiment with data mixtures to enhance model performance.
• Develop robust data modeling techniques to ensure datasets are structured and formatted for optimal training efficiency.
• Research and implement innovative data curation methods using Cohere's infrastructure.
• Collaborate with cross-functional teams to ensure data pipelines meet the demands of language models.

Required qualifications

• Strong software engineering skills.
• Proficiency in Python.
• Experience building data pipelines.
• Familiarity with curriculum learning, data mixing and data attribution.
• Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools.
• Experience working with large-scale datasets, including web data, code data, and multilingual corpora.
• Knowledge of data quality assessment techniques and experimentation with data mixtures.
• A passion for bridging research and engineering to solve complex data-related challenges in AI model training.

Nice to have

• paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)

Benefits

• Weekly lunch stipend of $75/£75 or equivalent in local currency.
• Full health and dental benefits, including a separate budget for mental health.
• RRSP matching, 401K, Pension Scheme.
• 100% Parental Leave top-up for up to 6 months, for either parent.
• Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, and a workspace improvement credit.
• Education & learning stipend for conferences, courses, and coaching.
• 6 weeks of paid vacation (30 working days!).
• Budget for traveling to other offices if you are remote, plus an annual company offsite.
• Daily lunch program, snacks, and regular community and social events for those in the office.
• Co-working benefit for those not near an office.
• $500 home office stipend.

About the company

Cohere is a security-first enterprise AI company that builds foundation AI models and end-to-end products. It is a global technology company headquartered in Toronto.