Software Engineer, GPU Infrastructure

The role

You will deploy and operate GPU Kubernetes clusters across clouds and work with cloud providers and hardware partners. You will automate fault detection and remediation and improve fleet utilization for foundation models.

What you will do

• Deploy and operate GPU Kubernetes clusters across clouds
• Bring up new hardware generations and providers
• Work with cloud providers and hardware partners on reliability, performance, and cost
• Automate node, GPU, and fabric fault detection and remediation
• Improve fleet utilization and track cost
• Cut the time it takes to get a faulty node back into service
• Troubleshoot production failures and performance problems

Required qualifications

• Run GPU Kubernetes clusters with thousands of GPUs for ML or HPC workloads
• Debugged Linux, networking, and performance problems
• Written software in Python or Go
• Worked with researchers, ML engineers, or another team whose goals were not identical to yours
• Found a bottleneck, proposed a fix, and driven it in a fast-moving environment

Nice to have

• Familiarity with RDMA, NCCL, or accelerator health

Benefits

• Weekly lunch stipend of $75/£75 or equivalent
• Full health and dental benefits
• Separate budget for mental health
• RRSP matching, 401K, Pension Scheme
• 100% Parental Leave top-up for up to 6 months
• Annual enrichment benefits for arts & culture, fitness/wellness, quality time, and workspace improvement
• Education & learning stipend for conferences, courses, and coaching
• 6 weeks of paid vacation
• Budget for traveling to other offices if remote
• Annual company offsite
• Daily lunch program, snacks, and social events for those in office
• Co-working benefit for those not near an office

About the company

Cohere is a security-first enterprise AI company that builds foundation AI models and end-to-end products. It is a global technology company headquartered in Toronto.