Forward Deployed Engineer, Inference

The role

You will act as a technical translation layer between core engineering teams and strategic partners. You will architect inference solutions and embed with customer engineering teams to integrate AI models into production environments.

What you will do

• Embed with customer engineering teams to architect connections between AI models and production environments
• Design advanced model serving topologies including multi-tenant architectures and cross-datacenter failover
• Optimize AI-in-a-box and private deployments
• Conduct benchmarking and profiling of customer LLM workloads
• Accelerate hardware validation for GPUs, TPUs, AMD, and custom ASICs
• Translate field friction into platform upgrades and product features
• Ensure Day-1 production reliability by assisting customers with deployment patterns, SLOs, and observability

Required qualifications

• 8+ years of engineering experience running production infrastructure, designing large-scale distributed systems, or deploying enterprise software
• Deep technical depth in production Kubernetes including multi-cluster and hybrid/multi-cloud ecosystems (GCP, Azure, AWS, OCI)
• Strong understanding of LLM architecture and inference dynamics
• Hands-on experience with modern inference engines and optimization stacks (e.g., vLLM, TensorRT-LLM, lmd, or Dynamo)
• Familiarity with distributed serving techniques
• Exceptional customer-facing technical leadership
• High agency, grit, and adaptability to write production code and manage client-side deployment blockers
• Strong programming foundations in Python, Go, or C++

Benefits

• Weekly lunch stipend of $75/£75 or equivalent
• Full health and dental benefits
• Separate budget for mental health
• RRSP matching, 401K, Pension Scheme
• 100% Parental Leave top-up for up to 6 months
• Annual enrichment benefits for arts, culture, fitness/wellness, quality time, and workspace improvement
• Education and learning stipend for conferences, courses, and coaching
• 6 weeks of paid vacation
• Budget for traveling to other offices if remote
• Annual company offsite
• Daily lunch program, snacks, and social events for office workers
• Co-working benefit for those not near an office

Before you apply

• May use AI-enabled tools to screen and assess applicants

About the company

Cohere is a security-first enterprise AI company that builds foundation AI models and end-to-end products. It is a global technology company headquartered in Toronto.