Senior Manager, Engineering - AI Inference

The role

You will lead an engineering team to make large language models run faster, cheaper, and more reliably in production. The role involves people management, technical direction, hands-on coding, and profiling the inference stack. You will also work with customer engineering teams to tailor deployments to their specific needs.

What you will do

• Bring current inference techniques into production and refine them.
• Design and optimize serving architectures.
• Work down into the serving stack from frameworks to CUDA kernels.
• Adapt and scale optimization methods across many kinds of ML models.
• Profile and tune deployments against latency, throughput, and cost targets.
• Tailor deployments to customer models and constraints.
• Build and support software and product features around the inference stack.
• Experiment quickly to find what works and ship well-tested results.
• Lead team delivery end to end from experiments to production.
• Partner with product and engineering leaders on technical strategy and roadmaps.

Required qualifications

• 2+ years of experience directly managing and leading an engineering team in a high-performance or ML-focused environment.
• Strong technical, hands-on experience in software engineering, low-level optimization, or ML infrastructure.
• A Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field.
• Hands-on experience shipping code in production with one or more general-purpose languages, such as Python or C++.
• Familiarity with methods for optimizing LLMs for high throughput / low latency inference.
• Comfort with modern LLM serving frameworks such as vLLM or SGLang.
• Profiling and analyzing performance down to the kernel level.
• A firm grasp of how GPUs are built and how they behave.
• Clear interest and hands-on experience with large language models.
• A working knowledge of AI/ML pipelines and the full path of developing and deploying ML models.
• Strong communication skills.

Nice to have

• A track record of making software systems run faster, especially for large language models.
• Experience with CUDA or comparable technologies.
• A strong command of software engineering fundamentals, with a record of building and shipping AI/ML inference systems.
• Experience with Docker and Kubernetes.
• Prior work building or tuning AI/ML projects, particularly in a customer-facing setting.

Benefits

• Competitive compensation and equity packages
• Restricted Stock Units
• Paid time off, paid holidays & leave of absence programs
• Comprehensive health, dental & vision insurance
• Employer contributions to HSA account
• Paid parental leave
• Paid life insurance, short-term and long-term disability
• Professional development & tuition reimbursement
• Mental health & wellness support
• Commuter benefits (parking & transit)
• Cell phone stipend
• 401(k) Retirement plan with company match up to 4% of salary

About the company

Crusoe is a vertically integrated AI infrastructure company that owns and operates each layer of the stack to power AI workloads. They use an energy-first approach to solve the demand for AI compute and power bottlenecks.