Inference Engineer

*HQ - San Francisco, CA. Posted 23 months ago

The role

The Inference Engineer will design and build low latency, scalable, and reliable model inference and serving stacks for foundation models. This role involves working with research and product teams to serve products and build robust inference infrastructure.

What you will do

Design and build low latency, scalable, and reliable model inference and serving stack for foundation models using Transformers, SSMs and hybrid models.
Work closely with research team and product engineers to serve products in a fast, cost-effective, and reliable manner.
Design and build robust inference infrastructure and monitoring for products.
Shape products and impact how AI is applied across various devices and applications.

Required qualifications

Strong engineering skills
Comfortable navigating complex codebases
Eye for writing clean and maintainable code
Experience building large-scale distributed systems with high demands on performance, reliability, and observability
Technical leadership with the ability to execute and deliver zero-to-one results amidst ambiguity
Background in or experience working on inference pipelines with machine learning and generative models
Experience implementing state of the art Machine Learning models and research to applied problems

Nice to have

experience with vLLM, SGLang, Continuous Batching or other inference frameworks
experience working in CUDA, Triton or similar

Benefits

Competitive base salary
attractive equity package
Fully covered medical insurance along with dental and vision for you and your family
9 weeks paternity leave
12 weeks maternity leave
401(k)
Monthly commuter stipend
Flexible PTO
Lunch, dinner and plenty of snacks, provided daily
Your own personal Yoshi

Before you apply

Visa sponsorship is provided and assessed on a case-by-case basis
Cartesia participates in E-Verify and will provide Form I-9 information to confirm employment eligibility after hire

About the company

Cartesia aims to architect AI that learns from and interacts with the world like humans do. The founding team met as PhDs at the Stanford AI Lab and invented State Space Models.