(Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra

The role

You will own the Kubernetes-native control plane that provisions and runs the GPU inference fleet. You will triage InfiniBand issues, diagnose model serving faults, and extend automated self-healing systems to maintain fleet reliability and efficiency.

What you will do

• Operate the provisioning state machine in production.
• Triage and extend automated self-healing systems.
• Own end-to-end reliability of the pipeline including SLOs and error budgets.
• Partner with the inference/ML platform team to feed failure modes back into platform abstractions.
• Work with machine learning researchers, engineers, and product managers to bring technologies into production.
• Introduce new techniques, tools, and architecture to improve model inference stack performance.
• Build tools for visibility into bottlenecks and sources of instability.
• Lead reliability cross-functionally across infra, ML platform, and product teams.

Required qualifications

• Experience operating a provisioning or host-lifecycle pipeline in production at fleet scale.
• Deep experience owning production reliability for large-scale distributed systems.
• Hands-on experience troubleshooting InfiniBand/RDMA networking issues.
• Familiarity with the operational realities of multi-tenant model serving.
• A demonstrated track record of eliminating operational toil through automation and tooling.
• Hands-on experience with Kubernetes controller/CRD patterns, reconciliation loops, and node-level health/remediation systems.
• Comfort with event-driven architectures, backpressure/circuit-breaking design, and observability.
• A software-engineering mindset.

Nice to have

• GPU/ML-serving infrastructure experience

Benefits

• startup equity
• health insurance
• other competitive benefits

Before you apply

• Candidates are limited to 6 applications within a 90-day period.

About the company

Together AI is the AI Native Cloud purpose-built for AI engineers. It provides high-performance inference, fine-tuning, reinforcement learning, and pre-training.