(Senior/Junior AI Inference Reliability Engineer, Compute Infra

The role

You will own the Kubernetes-native control plane that provisions and runs the GPU inference fleet. You will triage InfiniBand issues, diagnose model serving faults, and extend automated self-healing systems.

What you will do

• Operate the provisioning state machine in production
• Triage and extend automated self-healing
• Own end-to-end reliability of the pipeline
• Partner with the inference/ML platform team
• Work alongside machine learning researchers, engineers, and product managers
• Introduce new techniques, tools, and architecture to improve the model inference stack
• Build tools to provide visibility into bottlenecks and instability
• Lead reliability cross-functionally

Required qualifications

• Experience operating a provisioning or host-lifecycle pipeline in production
• Deep experience owning production reliability for large-scale distributed systems
• Hands-on experience troubleshooting InfiniBand/RDMA networking issues
• Familiarity with the operational realities of multi-tenant model serving
• A demonstrated track record of eliminating operational toil through automation and tooling
• Hands-on experience with Kubernetes controller/CRD patterns, reconciliation loops, and node-level health/remediation systems
• Comfort with event-driven architectures, backpressure/circuit-breaking design, and observability
• A software-engineering mindset

Nice to have

• GPU/ML-serving infrastructure experience

Benefits

• startup equity
• health insurance
• other competitive benefits

Before you apply

• Candidates are limited to 6 applications within a 90-day period

About the company

Together AI, the AI Native Cloud, is purpose-built for AI engineers. It provides high-performance inference, fine-tuning, reinforcement learning, and pre-training.