Principal Cloud Platform Engineer

The role

You will design and build software to deploy, schedule, and scale model workloads across shared accelerator hardware. You will also set uptime and latency targets and participate in an on-call rotation.

What you will do

• Shared ownership of the production inference service across regions
• Designing and building platform services, Kubernetes operators, and controllers
• Designing scheduling for shared accelerator capacity
• Rolling out upgrades without disrupting running workloads
• Designing platform security including tenant isolation and secrets management
• Standing up new regions with infrastructure as code and building CI/CD pipelines
• Building monitoring and alerting in Prometheus, Grafana, and Datadog
• Finding and eliminating performance bottlenecks and designing auto-scaling
• Leading incident response and postmortems in an on-call rotation
• Forecasting infrastructure needs and working with finance to manage cloud spend

Required qualifications

• B.S. in Computer Science, Computer Engineering, or related field
• 8+ years of experience in Site Reliability Engineering, platform engineering, or distributed systems roles
• Experience supporting a large-scale, customer-facing service in a public cloud environment (AWS, GCP, Azure)
• Strong programming skills in Python, Go, Rust, or Java, with recent hands-on coding
• Experience designing and building a platform service, control plane, or scheduler used by multiple teams or customers
• Kubernetes experience such as writing operators or controllers, running cluster upgrades, or configuring the scheduler
• Experience with multi-tenant systems, including resource quotas, isolation, and fair sharing of capacity
• Experience designing for failure, including retries, idempotency, and safe rollouts
• Deep understanding of monitoring and observability principles and tools (e.g., Prometheus, Grafana, ELK Stack, Datadog)
• Experience with Infrastructure as Code (e.g., Terraform, CloudFormation)
• Experience with CI/CD principles and tools (e.g., Jenkins, GitHub Actions, ArgoCD)
• Strong Linux/Unix system administration fundamentals

Nice to have

• Experience in a hybrid environment bridging cloud and on-premise/data center infrastructure
• Direct experience supporting ML/AI inference services in production
• Knowledge of model serving frameworks like vLLM, SGLang, or Ray, including batching and KV cache management
• Familiarity with GPU-accelerated computing and optimizing workloads for NVIDIA GPUs for purposes of mapping to RDUs
• Experience with batch or job scheduling systems such as Slurm, Volcano, or Kueue
• Experience with Kubernetes internals such as etcd, control plane high availability, or the scheduler
• Understanding of MLOps principles and practices
• Experience managing and tuning databases (SQL or NoSQL) and caching systems (Redis, Memcached)

Benefits

• Base salary plus equity
• 95% premium coverage for employee medical insurance
• 77% premium coverage for dependents
• Health Savings Account (HSA) with employer contribution
• Dental, Vision, Short/Long term Disability, Basic Life, Voluntary Life, and AD&D insurance plans
• Flexible Spending Account (FSA) options like Health Care, Limited Purpose, and Dependent Care
• Headspace subscription
• Gympass+ membership
• One Medical membership
• Counseling services with an Employee Assistance Program

Before you apply

• You must submit an application form for each position for which you believe you are qualified

About the company

SambaNova is a leader in next-generation AI infrastructure, delivering a full-stack inference platform for customers worldwide.