Andromeda

Customer Reliability Engineer

Remote / APAC. Posted a month ago

The role

The Customer Reliability Engineer triages and debugs customer issues at the Linux and Kubernetes layer. You will work with providers to resolve faults and build monitoring tools and scripts to automate checks.

What you will do

  • Triage and fix customer issues from start to finish
  • Debug at the Linux layer including processes, networking, storage, kernel logs, resource contention, systemd, and journald
  • Investigate Kubernetes problems like pods, node conditions, scheduling failures, and resource limits
  • Work GPU failures including driver and device-plugin issues, XID errors, and thermal throttling
  • Escalate issues with gathered evidence
  • Take part in a 24/7 on-call rotation
  • Respond to alerts and customer-reported outages
  • Keep customers updated during incidents
  • Write post-incident documentation, runbooks, alerts, or fixes
  • Work with compute providers to resolve faults and track recurring failures

Required qualifications

  • Real Linux troubleshooting ability from the command line
  • Working knowledge of Kubernetes
  • Can write a script in Python or Bash to automate something repetitive
  • Strong writing
  • Good judgment under pressure
  • Willing to join a 24/7 on-call rotation

Nice to have

  • Hands-on time with NVIDIA GPUs in production: drivers, CUDA, DCGM, the Kubernetes device plugin
  • Experience with high-performance networking (InfiniBand, RoCE) or NCCL
  • Experience with HPC or batch schedulers like Slurm
  • A previous customer-facing technical role: support engineering, TAM, solutions, professional services
  • Knowledge of Prometheus, Grafana, Datadog, or similar
  • IAC: Terraform, Ansible, or Helm
  • Genuine interest in AI infrastructure and how big training jobs behave

Benefits

  • Competitive compensation
  • Meaningful equity
  • Healthcare, dental, and vision coverage for you and your dependents
  • 401(k)
  • Unlimited PTO

About the company

Andromeda is a market and infrastructure platform to buy, sell, and operate compute. It enables fragmented compute to flow through one platform to deliver reliable capacity to model builders, research labs, and inference providers.