Machine Learning Systems Administrator / DevOps Engineer: Training Clusters & Inference Cloud

The role

You will maintain and build core infrastructure for model training and inference cloud. You will work on large-scale training clusters and deployments on heterogeneous compute.

What you will do

• Administration and automation of Linux-based cluster environments
• User onboarding/offboarding, access control, and security auditing
• Monitoring system resources, job scheduling, and cluster health
• Bring-up and validation of new hardware
• Supporting and improving developer workflows
• Enabling and supporting large-scale training jobs
• Deploying, operating, and scaling inference services across heterogeneous accelerators
• Managing container orchestration, CI/CD, and rollout pipelines
• Building observability for production services
• Owning service reliability, uptime, incident response, and postmortems

Required qualifications

• Experience with Linux system administration, user and access management, and automation
• Demonstrated expertise in scripting languages for system tooling and automation (bash, Python, etc.)
• Experience with containers (Docker) and scheduling/orchestration systems (Slurm, Kubernetes)
• Experience running production services, including monitoring and alerting (e.g., Prometheus, Grafana), deployment pipelines, and incident response
• Experience building tooling for cluster validation and reliability (accelerator, networking, and storage health checks)
• Experience setting up and managing developer tools and third-party services (e.g., cloud storage providers, Docker Hub, Slack, Gmail, Telegraf, experiment trackers)
• Excellent debugging and troubleshooting skills across compute, storage, and networking
• Strong communication skills and ability to collaborate across technical and non-technical teams

Nice to have

• Experience with infrastructure as code (e.g., Ansible, Terraform)
• Prior work supporting ML/AI infrastructure, including GPU management and workload optimization
• Experience deploying and operating LLM serving stacks (e.g., vLLM, SGLang, Ray)
• Experience with multiple accelerator platforms (e.g., NVIDIA CUDA, AMD ROCm, TPUs)
• Experience with high-performance networking (InfiniBand, RoCE) and distributed storage
• Experience working with cloud platforms such as AWS, Azure, or GCP
• Familiarity with Apptainer and with integrating containers into Slurm and Kubernetes

Benefits

• Comprehensive medical, dental, vision, and FSA plans
• Competitive compensation and 401(k) plan
• Relocation and immigration support on a case-by-case basis
• In-office snacks and meals provided
• Unlimited PTO and company holidays

About the company

Zyphra research methodology is grounded in methodical, step-by-step approaches to ambitious goals. Both deep research and engineering excellence are equally valued.