Cerebras

Cluster Operations Software Engineer

Sunnyvale, CA, Remote. Posted 3 months ago

The role

You will manage and operate machine learning compute clusters using the Wafer-Scale Engine. You will build software solutions for monitoring, automation, and reliability to ensure infrastructure health and maximize compute capacity.

What you will do

  • Deploy, configure, and debug container-based services using Docker.
  • Build and own software solutions for monitoring, automation, dashboards, and reliability.
  • Collaborate with cross-functional teams to translate requirements into O&M products.
  • Develop APIs, automation services, and integrations for fleet management.
  • Manage and operate multiple advanced AI compute infrastructure clusters.
  • Monitor and oversee cluster health to resolve potential issues.
  • Maximize compute capacity through optimization and resource allocation.
  • Provide 24/7 monitoring and support using automated tools and hands-on troubleshooting.
  • Handle engineering escalations and collaborate to resolve technical challenges.
  • Stay up-to-date with advancements in AI compute infrastructure.

Required qualifications

  • 6-8 years of relevant experience in managing and operating complex compute infrastructure.
  • Proficient in Python and Go.
  • Experience building operational platforms, workflow automation systems, and reliability tooling.
  • Experience and Expertise in distributed systems is a must.
  • Deep understanding of Linux-based compute systems and command-line tools.
  • Extensive knowledge of Docker containers and container orchestration platforms like k8s.
  • Proven ability to troubleshoot and resolve complex technical issues.
  • Experience with monitoring and alerting systems.
  • Proven track record to own and drive challenges to completion.
  • Excellent communication and collaboration skills.
  • Ability to work effectively in a fast-paced environment.
  • Willingness to participate in a 24/7 on-call rotation.

Nice to have

  • Operating and Managing large scale AI clusters.
  • Knowledge of technologies like Ethernet, RoCE, TCP/IP, etc.
  • Knowledge of cloud computing platforms (e.g., AWS, GCP, Azure).

About the company

Cerebras Systems builds the world's largest AI chip, the Wafer-Scale Engine. The company works with leading model labs, global enterprises, and AI-native startups.