Xai

Director, Site Operations - Memphis

Memphis, TN. Posted a month ago

The role

You will own node and rack uptime for SpaceXSI's AI supercompute cluster across 5+ sites that run 24/7. You will lead a 250+ person operations organization and the site reliability engineering team.

What you will do

  • Own node, rack and cluster health across 5+ sites running 24/7, accountable for customer Service Level Agreements and uptime
  • Lead a 250+ person operations organization of site managers, shift supervisors and technicians across four 24/7 shifts
  • Drive node and rack remediation through command-line and physical intervention, reducing mean time to repair
  • Coordinate with facilities operations on power and cooling faults and proactive maintenance
  • Coordinate with network engineering on cluster upgrades and with tenant representatives on remediation and planned and unplanned downtime
  • Direct vendors through hardware rework and field operations so repairs and capacity work happen on time
  • Lead site reliability engineering: proactive health monitoring, fault mitigation at scale, root cause analyses and reliability procedures
  • Lead continual improvement and efficiency initiatives using operational data to raise uptime, repair time and SLA performance
  • Command incident response for cluster-impacting events, with clear direction, fast recovery and tight communication
  • Standardize best practices across sites and grow the organization as the cluster expands

Required qualifications

  • Bachelor's degree and 7+ years of experience in large scale operations with 5+ years leading people leaders of technical teams; OR 10+ years of experience in large scale operations with 5+ years leadi
  • Willingness to travel frequently to data center locations
  • Physical capability to lift up to 50 lbs unassisted, stand for long periods and use a ladder occasionally
  • Willingness to work extended hours and/or weekends as needed

Nice to have

  • Proven ability to lead large, multi-site, 24/7 operations organizations
  • Deep expertise in server hardware, cluster reliability and data center technologies
  • Experience supporting compute-heavy environments such as AI, machine learning or high-performance computing
  • A track record of owning uptime, Service Level Agreements or reliability metrics for large compute clusters
  • Experience leading site reliability engineering or reliability-focused teams
  • Strong analytical skills and the ability to explain technical concepts to diverse audiences
  • A history of partnering with vendors at scale and scaling operations across multiple sites
  • Familiarity with tooling and automation such as Jira, Python and Bash
  • Enthusiasm for SpaceXSI's mission
  • Ability to thrive in a dynamic environment with on-call ownership of cluster-impacting events

Before you apply

  • SpaceXSI is an equal opportunity employer

About the company

SpaceXSI is building Super Intelligence to help humanity understand the universe. Its teams develop Colossus supercomputers, Grok models and applications including Grok Bot, Grok Build, Grok Imagine, Grok Voice and X.

Director, Site Operations - Memphis at Xai | JobZoey