Rhoda Ai
Software Engineer, Inference Infrastructure
Mountain View. Posted 6 months ago
The role
You will build and operate systems that power the model deployment stack. This involves running large foundation models across cloud and on-prem environments with a focus on resource management, scheduling, and scalability.
What you will do
- Design and operate large-scale infrastructure to run model workloads across cloud and on-prem environments
- Build and maintain Kubernetes-based deployment pipelines for managing distributed ML workloads
- Own resource scheduling and orchestration across GPU clusters
- Optimize utilization, workload balancing, and cost-performance tradeoffs
- Integrate and manage ML frameworks and model serving systems across research and production use cases
- Build tooling for model deployment, versioning, and observability
- Contribute to the reliability and scalability of the infrastructure stack
Required qualifications
- 3+ years of experience in ML infrastructure, MLOps, or distributed systems
- Strong proficiency with Kubernetes and containerized deployment pipelines
- Experience with GPU orchestration and resource scheduling across large distributed jobs
- Experience with cloud providers (e.g., AWS, GCP) and hybrid cloud/on-prem infrastructure
- Familiarity with ML frameworks (e.g., PyTorch, JAX) and model serving tools (e.g., Triton, Ray Serve, TorchServe)
- Strong debugging instincts and ownership mentality
Nice to have
- Experience with streaming systems or high-throughput data transport (e.g., Kafka, gRPC, NATS)
- Background in networking, low-latency systems, or network-aware scheduling
- Experience with edge/cloud hybrid deployment patterns and the latency constraints that come with them
- Familiarity with on-robot or embedded inference environments
- Experience with large-scale cluster topology and scheduling systems (e.g., SLURM, Ray, Volcano)
About the company
Rhoda AI is building the next generation of generalist intelligent robots. They own the full robotics stack from hardware to foundation world models and have raised over $450M.