Software Engineer - Cloud Infrastructure

The role

You will own the architecture and evolution of the cluster platform for a GPU-accelerated AI inference cloud. This includes designing cluster connectivity, extending Kubernetes, and managing the network path for inference traffic.

What you will do

• Own the architecture of multi-cluster, multi-tenant Kubernetes fleet.
• Extend Kubernetes with custom controllers, operators, and CRDs.
• Design GPU scheduling and capacity strategy.
• Build autoscaling for inference traffic.
• Own the Kubernetes network data plane including CNI, IPAM, DNS, ingress, and load balancing.
• Design cross-AZ, cross-region, and cross-cluster connectivity.
• Debug production network issues.
• Define SLOs for platform-critical systems and lead post-incident hardening.
• Deliver infrastructure as code with Terraform, Helm, and GitOps.
• Partner with inference engine, platform, SRE, and security teams.

Required qualifications

• 5+ years designing, building, and operating large-scale Kubernetes infrastructure in production.
• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent.
• Proven experience operating large-scale, high-traffic network services in production.
• Deep understanding of Kubernetes internals: API server, scheduler, controller loops, kubelet, and etcd.
• Strong command of Kubernetes and cloud networking: CNI, kube-proxy/eBPF datapaths, DNS, load balancing, service mesh, and VPC routing.
• Proficiency with AWS, Terraform, Helm, and Ansible.
• Programming skills in Go or Python.
• Strong debugging skills across distributed systems, containers, and the Linux networking stack.
• Clear written and verbal communication.

Nice to have

• Large-scale Kubernetes operations in a high-traffic domain such as gaming, e-commerce, or public cloud.
• Cilium and eBPF, including kube-proxy replacement or upstream contributions.
• Cluster provisioning and lifecycle management with Kubespray or similar Ansible-based tooling.
• GPU orchestration: NVIDIA GPU Operator, device plugins, or Dynamic Resource Allocation (DRA).
• High-performance networking for distributed workloads: RDMA/RoCE, InfiniBand, EFA, SR-IOV, or NCCL tuning.
• Multi-cloud, hybrid-cloud, or bare-metal Kubernetes operations.
• Contributions to Kubernetes, Cilium, Istio, or other CNCF projects.

Benefits

• Flexible working hours
• Daily lunch and dinner provided
• unlimited snacks and beverages
• Supportive and highly collaborative work environment
• Health check-up support
• top-tier equipment/hardware support
• A front-row seat to the generative AI infrastructure revolution
• Competitive compensation
• startup equity
• health insurance

About the company

FriendliAI is the fastest inference cloud for agents, built to run frontier open-weight models in production at scale. It is a small, fast-moving team building a platform for generative AI infrastructure.