Staff / Principal Software Engineer, Inference Platform - USA

The role

You will design the core architecture and write code for the Inworld Inference platform. You will build core services, manage GPU capacity allocation, and own the system in production.

What you will do

• Design and build the platform's core services including control plane, model and deployment lifecycle APIs, tenant isolation, metering and quotas
• Own how GPU capacity is allocated and scheduled across shared and reserved workloads
• Build the path that takes a customer's model from weights to production serving
• Write production code, mostly in Go, every week
• Define the APIs and abstractions that customers and internal teams build on
• Lead design reviews and write the technical direction for the platform
• Run what you build by setting SLOs, taking part in incidents, and turning postmortems into design changes

Required qualifications

• 8+ years building backend or distributed systems in Go, Rust, Java or C++
• Designed and shipped a multi-service system end to end
• Deep understanding of distributed systems: consistency, failure modes, queuing, load balancing and capacity modelling
• Experience building a platform or developer product with external customers: cloud compute, PaaS, serverless, or an API platform
• Clear technical writing

Nice to have

• Schedulers or resource managers for scarce hardware (GPU clusters, batch systems, cloud compute)
• LLM inference serving (vLLM, SGLang, TensorRT-LLM) or ML training infrastructure
• Multi-tenant systems with usage-based billing or contractual SLAs
• Comfortable with Kubernetes, Terraform and GitOps (ArgoCD)

Benefits

• equity
• benefits

Before you apply

• Candidates must be based in the SF Bay Area or willing to relocate

About the company

Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. They build first-party speech models, serve LLMs, and run the inference behind modular APIs.