Senior Inference Reliability Engineer

The role

The Senior/Staff Inference Reliability Engineer will own the end-to-end reliability and production performance of customer inference workloads. You will ensure customer endpoints meet expectations for availability, latency, throughput, quality, and cost by investigating problems across the entire serving path.

What you will do

• Own the production health of customer inference workloads.
• Establish service-level indicators, objectives, performance baselines, and escalation paths.
• Build telemetry, dashboards, alerts, and automated diagnostics.
• Lead investigations of complex latency, throughput, capacity, and reliability regressions.
• Lead customer-impacting incidents and establish operational practices.
• Convert incidents into automated tests, safeguards, runbooks, and platform improvements.
• Partner with the LLM Performance team to validate engine-level optimizations.
• Analyze workload behavior, capacity requirements, and cost efficiency.
• Identify recurring patterns to improve the inference platform and product roadmap.

Required qualifications

• 5+ years of experience in production engineering, site reliability engineering, infrastructure engineering, distributed systems, ML infrastructure, database reliability, or performance engineering.
• Demonstrated ownership of a critical production service or workload.
• Experience diagnosing complex latency, throughput, capacity, or reliability problems across multiple system layers.
• Strong software-engineering ability beyond infrastructure configuration and CI/CD automation.
• Hands-on experience building observability, automation, diagnostic tooling, or production safeguards.
• Strong communication and technical leadership skills.
• Experience with Kubernetes, Linux, networking, and cloud-native infrastructure.

Nice to have

• Prior LLM-inference experience is valuable
• Experience operating across multiple cloud providers, regions, hardware configurations, or infrastructure suppliers is especially valuable
• Experience with GPUs, ML infrastructure, model serving, vLLM, SGLang, Triton, TensorRT-LLM, or similar technologies is valuable

About the company

Parasail is redefining AI infrastructure by enabling seamless deployment across a distributed network of GPUs. Their mission is to empower AI developers with a fast, cost-efficient, and scalable cloud experience.