Senior AI Compute Infrastructure Engineer

The role

You will own and operate GPU and accelerator clusters for training, inference, evaluation, and experimentation. You will design infrastructure to run models locally and build scheduling, orchestration, and observability systems for AI workloads.

What you will do

• Own and operate GPU and accelerator clusters including drivers, runtimes, kernels, and scheduling.
• Design infrastructure to run models locally on GPUs to reduce dependency on external providers.
• Build and improve scheduling, orchestration, placement, quota management, and utilization systems.
• Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost.
• Partner with ML engineers and researchers to remove bottlenecks in training and deployment workflows.
• Build observability for GPU utilization, memory pressure, queue depth, and capacity pressure.
• Drive reliability, incident response, alerting, runbooks, and post-incident improvements.
• Evaluate and integrate new hardware, cloud instance families, and serving frameworks.
• Build tooling to make GPU usage visible and easy for internal teams to consume.
• Contribute to long-term architecture decisions regarding performance, cost, and scalability.

Required qualifications

• 5+ years of infrastructure engineering experience in GPU compute, ML infrastructure, distributed systems, high-performance computing, or large-scale production platforms.
• Hands-on experience operating GPU clusters or accelerator-backed infrastructure in production or production-like environments.
• Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging.
• Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve, or equivalent systems.
• Proficiency in Python for infrastructure automation, tooling, debugging, integration, and operational workflows.
• Practical understanding of performance tradeoffs across batching, concurrency, memory usage, GPU utilization, model size, latency, throughput, availability, and cost.
• Track record of optimizing compute costs while maintaining performance, reliability, and availability.
• Experience building observable systems with metrics, logs, traces, dashboards, alerts, and incident workflows.
• Ability to work in high-stakes, always-on environments.
• Clear communicator who can translate infrastructure tradeoffs for various stakeholders.

Nice to have

• Experience at a frontier AI lab, hyperscaler, high-frequency trading firm, research platform, or high-scale ML organization.
• Familiarity with custom silicon or specialized accelerators such as TPUs, AWS Trainium, Gaudi, or similar platforms.
• Background in capacity planning, procurement input, reserved capacity strategy, cloud accelerator economics, or GPU fleet cost management.
• Experience with distributed training frameworks such as DeepSpeed, Megatron-LM, FSDP, Ray, or equivalent systems.
• Experience debugging CUDA, NCCL, kernel, driver, runtime, memory, networking, or low-level performance issues.
• Experience with Rust, C++, Go, CUDA, or other systems languages used for performance-critical infrastructure.
• Crypto, financial services, trading infrastructure, or security-sensitive production infrastructure experience.

Before you apply

• May ask candidates to complete job-related skills or work-style assessments.
• Applicants are permitted to redact or remove information on their resume that identifies age, date of birth, or dates of attendance or graduation.

About the company

Payward is the parent company behind Kraken, NinjaTrader, Breakout, xStocks, Payward Services and CF Benchmarks. Kraken is one of the world's longest-standing crypto platforms, trusted by over 10 million individuals and institutions.