Perplexity
Member of Technical Staff (AI Inference Engineer)
San Francisco. Posted 7 months ago
The role
This person will support transformer-based, text-generation, and multimodal models in the inference infrastructure. They will work on GPU kernel migration, developing a Rust-native serving runtime, and performance optimization.
What you will do
- Support transformer-based retrieval, text-generation, and multimodal models in inference infrastructure
- Port in-house CUDA kernels to NVIDIA's CuTe DSL
- Develop internal Rust-based inference server
- Profile and fix bottlenecks from network ingress through continuous batching and GPU kernel interleaving
- Build dashboards, alerts, and automated remediation
- Respond to and learn from production incidents
Required qualifications
- 3+ years of professional software engineering experience with meaningful work on ML inference or high-performance systems
- Deep experience with GPU programming and performance work (CUDA, Triton, CUTLASS, or similar)
- Understand modern LLM architectures and are able to bring them up reliably in a production environment
- Built and operated production distributed systems under real load
- Comfortable working across languages and layers: Rust, Python, and CUDA/CuteDSL
- Familiarity with at least one deep learning framework (PyTorch, JAX, TensorFlow)
- Understanding of GPU architectures (memory hierarchy, warp scheduling, tensor cores)
- Understanding of common LLM architectures and inference optimization techniques (e.g. quantization, speculative decoding, prefill-decode disaggregation)
Nice to have
- Deep systems programming experience
- ML compilers and framework internals: PyTorch internals, torch.compile, custom operators
- Distributed GPU communication: NCCL, NVLink, InfiniBand, RDMA libraries, model/tensor parallelism
- Low-precision inference: INT8/FP8/FP4 quantization, mixed-precision serving
- Profiling and debugging tools: Nsight Compute/Systems, CUDA-GDB, PTX/SASS analysis
- Container orchestration: Kubernetes, GPU scheduling, autoscaling inference workloads
About the company
Perplexity builds and runs the inference engine behind every Perplexity query and deploys dozens of model architectures at scale.