Software Engineer – GPU Kernel

The role

You will design, build, and optimize low-level compute kernels for a GPU-accelerated AI inference platform. You will write GPU code and implement advanced optimizations for NVIDIA and AMD GPUs.

What you will do

• Design, implement, and optimize high-performance GPU kernels for AI inference
• Develop and maintain GPU code in CUDA and C++, including low-level assembly
• Implement reduced-precision and quantized kernels (FP8/FP4)
• Benchmark and ensure cross-vendor performance parity between NVIDIA and AMD hardware
• Contribute to internal GPU libraries and tune performance of performance-critical components
• Accelerate multi-modal model pipelines
• Investigate and integrate next-generation GPU features

Required qualifications

• 3+ years of experience in GPU programming, HPC, or performance-critical systems
• Bachelor’s or Master’s degrees in Computer Science, Computer Engineering, Electrical Engineering, or a related field
• Strong proficiency in CUDA for NVIDIA GPUs or ROCm/HIP for AMD GPUs
• Deep understanding of GPU architecture: warps, threads, memory hierarchy, synchronization, and latency-throughput trade-offs
• Proficiency in C++
• Experience with GPU profiling and performance tuning
• Strong numerical background with understanding of precision trade-offs and quantization techniques

Nice to have

• Experience optimizing transformer, multi-modal, or Mixture-of-Experts (MoE) architectures at the kernel level
• Familiarity with the latest GPU libraries and frameworks (CUTLASS, Triton, …)
• Inter-GPU communication programming experience
• Open-source contributions related to GPU performance or ML acceleration
• Research or conference presentations on GPU optimization, HPC, or numerical computing

Benefits

• Flexible working hours
• Daily lunch and dinner provided
• unlimited snacks and beverages
• Supportive and highly collaborative work environment
• Health check-up support
• top-tier equipment/hardware support
• Competitive compensation
• startup equity
• health insurance

About the company

FriendliAI is building an AI inference platform that makes large language and multi-modal models fast, efficient, and deployable at scale. They power high-throughput, low-latency AI workloads for organizations worldwide.