Software Engineer – AI Inference Engine

The role

You will design, implement, and optimize GPU kernels and supporting infrastructure for generative and agentic AI workloads. This role focuses on optimizing the performance and efficiency of the core inference engine.

What you will do

• Design and optimize custom GPU kernels for AI workloads
• Contribute to the development of the kernel compiler, memory planner, runtime, and other core components
• Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performance
• Analyze performance bottlenecks across the software and hardware stack
• Drive support for new model architectures and tensor compute patterns
• Maintain production-grade performance infrastructure, including profiling, benchmarking, and validation tools

Required qualifications

• 5+ years of experience in production or high-impact research environments
• Production-level expertise in Python and C++
• Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
• Experience developing machine learning frameworks or performance-critical runtime systems
• Hands-on experience writing and optimizing GPU kernels
• Hands-on experience profiling GPU kernels
• Experience working with generative AI models such as transformer and diffusion models

Nice to have

• Experience developing machine learning compilers or code generation systems
• Familiarity with dynamic shape compilation, memory planning, and kernel fusion
• Contributions to inference engines, compilers, or high-performance numerical libraries
• Understanding of multi-GPU and distributed inference strategies

Benefits

• Flexible working hours
• Daily lunch and dinner provided
• Unlimited snacks and beverages
• Supportive and highly collaborative work environment
• Health check-up support
• Top-tier equipment/hardware support
• Competitive compensation
• Startup equity
• Health insurance

About the company

FriendliAI is building an AI inference platform that makes large language and multi-modal models fast, efficient, and deployable at scale. They power high-throughput, low-latency AI workloads for organizations worldwide.