Senior Software Engineer, AI Gateway - Shanghai

The role

You will design, build, and maintain the Kong AI Gateway runtime and its underlying data plane stack. You will implement features like proxying, routing, load balancing, and security for AI workloads.

What you will do

• Design, build and maintain Kong's AI Gateway runtime and its underlying data plane stack
• Architecture, design, and implement new features including proxying, routing, load balancing, guardrails, compression, and identity
• Build and maintain security, performance, and compliance foundations
• Engage with the open source community and extend Kong core
• Fill the gaps between Kong AI Gateway and third-party AI/ML providers, deployment platforms, and tooling
• Perform any additional tasks required by the manager

Required qualifications

• 8+ years of programming in server-side applications and components
• Experience designing, prototyping, building and debugging highly scalable, distributed, reliable and resilient applications
• Solid hands-on low-level technical expertise in Linux systems troubleshooting, and profiling
• Experience in distributed systems, Linux networking stack and computer systems programming
• Knowledge of APIs on both L7 and L4 transports and protocols
• Comfortable working with a remotely distributed team
• Strong sense of ownership
• Excellent verbal and written communication skills
• Familiarity with AI LLM concepts and applications development

Nice to have

• Hands-on experience with NGINX/OpenResty/Kong/LuaJIT
• Hands-on experience with Envoy or Istio
• Hands-on experience with Rust
• Hands-on experience leveraging AI agents to deliver intricate software systems efficiently and with superior quality

About the company

Kong Inc. is the AI Connectivity Company building the connectivity layer of AI. Their unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic.