Senior Software Engineer, AI Gateway - Shanghai

The role

You will engineer the core system powering live AI traffic. You will take ownership of the runtime and data plane responsible for routing, translating, securing, and analyzing requests.

What you will do

• Design, build and maintain Kong’s AI Gateway runtime and its underlying data plane stack.
• Architecture, design, and implement new innovative features including proxying, routing, load balancing, guardrails, compression, and identity.
• Build and maintain security, performance, and compliance foundations.
• Engage with the open source community and extend Kong core.
• Fill the gaps between Kong AI Gateway and third-party AI/ML providers, deployment platforms, and tooling.
• Perform any additional tasks required by the manager.

Required qualifications

• 8+ years of programming in server-side applications and components.
• Experience designing, prototyping, building and debugging applications that are highly scalable, distributed, reliable and resilient.
• Solid hands-on low-level technical expertise in Linux systems troubleshooting, and profiling.
• Experience in distributed systems, Linux networking stack and computer systems programming.
• Comfortable working with a remotely distributed team.
• Strong sense of ownership and excellent verbal and written communication skills.
• Familiarity with AI LLM concepts and applications development.

Nice to have

• Hands-on experience with NGINX/OpenResty/Kong/LuaJIT.
• Hands-on experience with Envoy or Istio.
• Hands-on experience with Rust.
• Hands-on experience leveraging AI agents to deliver intricate software systems efficiently and with superior quality.

About the company

Kong Inc. is the AI Connectivity Company building the connectivity layer of AI. Their unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic.