Software Engineer, AI Gateway

The role

You will join the engineering team building the Kong AI gateway. You will work on core features such as multi-LLM routing and balancing, semantic caching and routing, and MCP.

What you will do

• Build and maintain API management software for AI and LLMs and its underlying stack.
• Implement new features including proxying capabilities, load balancing, and databases support.
• Troubleshoot bugs, perform performance tuning and low-level optimizations.
• Fill the gaps between Kong and third-party deployment platforms and tooling.
• Perform any additional tasks required by manager.

Required qualifications

• 3+ years of programming in server-side applications and components.
• Hands on experience with either Golang or Rust
• Experience building and debugging applications that are highly scalable, distributed, reliable and resilient.
• Solid hands-on low-level technical expertise in Linux systems troubleshooting, and profiling.
• Experience in distributed systems, Linux networking stack and computer systems programming.
• Knowledge of APIs on both L7 and L4 transports and protocols.
• Comfortable working with a remotely distributed team.
• Strong sense of ownership.
• Excellent verbal and written communication skills.
• Familiarity with AI LLM concepts and applications development

Nice to have

• Hands on experience with NGINX/OpenResty/Kong/LuaJIT.
• Hands on experience with Envoy or Istio.

About the company

Kong Inc., the AI Connectivity Company, is building the connectivity layer of AI. Their unified API and AI platform enables organizations to secure, manage, accelerate, govern, and monetize the flow of intelligence across APIs and AI traffic.