Senior AI Product Engineer, Backend

The role

You will write maintainable, scalable, and performant backend code primarily in Go, Java, and Python. You will build high-volume analytics systems and design APIs for Machine Learning and LLM workflows.

What you will do

• Write maintainable, scalable, and performant backend code primarily in Go, Java, and Python, with opportunities to work in TypeScript.
• Build high-volume and highly available analytics systems.
• Design and build APIs specific to our customers’ Machine Learning and LLM workflows.
• Prototype, optimize, and maintain scalable backend services that power the Arize core platform.
• Extend, and contribute back to, open source OLAP databases and distributed message queue frameworks.
• Develop and integrate collection tools for robust monitoring of ML and LLM pipelines.
• Research and implement cutting-edge visualization & dimensionality reduction algorithms in a distributed environment.
• Collaborate with product, design, and customer engineering teams to enhance and expand product offerings.
• Contribute to the build our own in-house AI Agents

Required qualifications

• 5+ years of experience working with high-performance backend systems.
• Strong experience writing Go, Python, TypeScript/Node, Java, or similar server programming languages.
• Enthusiasm and interest in the AI and LLM ecosystem, with a desire to learn and stay updated on emerging technologies.
• Previous work building and operating highly complex SaaS platforms/systems.
• Knowledge of working with public clouds & container orchestration - AWS, GCP, Azure, Kubernetes, etc.

Nice to have

• Experience with distributed stream processing - Kafka, Gazette, or similar.
• Experience with OLAP systems.
• Familiarity with system observability tooling like Prometheus.
• Working knowledge of Machine Learning and/or Data Science.
• First-hand experience working with large language models (LLMs) or developing AI products.

Benefits

• Medical
• Dental
• Vision
• 401(k) plan
• Unlimited paid time off
• Generous parental leave plan
• Additional support for mental health and wellness
• WFH monthly stipend for employees not in NYC or SF

About the company

Arize AI is the leading AI & Agent Engineering observability and evaluation platform. The company serves over 150 leading enterprises and Fortune 500 companies.