Applied AI Engineer

The role

This person will build and run production LLM-based agents and assistants. They will work on agent runtime, orchestration, evaluation, and grounding using RAG on GCP.

What you will do

• Build and run LLM-based agents and assistants.
• Build agent runtime and orchestration including failure handling and sandboxing.
• Build evaluation sets, test harnesses, and AI-graded evals.
• Work across multiple model providers and manage version upgrades.
• Trace agent actions, monitor behavior, and attribute AI spend.
• Connect agents to company data through retrieval, context design, and prompt engineering.
• Work with operations teams to design workflows.
• Protect patient data and defend against prompt injection.
• Build on GCP and partner with Data Platform teams.
• Contribute to code and design review and help newer engineers.

Required qualifications

• Bachelor's degree in Computer Science, Engineering, or a related technical field
• 5 to 15+ years of software engineering experience
• Hands-on experience building and running LLM-based systems in production
• Real hands-on depth in at least one core area: agent runtime, evaluation, retrieval, or observability and cost
• Experience measuring whether an AI system works with evaluation data
• Strong production engineering across testing, debugging, services, APIs, deployment and on-call
• Experience handling sensitive data with a security-first mindset
• Comfort working in a greenfield space
• Clear communication with technical and non-technical partners

Nice to have

• Healthcare (PHI, HIPAA), finance or insurance experience
• GCP Experience
• Experience migrating a system between model versions or providers
• Machine learning experience beyond LLMs
• Built a platform used by other engineering teams
• For Senior Staff, experience leading a small team's technical direction

Benefits

• Medical, Dental, and Vision plans
• Flexible Spending/Health Savings Accounts
• Flexible PTO
• 401(k) + Company Match
• Life Insurance
• Pet insurance

About the company

OpenLoop provides telehealth support solutions designed to streamline care delivery for companies offering virtual support across all 50 states. The company has a relatively flat organizational structure with core values of Autonomy, Competence and Belonging.