Applied AI Engineer

The role

You will research, prototype, and build agent-powered features from ambiguous problem statements. You will develop eval infrastructure and manage retrieval and context pipelines to improve agent quality.

What you will do

• Take open problems from research and prototyping through to production
• Design and ship agent-powered features like root-cause analysis and incident triage
• Build eval infrastructure including golden datasets and regression suites
• Own retrieval and context pipelines over customer metadata and query history
• Instrument agent behavior in production including traces and cost and latency budgets
• Partner with data science on detection quality and experiment design
• Partner with PM on agent functionality
• Set technical standards for building with LLMs

Required qualifications

• Built agents in production with real users
• Run evals and monitored agents after launch
• Python
• ML or data science background
• 3+ years in ML, data science, or software engineering
• Experience at companies that ship quickly
• Ability to work from an ambiguous problem statement
• Daily use of AI tools like Claude

Nice to have

• statistics and hypothesis testing, applied rather than theoretical
• Building and maintaining MCP servers
• Experience in the data and cloud space such as Snowflake, Databricks, dbt, Airflow

Benefits

• Competitive compensation
• equity
• remote-first environment

About the company

Monte Carlo is the agent trust platform that autonomously observes, fixes, and continuously improves agents and the data that powers them. More than 400 enterprises rely on Monte Carlo.