Senior Manager, Engineering - AI Validation

The role

You will build and lead the AI Validation function to test, evaluate, and govern AI models and agentic capabilities. This role involves creating scalable test strategies and evaluation frameworks for embedded, cloud, and LLM-based products.

What you will do

• Define and drive AI validation strategy across embedded, in-vehicle, cloud-connected, and cloud-native AI systems
• Lead, hire, mentor, and grow an AI Validation organization
• Own end-to-end validation strategy for AI/ML models, LLMs, RAG pipelines, and agentic AI workflows
• Architect and operationalize scalable evaluation frameworks and benchmarking platforms
• Design and maintain evaluation harnesses for RAG and agentic systems
• Evaluate, integrate, and optimize open-source and commercial AI validation technologies
• Establish AI governance, Responsible AI practices, model lineage, safety guardrails, and compliance processes
• Act as the quality gatekeeper for AI-enabled releases
• Collaborate across engineering teams to define validation strategies and prototype new approaches
• Drive continuous improvement by tracking industry advances in AI evaluation

Required qualifications

• Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field
• 10+ years of experience in software or systems engineering including embedded, cloud, networking, security, or automotive domains
• 3+ years leading high-performing engineering or QA organizations
• Hands-on experience developing, deploying, testing, or operating AI/ML systems
• Expertise in modern ML workflows, neural networks, and MLOps
• Deep understanding of LLMs, RAG architectures, vector databases, embeddings, retrieval optimization, and agentic AI frameworks
• Experience designing and implementing scalable evaluation frameworks for AI systems
• Expertise with hybrid evaluation methodologies combining deterministic validation and probabilistic LLM-as-a-Judge techniques
• Practical experience with RAG evaluation frameworks such as RAGAS
• Experience validating hallucination, grounding, citation accuracy, bias, fairness, toxicity, and factual consistency
• Experience designing systems that verify external knowledge claims
• Experience testing cloud-native platforms and cloud-managed embedded products

Nice to have

• MS degree
• Experience validating AI or agentic systems in safety-critical or regulated industries (automotive, aerospace, medical)
• Track record building and scaling an AI test/evaluation platform or developer experience
• Demonstrated wins moving AI testing practices from ad hoc to standardized, organization-wide adoption
• Experience implementing enterprise-grade AI governance in production systems
• Deep experience evaluating LLM and RAG systems at scale

Benefits

• Health care plan (Medical, Dental & Vision)
• Flexible and Dependent Care Expense program
• Retirement plan (401k)
• Life Insurance (Basic, Voluntary & AD&D)
• Unlimited paid time off per year
• 14+ paid holidays
• Complimentary lunches, snacks, and beverages during on-site working days
• Wellness benefit allowance
• Phone & Internet reimbursement
• Computer Accessory Allowance

About the company

Sonatus is driving the transformation to AI-enabled software-defined vehicles. Headquartered in Sunnyvale, CA, the company has 250+ employees worldwide and technology in production across more than 9 million vehicles.