Senior Data Scientist, Applied AI

The role

The Senior Data Scientist builds and evaluates AI solutions for assigned use cases. This person will develop LLM, RAG, and tool-using agent workflows using Python code.

What you will do

• Build, test, and improve LLM, RAG, and tool-using agent workflows.
• Select and apply evaluation criteria, baselines, and test datasets.
• Compare models, embeddings, and retrieval methods.
• Document recommendations and tradeoffs for assigned projects.
• Investigate retrieval misses, incorrect answers, and workflow failures.
• Build and update test cases, benchmark datasets, and regression checks.
• Monitor and investigate performance changes for owned workflows.
• Communicate findings and technical tradeoffs to engineering, product, and domain partners.

Required qualifications

• Bachelor's degree required
• 5+ years of relevant experience in data science, machine learning, applied AI, software engineering, or a related field
• Hands-on experience building and evaluating LLM applications
• Strong Python programming, software engineering and testing, and analytical problem-solving skills
• Strong knowledge of LLMs, embeddings, semantic search, retrieval systems, and RAG architectures
• Strong hands-on experience with Databricks, including Delta Lake, Unity Catalog, Lakebase, Lakeflow Jobs, and MLflow
• Ability to design representative evaluations for an assigned use case, analyze errors, and use results to improve a system
• Ability to weigh quality, reliability, latency, and cost when recommending an approach
• Clear communication and effective collaboration with technical and nontechnical partners
• Experience with agent frameworks and AI evaluation platforms

Nice to have

• master's degree in a quantitative or technical field
• Experience with healthcare data

Benefits

• medical, dental, and vision coverage
• flexible paid time off
• 401(k) plan with employer contribution

About the company

Definitive Healthcare is a global company headquartered just outside Boston, Massachusetts, that provides data on the Healthcare market. Since 2011, they have supported more than 2,100 customers around the world.