Conversational Modelling Research Engineer

The role

This person will conduct research on Large Multimodal Models for Conversational Avatars. They will develop methods to model verbal and non-verbal conversation aspects and partner with the Applied ML team to move research to production.

What you will do

• Conduct research on Large Multimodal Models in the context of Conversational Avatars
• Develop methods to model verbal and non-verbal aspects of conversation
• Adapt and control avatar behavior in real time with low-latency
• Experiment with fine-tuning, adaptation, and conditioning techniques for AudioVisual Multimodal Models
• Partner with the Applied ML team to take research from prototype to production
• Stay up to date with cutting-edge advancements

Required qualifications

• PhD (or near completion) in a relevant field, or equivalent research experience
• Hands-on experience with Large Multimodal Models
• Strong foundation in generative (language) models
• Experience in fine-tuning/adapting VLMs for control, conditioning, or downstream tasks
• Solid background in deep learning and foundation modes
• Strong PyTorch skills
• Comfort building deep learning pipelines

Nice to have

• Knowledge of large-scale model training and optimization
• Experience in duplex-conversational model
• Broader understanding of generative AI across modalities
• Exposure to software development best practices
• A flexible, experimental mindset
• Publications at EMNLP, COLING, NeurIPS, ICLR, CVPR, ICCV

About the company

Tavus is a research lab pioneering human computing by building AI Humans. It is a Series B company backed by Sequoia Capital, Y Combinator, and Scale Venture Partners.