Research Engineer - Model Architectures

The role

You will design and test novel model architectures and training methodologies. You will work with the pre-training team to improve core modeling capabilities and address fundamental bottlenecks.

What you will do

• Design and rigorously test novel model architectures and training methodologies
• Improve core modeling capabilities
• Address fundamental bottlenecks in contemporary models
• Work closely with the pre-training team to integrate insights into next-generation models

Required qualifications

• Strong research taste and intuition
• Ability to work through a research project from conception to execution to write-up
• Strong implementation and prototyping ability
• Ability to work well and cooperate with others in a high-paced research setting
• Curiosity, interest, and joy in understanding intelligence
• Understanding of modern training pipelines and the hardware requirements to design efficient architectures for GPU hardware
• Strong grasp of proper experimental methodology for running rigorous ablations and other hypothesis testing
• High proficiency with PyTorch and Python
• Strong ability to jump into large pre-existing codebases and rapidly get up to speed and become productive

Nice to have

• Previous experience with long-term memory, RAG/retrieval systems, dynamic/adaptive computation, and alternative approaches to credit assignment
• Experience with reinforcement learning, control theory, and signal processing
• A joy in inventing and seriously assessing ‘crazy’ ideas, and the ability to have a unique perspective on things
• Previously published machine learning research in well-respected venues
• Postgraduate degree in a scientific subject (Computer Science, EE/EECS, Math, Physics)

Benefits

• Comprehensive medical, dental, vision, and FSA plans
• Competitive compensation and 401(k) plan
• Relocation and immigration support on a case-by-case basis
• In-office snacks and meals provided
• Unlimited PTO and company holidays

About the company

Zyphra is an artificial intelligence company based in San Francisco, California.