Software Engineer - Model Performance Systems

The role

You will build an automated benchmarking and diagnostic suite for next-generation AI infrastructure. This role involves defining roadmaps, driving technical decisions, and developing tools for model evaluation and system profiling.

What you will do

• Evaluate, run and automate standard LLM quality benchmarks and custom performance suites
• Develop and maintain internal GPU-enabled development environments
• Build and contribute to open-source tools to automate model evaluation and analysis
• Use profilers to collect performance profiles, identify bottlenecks, and debug the compute/networking stack
• Develop real-time dashboards and alerts to monitor system health and performance
• Automate performance testing via CI/CD pipelines
• Build tools to identify the best configuration for latency, cost, and quality

Required qualifications

• Technical depth
• Strong communication skills
• Ability to navigate vague requirements
• Ability to mentor other engineers
• Understanding of GPU memory subsystems, InfiniBand, and data movement across a cluster
• Automation mindset including passion for stress-testing and fuzzy testing
• Desire to understand the underlying math of Transformers and how it translates into FLOPs and memory requirements
• Familiarity with Python
• Eagerness to master the NVIDIA software stack

Nice to have

• C++ familiarity

Benefits

• Competitive compensation, including meaningful equity
• 100% coverage of medical, dental, and vision insurance for employee and dependents (U.S. only)
• Flexible PTO policy including company wide Winter Break
• Paid parental leave
• Fertility and family-building stipend through Carrot
• Company-facilitated 401(k) (U.S. only)
• Exposure to a variety of ML startups

About the company

Baseten powers mission-critical inference for AI companies by uniting applied AI research, flexible infrastructure, and seamless developer tooling. The company recently raised a $1.5B Series F.