Model Serving Engineer

The role

You will work across the full inference stack to turn the NEXUS Large Tabular Model into a reliable and scalable production system. You will optimize Python inference code, define distributed inference architecture, and own the Triton serving layer.

What you will do

• Optimize Python inference code for performance under real concurrency constraints
• Work closely with research to understand model internals and support architecture evolution
• Collaborate with research and infrastructure teams to reason about hardware utilization and serving tradeoffs
• Define and evolve the architecture behind distributed inference and asynchronous execution stack
• Own the Triton serving layer for NEXUS
• Build observability and performance tooling across the serving stack
• Solve cross-cutting serving challenges across different environments
• Evaluate and integrate new inference runtimes, serving strategies, and infrastructure approaches

Required qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field (or equivalent practical experience)
• 5+ years of experience in model serving, ML infrastructure, or a closely related backend engineering role
• Deep expertise in Python concurrency, including GIL behavior, multi-threading, thread safety, multiprocessing
• Experience building asynchronous and message-driven systems
• High-performance, large-scale distributed systems
• Ability to read and reason about ML model implementations at a computational level
• Experience profiling and optimizing performance across CPU, memory, I/O, and GPU workloads

Nice to have

• Understanding of GPU architecture, performance characteristics, and resource utilization in high-performance compute workloads
• Experience working with tabular and structured-data ML systems
• Understanding of neural networks and modern deep learning architectures
• Experience with Kubernetes and cloud infrastructure
• Familiarity with DevOps and production infrastructure tooling, including containers, Helm, observability, and CI/CD systems

Benefits

• Competitive compensation with salary and equity
• Comprehensive health coverage for you and your dependents
• Paid parental leave for all new parents, inclusive of adoptive and surrogate journeys
• Relocation support for employees moving to join the team in one of our office locations

About the company

Fundamental is an AI research lab pioneering the future of enterprise decision-making. Their flagship model, NEXUS, is a Large Tabular Model purpose-built for structured records.