Fundamental
Senior Applied Research Engineer
Barcelona. Posted 7 months ago
The role
You will profile distributed training runs to identify bottlenecks in compute, GPU memory, and inter-GPU communication. You will design model scaling and memory optimization techniques and drive model productionization and serving.
What you will do
- Profile end-to-end distributed training runs to identify bottlenecks across compute, GPU memory, and inter-GPU communication
- Contribute to architectural decisions to improve efficiency and reliability of large-scale training jobs
- Develop Triton/CUDA kernels when needed
- Design and implement model scaling, parallelization, and memory optimization techniques
- Collaborate with ML Researchers to diagnose architectural inefficiencies
- Ensure new research ideas scale efficiently in practice
- Spread internal knowledge about model efficiency and optimization
- Drive the productionization and serving of models from the research side
- Improve inference efficiency through techniques such as quantization
Required qualifications
- Strong understanding of modern ML architectures and large-scale training pipelines
- Experience running distributed training jobs on multi-GPU systems
- Advanced profiling and debugging skills across CPU, GPU, memory usage, latency, and inter-GPU communication
- Strong programming skills in Python
- Experience with model scaling and parallelization strategies, including tensor and pipeline parallelism
Nice to have
- Familiarity with NCCL, MPI, and distributed communication primitives
- Knowledge of PyTorch and Triton internals
- Programming experience with C++ and CUDA
Benefits
- Competitive compensation with salary and equity
- Comprehensive health coverage for you and your dependents
- Paid parental leave for all new parents, inclusive of adoptive and surrogate journeys
- Relocation support for employees moving to join the team in one of our office locations
- A mission-driven, low-ego culture that values diversity of thought, ownership, and bias toward action
About the company
Fundamental is an AI research lab pioneering the future of enterprise decision-making. Their flagship model, NEXUS is a Large Tabular Model purpose-built for structured records.