Pluralis Research
Research Scientist - Pre-training
San Francisco, Remote. Posted a month ago
The role
You will build a training system to scale Protocol Learning from 8B to frontier scale. This involves training large models on heterogeneous hardware across different regions using the internet.
What you will do
- Implement and optimize model-parallel training including data, pipeline, and tensor parallelism.
- Implement techniques to reduce communication overhead while maintaining model convergence.
- Make runs survive node churn through robust checkpointing, state synchronization, and recovery.
- Build monitoring to show throughput, bottlenecks, and model quality across hundreds of devices.
Required qualifications
- Hands-on distributed training in PyTorch with FSDP, DeepSpeed, Megatron, or own implementation.
- Understanding of data, tensor, and pipeline parallelism.
- Production-quality Python.
- Knowledge of concurrency, failure handling, and profiling.
- Shipped systems, research code, open-source work, or serious personal projects.
- Belief that Protocol Learning is the viable third path for collective, trustless, and sovereign AI.
- Professional-level English proficiency (written and spoken).
- Comfort working across timezones.
Nice to have
- Hands-on experience training or serving large language models such as Nemotron, Qwen or OLMo.
- Experience with P2P networking and NAT traversal.
- Experience with post-training and RL.
- Experience with inference and serving systems.
- Experience at proprietary, open-weight and open-source AI labs
Benefits
- Equity-Heavy Package
- High base salary
- Remote-First Culture
- Optional full visa sponsorship
- Relocation support to Australia or the US
Before you apply
- Optional full visa sponsorship and relocation support to either Australia or the US is available.
About the company
Pluralis Research works on Protocol Learning to train and serve large models in a fully decentralized way on small consumer-grade devices. They are a world-class, deeply technical team of ML researchers backed by Union Square Ventures and other tier-1 investors.