Software Engineer, Collective Communication

The role

You will design and implement custom networking collectives integrated into the training stack. You will work with C++ and CUDA to enable efficient training of flagship models on custom built supercomputers.

What you will do

• Collaborate closely with ML researchers to design and implement efficient collective operations in C++ and CUDA.
• Ensure that our largest training jobs take full advantage of the different network transports used in our supercomputers.
• Work on simulations to inform our future supercomputer network designs.

Required qualifications

• Background in low level performance critical software.
• Comfortable writing low level performance sensitive CPU and/or GPU code.

Nice to have

• Experience with collective communication
• Have written distributed algorithms using RDMA in the past
• Familiar with network simulation techniques

Benefits

• Relocation assistance

Before you apply

• Background checks for applicants will be administered in accordance with applicable law.

About the company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.