Fireworks

Member of Technical Staff, Data Infrastructure

San Mateo, Remote. Posted 9 days ago

The role

This person will build a unified, high-performance data layer spanning parallel file systems, object storage, distributed caches, and key-value stores. The role involves managing how data is stored, cached, and moved across clouds for GPU workloads.

What you will do

  • Build a unified, high-performance data layer across multiple clouds.
  • Manage datasets, checkpoints, model weights, and KV cache placement, replication, and movement.
  • Ensure training jobs start without a data-staging step.
  • Ensure inference replicas cold-start without waiting on weight downloads.
  • Own how data is stored, cached, and moved across clouds.

Required qualifications

  • Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience).
  • 5+ years of experience designing, building and operating storage, caching or data infrastructure in production at scale.
  • Strong software development skills in C/C++, Rust or Go, with experience writing performance-critical data path code.
  • Python or Go for automation.
  • Deep understanding of distributed storage fundamentals: consistency models, replication and consensus (e.g., Raft/Paxos), metadata scaling, caching, and failure recovery.
  • Hands-on experience with Kubernetes, including CSI, to mount high-performance volumes into ephemeral GPU pods.
  • Experience deploying and operating storage clusters (e.g., WEKA, VAST, Lustre, Ceph, MinIO) across clouds using infrastructure-as-code tools such as Terraform or Pulumi.
  • Proficiency with performance benchmarking and profiling tools (e.g., fio, iperf3, NCCL tests, perf, eBPF) to identify and resolve bottlenecks across the kernel, network, cache and storage backend.

Nice to have

  • Experience as a core contributor or owner of a storage, caching or KV system, either in-house or open source (e.g., Ceph, Alluxio, JuiceFS, RocksDB, FoundationDB, TiKV, Redis/Dragonfly, SeaweedFS, Min
  • Experience with high-performance computing or large-scale AI clusters (e.g., Slurm, Kubernetes GPU pools) and familiarity with AI I/O patterns such as checkpoint bursts, random dataset reads, and mode
  • Familiarity with Linux internals, including kernel I/O paths, POSIX semantics, page cache behavior, io_uring/SPDK, and OS-level tuning (sysctl, I/O schedulers, NUMA).
  • Familiarity with high-performance networking (RDMA, RoCE v2, InfiniBand) and GPUDirect Storage.
  • Experience moving, replicating or caching petabyte-scale datasets across clouds (e.g., AWS, GCP and specialized neoclouds), with attention to egress cost and consistency.
  • Preference for software-defined storage over proprietary hardware appliances.

About the company

Fireworks is a Series D company valued at $17.5 billion that provides a platform for specialized intelligence. The platform enables companies to build, train, and serve AI models tailored to their own data, workflows, and products.