Inference Software Engineer

The role

This person will support porting state-of-the-art models to the company architecture. They will build and scale the runtime and optimize routing and communication layers.

What you will do

• Support porting state-of-the-art models to our architecture
• Help build programming abstractions and testing capabilities to rapidly iterate on model porting
• Build, enhance, and scale our runtime
• Optimize routing and communication layers using our collectives
• Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues

Required qualifications

• Proficiency in C++ or Rust
• Understanding of performance-sensitive or complex distributed software systems like Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, I
• Familiarity with PyTorch or JAX
• Ported applications to non-standard accelerator hardware or hardware platforms

Nice to have

• Developed low-latency, high-performance applications using both kernel-level and user-space networking stacks
• Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns
• Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE)
• Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths

Benefits

• Medical, dental, and vision packages with generous premium coverage
• $500 per month credit for waiving medical benefits
• Housing subsidy of $2.5k per month for those living within walking distance of the office
• Relocation support for those moving to San Jose (Santana Row)
• Various wellness benefits covering fitness, mental health, and more
• Daily lunch + dinner in our office
• Unlimited compute budget subject to ROI justification

About the company

Etched is building hardware for frontier intelligence by co-designing chips, racks, software, and manufacturing. They are a fully in-person team in San Jose (Santana Row).