Inference Intern

The role

Inference Architecture interns will develop and optimize compute architectures for inference workloads. You will work on architectural problems and performance modeling.

What you will do

• Support porting state-of-the-art models to our architecture.
• Help build programming abstractions and testing capabilities to rapidly iterate on model porting.
• Assist in building, enhancing, and scaling our runtime.
• Contribute to optimizing routing and communication layers using our collectives.
• Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.
• Develop and leverage a deep understanding of our architecture to co-design both HW instructions and model architecture operations.
• Implement high-performance software components for the Model Toolkit.

Required qualifications

• Progress towards a Bachelor’s, Master’s, or PhD degree in computer science, computer engineering, applied mathematics, or a related field
• Proficiency in Python, C++
• Understanding of performance-sensitive or complex distributed software systems, e.g. Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink,
• Ported applications to non-standard accelerator hardware or hardware platforms.
• Deep knowledge of transformer model architectures and/or inference serving stacks (vLLM, SGLang, etc.)

Nice to have

• Proficiency in Rust
• Low-latency, high-performance applications using both kernel-level and user-space networking stacks.
• Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns.
• Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).
• Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.
• Familiarity with PyTorch or JAX.
• Math competitions (AIME, AMC, etc)

Benefits

• Generous housing support for those relocating
• Daily lunch and dinner in our office
• Direct mentorship from industry leaders and world-class engineers

About the company

Etched is building hardware for frontier intelligence by co-designing chips, racks, software, and manufacturing. They are a fully in-person team in San Jose.