Technical Director, Large-Scale AI Model Inferencing

The role

This Principal Engineer will own requirements for full-stack AI memory solutions at scale. You will connect model behavior to memory-system design and develop products that exploit memory bandwidth and capacity.

What you will do

• Serve as expert on how different model families consume and move memory
• Model the memory footprint, bandwidth demand, and access patterns of frontier open-weight models
• Track the model landscape as a roadmap input
• Own deep expertise in production inference stacks
• Drive inference performance engineering
• Own the latency, throughput, and cost envelope
• Define benchmarking and characterization methodology
• Define engineering requirements for tiered memory systems for inference at fleet scale
• Design expert-weight offloading solutions for MoE serving
• Translate model knowledge into product requirements and reference architectures

Required qualifications

• BS in Computer/Electrical/Electronic Engineering or Computer Science, and 10+ years of relevant experience
• 12+ years in systems engineering
• 4+ years hands-on in large-scale LLM inference or GPU systems performance
• First-principles understanding of transformer-class model internals
• Working expertise with MoE model behavior
• Direct experience with at least one major inference stack's memory-management internals
• Strong performance-engineering skills
• Track record of building systems software at the memory/storage/IO layer
• Ability to write models and simulators
• Excellent written and verbal communication

Nice to have

• MS in Computer/Electrical/Electronic Engineering or Computer Science with 8 years of relevant experience
• Experience with State Space Model or hybrid SSM-attention serving
• Contributions to open-source inference/caching projects
• Experience with CXL memory pooling, CXL-attached tiering, or near-memory processing
• Experience with SSD/NVMe as a KV or expert cache tier
• Background in memory/storage product companies

Benefits

• Incentive opportunities
• Medical/Dental/Vision
• 401k
• Charitable giving match
• 4+ weeks of paid time off a year, plus holidays and sick leave
• Stipend for fertility care or adoption
• Caregiver support
• Medical travel support
• On-demand apps and free confidential therapy sessions
• Onsite Café and gym
• Virtual classes
• Flexible environment

Before you apply

• Each candidate is limited to 10 applications across all open jobs within a 6-month period
• AI tools should not be used to generate or assist with submitted content or live interview responses
• By submitting an application, you agree not to disclose any confidential or proprietary information belonging to a current or former employer

About the company

Samsung Semiconductor provides technology solutions that power tools including smartphones, electric vehicles, hyperscale data centers, and IoT devices. It is a global leader in innovative designs.