Member of Technical Staff - Inference

The role

You will build and optimize the inference stack that runs personal AI models locally on Core. You will improve latency, throughput, and memory efficiency while profiling performance across the hardware and software stack.

What you will do

• Build and optimize the inference stack that runs personal AI models locally on Core.
• Improve latency, throughput, and memory efficiency across interactive requests and background agent workloads.
• Profile performance across the hardware and software stack, optimizing kernels, memory movement, and runtime execution.
• Build benchmarks that reflect real product workloads, including long contexts, concurrent agents, and sustained operation.
• Work closely with the hardware, post-training, and product teams.

Required qualifications

• Experience building or optimizing inference systems for language models.
• Strong systems programming skills in C++ or similar languages.
• Proficiency in Python.
• Familiarity with GPU architecture, memory management, and performance profiling.
• Experience with inference runtimes, quantization, or GPU kernel development using CUDA, Triton, or equivalent tools.
• Strong experimental judgment to identify bottlenecks and evaluate tradeoffs between speed, memory, power, and model quality.
• Ability to work on-site in San Francisco.

About the company

Ghost is a personal AI hardware company. They are building an ecosystem of devices and software to give users greater control over their intelligence.