Embedded AI Engineer, On-Device Models

The role

You will write and optimize custom kernels and operators for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs. You will also perform target-side optimization to meet latency, memory, power, and thermal budgets.

What you will do

• Write and optimize custom kernels and operators in C, C++, Rust, and platform assembly or intrinsics
• Collapse models onto device execution units through quantization, operator fusion, memory layout, and architecture-specific compilation
• Integrate with vendor NPU/DSP toolchains and edge inference runtimes
• Deliver kernels and runtime components as reusable building blocks
• Build performance-critical runtime code for embedded Linux, bare-metal, and RTOS targets
• Establish per-platform benchmarking and validation for latency, accuracy, power, memory footprint, and utilization
• Partner with silicon and platform vendors on SDK integration and low-level performance tuning
• Feed hardware constraints back to Applied ML and Research

Required qualifications

• Experience delivering production systems on resource-constrained hardware such as embedded systems, mobile, edge AI, or small low-power devices
• Strong proficiency in C, C++, and/or Rust
• Experience writing performance-critical code for constrained environments
• Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation
• Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains
• Strong understanding of hardware-software interaction including CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management
• Experience working close to the metal in bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development
• Strong communication skills and a builder mindset

Nice to have

• Experience with real-time audio processing on embedded platforms
• Depth in ML optimization techniques
• Background in hardware evaluation and benchmarking
• Experience shipping AI features in consumer products at scale
• Familiarity with model compilation and optimization toolchains
• Experience with secure, robust on-device deployment practices

About the company

Deepgram is a research company that productizes as fast as possible. Their speech models and real-time infrastructure power AI systems for more than 5,000 organizations.