Software Engineer, AI Infrastructure - LVM Inference & Evaluation

The role

You will design, build, and optimize AI infrastructure for real-time computer vision, LLM, LVM, and multimodal inference workloads. You will work on systems to run deep learning models across many terabytes of video data in real time.

What you will do

• Design, build, and maintain AI infrastructure for real-time workloads.
• Build scalable systems for running models across large volumes of video and sensor data.
• Optimize inference performance across latency, throughput, GPU utilization, reliability, and cost.
• Develop evaluation harnesses and benchmarking systems.
• Build infrastructure for continuous model evaluation, experimentation, and deployment.
• Partner with research scientists to productionize AI advances.
• Improve model-serving architecture including batching, caching, routing, quantization, and parallelism.
• Develop data engines and feedback loops for training data and model improvement.
• Create observability, monitoring, and debugging tools for production AI systems.
• Define best practices for deploying and operating AI systems in enterprise environments.

Required qualifications

• 2+ years of industry experience building infrastructure, distributed systems, machine learning platforms, or production AI systems.
• BS/MS in Computer Science or a related technical field, or equivalent practical experience.
• Strong programming background, especially in Python, with solid software engineering fundamentals.
• Experience designing and building scalable machine learning infrastructure for training, inference, evaluation, and deployment.
• Hands-on experience running deep learning models in production, including LLMs, LVMs, vision-language models, or multimodal models.
• Strong understanding of inference optimization techniques, including batching, caching, quantization, parallelism, memory optimization, GPU utilization, and latency reduction.
• Experience with model-serving frameworks or systems such as vLLM, Triton Inference Server or similar technologies.
• Experience building evaluation frameworks, test harnesses, benchmarks, regression tests, or model-quality measurement systems.
• Strong background in machine learning and deep learning.
• Experience designing data engines or pipelines for collecting, managing, and curating training and evaluation data.
• Familiarity with integrating advanced AI systems such as LLMs, LVMs, RAG pipelines, embedding models, or multimodal models into production applications.
• Experience with cloud infrastructure, containers, orchestration, distributed systems, and GPU-based workloads.

Nice to have

• Experience operating large-scale GPU infrastructure or distributed inference systems.
• Experience with CUDA, NCCL, PyTorch, TensorRT, ONNX, or similar ML systems technologies.
• Experience with video understanding, real-time computer vision, multimodal AI, or physical-world AI systems.
• Experience with model compression, speculative decoding, distillation, pruning, or low-latency serving techniques.
• Experience with prompt evaluation, model regression testing, human-in-the-loop evaluation, or automated quality gates.
• Familiarity with retrieval-augmented generation, vector databases, embedding models, re-rankers, or search infrastructure.
• Experience building internal ML platforms or tools used by researchers and applied ML teams.
• Computer vision experience

Benefits

• Stock options
• Medical
• Dental
• Vision
• Life
• EAP
• Legal Services
• 401k plan
• Flexible time off
• Winter Break
• Latest tech and swag

About the company

Ambient.ai is the category creator and leader in Agentic Physical Security. The company is building a platform powered by Ambient Pulsar to unify monitoring, access control, threat assessment, response, and investigations.