Staff Engineer, Test Automation - San Mateo (R5792)

The role

You will lead automated testing and verification for the Hivemind autonomy software, including test architecture, validation infrastructure, MLOps quality systems, and CI/CD pipelines. The work covers flight code, machine learning models, simulation, operator applications and hardware-in-the-loop systems for unmanned aircraft and robotic platforms.

What you will do

• Own the technical strategy, architecture and roadmap for automated testing, verification and MLOps quality across the Hivemind ecosystem.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs and operator-facing applications.
• Lead functional, integration, regression, performance and end-to-end testing across simulation, software-in-the-loop and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness and latency.
• Establish CI/CD and continuous training workflows with versioning and traceability for datasets, models and deployment artifacts.
• Develop scenario-based validation for autonomy models, including edge cases, degraded sensing and distribution shifts.
• Create observability, analytics and failure-triage capabilities for software behavior, model drift and production performance.
• Build Python automation for test execution, parallelization, reporting and developer productivity.
• Collaborate with software, autonomy, machine learning and systems engineers to define verification strategies.
• Develop and govern AI-assisted engineering workflows for test generation, log analysis and failure triage.

Required qualifications

• Typically 8+ years of relevant experience in software engineering, test infrastructure, developer tooling, MLOps, systems integration or systems verification, or an equivalent combination of experienc
• 5+ years of experience building scalable automation frameworks or developer tooling in Python.
• Demonstrated success designing test, CI/CD or MLOps infrastructure used across multiple engineering teams.
• Experience validating machine learning systems across the data, training, evaluation, packaging, deployment and monitoring lifecycle.
• Understanding of ML quality risks such as data leakage, training-serving skew, nondeterminism, distribution shift and model regression.
• Experience defining model-performance baselines, automated evaluation suites and release thresholds.
• Experience testing GPU-accelerated infrastructure and workloads in Kubernetes environments.
• Experience with performance benchmarking, profiling and observability for GPU workloads.
• Experience validating multi-tenant Kubernetes environments, including RBAC, resource quotas and workload isolation.
• Experience qualifying integrated hardware and software systems, including compute, GPU, storage, networking, drivers and firmware.
• Strong system-design skills and experience testing distributed systems, backend services and APIs.
• Strong understanding of asynchronous and concurrent Python programming, and experience with package management and reproducible environments.

Nice to have

• Experience with model registries, experiment tracking and automated artifact promotion using MLflow, Kubeflow, Weights & Biases, SageMaker or Vertex AI.
• Experience with GPU scheduling and orchestration platforms such as Run:ai, NVIDIA GPU Operator, Kueue or Volcano.
• Experience with NVIDIA GPU infrastructure, including CUDA, drivers, container runtimes and Multi-Instance GPU.
• Experience with GPU profiling tools such as NVIDIA Nsight or PyTorch Profiler.
• Experience validating autonomy models in simulation and on deployed systems.
• Experience testing models on embedded or edge-compute platforms, including latency, memory, power and quantization.
• Experience qualifying production servers or appliances, including burn-in, provisioning and firmware validation.
• Experience with fault-injection and recovery testing across distributed compute, storage, networking and GPU infrastructure.
• Experience validating installation, upgrades and rollback in cloud, on-premises, disconnected or air-gapped environments.
• Experience with containers, Kubernetes, infrastructure as code and reproducible test environments.

Benefits

• Full-time regular employees receive a bonus.
• Full-time regular employees receive benefits.
• Full-time regular employees receive equity.
• Temporary employees receive a temporary benefits package applicable after 60 days of employment.

Before you apply

• All offers are contingent on a cleared background check and possible reference check.
• Compensation is influenced by skill set, experience level, licenses and certifications, and specific work location.
• Military fellows and part-time employees are not eligible for benefits.

About the company

Shield AI is a venture-backed defense-tech company that builds intelligent systems to protect service members and civilians. Its products include Hivemind autonomy software, V-BAT and X-BAT aircraft, and Aechelon simulation technologies.