Staff Engineer, Test Automation (R5792)

The role

This Staff Automated Test Engineer will set the technical direction for automated testing, verification, and MLOps quality across the Hivemind autonomy platform. The work covers test frameworks, ML validation pipelines, CI/CD, and simulation and hardware-in-the-loop testing for unmanned aircraft and robotic systems.

What you will do

• Own the technical strategy, architecture and roadmap for automated testing and MLOps quality across the Hivemind ecosystem.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs and operator applications.
• Lead functional, integration, regression, performance and end-to-end testing across simulation, software-in-the-loop and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness and latency.
• Establish CI/CD and continuous training workflows with traceability for datasets, models and deployment artifacts.
• Develop scenario-based validation for autonomy models, including edge cases and degraded sensing or communications.
• Create observability, analytics and failure-triage capabilities for software behavior, model drift and production performance.
• Build Python automation for test execution, parallelization, reporting and environment setup.
• Create test harnesses, simulators, mocks and synthetic data capabilities.
• Develop and govern AI-assisted engineering workflows for test generation, log analysis and failure triage.

Required qualifications

• Typically 8+ years of relevant experience in software engineering, test infrastructure, developer tooling, MLOps, systems integration or systems verification, or an equivalent combination of experienc
• 5+ years of experience building scalable automation frameworks or developer tooling in Python.
• Demonstrated success designing test, CI/CD or MLOps infrastructure used across multiple engineering teams.
• Experience validating machine learning systems across the data, training, evaluation, packaging, deployment and monitoring lifecycle.
• Understanding of ML quality risks such as data leakage, training-serving skew, nondeterminism, distribution shift and model regression.
• Experience defining model-performance baselines, automated evaluation suites and release thresholds.
• Experience testing GPU-accelerated infrastructure and workloads in Kubernetes environments.
• Experience with performance benchmarking, profiling and observability for GPU workloads.
• Experience validating multi-tenant Kubernetes environments, including RBAC, resource quotas and workload isolation.
• Experience qualifying integrated hardware and software systems, including compute, GPU, storage, networking, drivers and firmware.
• Strong system-design skills and experience testing distributed systems, backend services and APIs.
• Strong understanding of asynchronous and concurrent Python programming, and experience with reproducible environments and build systems such as Conan, pip, Poetry or Nix.

Nice to have

• Experience with model registries, experiment tracking and artifact promotion using MLflow, Kubeflow, Weights & Biases, SageMaker or Vertex AI.
• Experience with GPU scheduling platforms such as Run:ai, NVIDIA GPU Operator, KAI Scheduler, Kueue or Volcano.
• Experience with NVIDIA GPU infrastructure, including CUDA, drivers, container runtimes and Multi-Instance GPU.
• Experience with GPU profiling tools such as NVIDIA Nsight or PyTorch Profiler.
• Experience validating autonomy models in simulation and on deployed systems.
• Experience testing models on embedded or edge-compute platforms, including latency, memory, power and quantization.
• Experience with reliability, fault-injection and recovery testing across distributed infrastructure.
• Experience with containers, Kubernetes, cloud infrastructure and infrastructure as code.
• Proficiency with Go or TypeScript for automation tooling or UI test development.
• Experience integrating Python with native C or C++ applications.

Benefits

• Bonus
• Benefits package (temporary employees receive a temporary benefits package after 60 days of employment; military fellows and part-time employees are not eligible for benefits)
• Equity

Before you apply

• All offers are contingent on a cleared background and possible reference check.
• Pay depends on factors such as skill set, level of experience, licences and certifications, and work location.

About the company

Shield AI is a venture-backed defense-tech company that builds intelligent systems to protect service members and civilians. Its products include Hivemind autonomy software, V-BAT and X-BAT aircraft, and Aechelon simulation technologies.