Senior Software Engineer, Machine Learning Infrastructure & Automation

The role

You will build automation, infrastructure, and developer tooling to support the development, testing, and deployment of generative AI models. You will own ML CI/CD infrastructure and build systems for model validation, performance benchmarking, and deployment reliability.

What you will do

• Design, build, and maintain automated testing, validation, and deployment pipelines for ML models and inference services.
• Reduce CI execution times through parallelization, caching, test selection, and efficient compute use.
• Develop systems to test model outputs, detect quality regressions, and validate changes.
• Build continuous performance testing for inference latency, throughput, GPU utilization, and cost.
• Automate model pricing, billing configurations, API schemas, and deployment validation.
• Develop AI-powered automation and agentic coding systems to diagnose CI failures and propose fixes.
• Build automated safeguards, deployment verification, rollback mechanisms, and monitoring.
• Identify and automate repetitive tasks across the ML team to eliminate engineering toil.

Required qualifications

• 5+ years of software engineering background
• Proficiency in Python
• Experience building production infrastructure and developer tooling
• Experience designing and operating CI/CD systems using GitHub Actions or comparable technologies
• Deep understanding of automated testing, build systems, dependency management, caching, and parallel execution
• Experience working with containerized workloads, Docker, and cloud infrastructure
• Ability to design reliable distributed systems and debug complex infrastructure failures
• Strong understanding of observability, including logs, metrics, tracing, and automated alerting
• Ability to work independently, identify high-impact problems, and build end-to-end solutions

Nice to have

• Experience with ML infrastructure, PyTorch, GPU workloads, or model-serving systems
• Familiarity with NVIDIA GPU architectures and multi-GPU environments
• Experience building automated inference benchmarks or ML quality evaluation frameworks
• Experience with agentic coding tools such as Codex or Claude Code, or building custom AI engineering agents
• Experience optimizing CI/CD pipelines at scale, including distributed test execution and ephemeral compute environments
• Experience developing internal developer platforms or infrastructure-as-code tooling

Benefits

• Interesting and challenging work
• A lot of learning and growth opportunities
• Health, dental, and vision insurance (US)
• Regular team events and offsites

About the company

fal is the generative media ecosystem powering the next generation of AI products. They build infrastructure, tools, and model access for developers and enterprises.