Senior ML Infrastructure / DevOps Engineer

Role overview

Pathway is seeking a Senior ML Infrastructure / DevOps Engineer to own production infrastructure for ML training and low-latency inference. You’ll work closely with R&D while focusing on clusters, networks, storage, observability, and automation, across multiple cloud providers.

Responsibilities

• Operate and scale GPU-heavy clusters for daily R&D training and low-latency inference.
• Design, build, and automate the ML platform (not just run playbooks).
• Work across multiple cloud providers, addressing networking, scheduling, and cost/performance optimization.
• Design, operate, and scale GPU/CPU clusters using Slurm and Kubernetes, including autoscaling, queueing, and quota management.
• Automate provisioning and configuration with infrastructure-as-code (Terraform, CloudFormation) and cluster tooling.
• Build and maintain ML pipelines for data ingestion, training, evaluation, and deployment with reproducibility, traceability, and rollback.
• Implement and evolve ML-centric CI/CD for testing, packaging, and deployment of models/services.
• Own monitoring, logging, and alerting for training and serving (GPU/CPU utilization, latency, throughput, failures, and data/model drift) using Grafana, Prometheus, Loki, CloudWatch.
• Handle terabyte-scale datasets and related storage/networking/performance challenges.
• Partner with ML engineers/researchers to productionize experimental work.
• Participate in on-call rotation; lead incident response and post-mortems.

Requirements

• 5+ years in DevOps/SRE/Platform/Infrastructure roles running production systems, ideally for high-performance or ML workloads.
• Strong Linux background as a daily driver, including shell scripting and cluster/service configuration.
• Comfort debugging at OS and network layers (e.g., systemd, filesystems, iptables/security groups, DNS, TLS, routing).
• Strong experience with workload management, containerization, and orchestration in production (Slurm, Docker, Kubernetes).

Nice to have

• Deep experience scaling ML infrastructure across multiple cloud providers with networking/scheduling/cost optimization.