Senior Software Engineer - Compute

At the heart of orchestrating this monumental compute infrastructure is the Compute team. This team is dedicated to building and maintaining the foundational technology that solves the fundamental challenges of resource scheduling, task isolation, and distributed state consistency across our massive batch compute fleet. At our scale, traditional off-the-shelf orchestrators break. The Compute team builds the custom engine - BatchAPI - that manages the lifecycle of millions of tasks - built on top of K8s primitives but implements our own custom scheduler. We deal with the 'unsolved' problems of distributed computing: maximizing hardware utilization while ensuring that a failure in one node doesn't cascade across the entire cluster. This engine is engineered to handle massive scale, ensuring reliability, efficiency, and rapid turnaround for our engineers.