Principal Software Engineer - System Optimization

The Artificial Intelligence Frameworks team at Microsoft builds the software foundation that enables AI to run everywhere. We operate at the intersection of AI algorithmic innovation, purpose-built hardware, large-scale systems, and production software, working closely with internal hardware teams and external partners. Our team is composed of highly capable, deeply motivated engineers who value technical excellence, collaboration, and an inclusive culture. We own inference performance for OpenAI and other state-of-the-art large language models, working directly with OpenAI on models deployed through Azure OpenAI Service. These systems power some of the largest AI workloads on the planet—serving trillions of inferences per day across Microsoft products. As a Principal Software Engineer - System Optimization you will provide technical leadership across the AI inference software stack, with a strong focus on high-performance, large-scale serving systems. You will lead the benchmarking and optimization of cutting-edge LLMs across GPUs and Microsoft’s custom AI hardware, architect improvements to distributed serving pipelines, and drive deep performance investigations across complex, multi-layered systems. You will own critical performance and efficiency metrics, design and implement durable optimizations, and influence technical direction across teams. In close partnership with research, hardware, and production engineering groups, you will help deliver next-generation AI capabilities into Microsoft’s most widely used products—directly shaping Azure’s efficiency and the future of Microsoft’s AI infrastructure.