CrawlJobs Logo

Senior Software Engineer - AI Frameworks

United States, Redmond Employment contract 119800.00 - 234700.00 USD / Year · Job Posted May 03, 2026
Apply Position
Job Link Share

Job Description

The AI Frameworks team at Microsoft accelerates and optimizes large language model deployment on Microsoft's MAIA AI accelerators and GPUs. We build software across the stack, from PyTorch and inference systems such as vLLM and SGLang to performance-critical runtime and kernel components. Our team operates at the intersection of AI algorithmic innovation, purpose-built AI hardware, systems, and software, with a highly collaborative and inclusive culture. We are seeking a self-motivated Senior Software Engineer - AI Frameworks who thrives on technical innovation, enjoys diving deep into technical details, and adapts quickly in a fast-moving environment. This is a unique opportunity to directly shape the software that powers Microsoft's most advanced AI infrastructure—from custom silicon to the models running on it. Microsoft's mission is to empower every person and every organization on the planet to achieve more. As employees, we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Job Responsibility

  • Architect and implement efficient tensor computation primitives and software abstractions for custom AI accelerators
  • Develop and extend PyTorch features for model onboarding, optimization, and execution on custom AI accelerators
  • Contribute to and improve AI inference stacks such as vLLM and SGLang, including scheduling, KV cache management, and serving pipelines
  • Design, develop, profile, and optimize high-performance kernels for NPUs (MAIA) and GPUs to accelerate LLM inference and training workloads
  • Collaborate across disciplines to define requirements and deliver practical solutions to new technical challenges

Requirements

  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR equivalent experience.
  • Master's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR equivalent experience.
  • Experience with PyTorch internals, custom operators, hardware backend, or torch.compile/Dynamo-based optimization flows.
  • Experience with AI inference stacks such as vLLM, SGLang, or similar large-scale model serving systems.
  • Experience with NPU or GPU kernel development and optimization (e.g., CUDA, Triton, or accelerator-specific toolchains).
  • Familiarity with common LLM concepts such as attention mechanisms, KV caching, quantization (PTQ/QAT), and distributed parallelism strategies (TP, PP, DP).

Nice to have

  • Experience with PyTorch internals, custom operators, hardware backend, or torch.compile/Dynamo-based optimization flows
  • Experience with AI inference stacks such as vLLM, SGLang, or similar large-scale model serving systems
  • Experience with NPU or GPU kernel development and optimization (e.g., CUDA, Triton, or accelerator-specific toolchains)
  • Familiarity with common LLM concepts such as attention mechanisms, KV caching, quantization (PTQ/QAT), and distributed parallelism strategies (TP, PP, DP)

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Software Engineer - AI Frameworks

8 matching positions

Senior Software Engineer - AI Frameworks

The AI Frameworks team at Microsoft develpos AI Software that enables running AI...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++ OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:  Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Apply engineering principles for defining robust and maintainable architectures and designs
  • Collaborate broadly across multiple disciplines, from hardware designers to ML Developers
  • Help establish and drive the adoption of good coding standards and patterns
  • Perform software development in C/C++ and other languages
  • Identify requirements, scope solutions, estimate work, schedule deliverables
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Managed AI - AI Platform

Be a part of the AI revolution with sustainable technology at Crusoe. Here, you'...
Location
Location
United States , San Francisco, CA; Sunnyvale, CA
Salary
Salary:
172425.00 - 209000.00 USD / Year
crusoe.ai Logo
Crusoe
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Advanced degree in Computer Science/Engineering
  • 4-5+ years of industry experience with demonstrated history of consistent success leading a varied portfolio of initiatives across your function
  • Experience with distributed systems, cloud services (compute, storage, networking, database), and delivering early-stage projects quickly
  • Experience with Generative AI (LLMs, Multimodal) and familiar with AI infrastructure (training, inference, ETL pipelines)
  • Proficient with container runtimes (e.g., Kubernetes), microservices, REST APIs, gRPC, and the full software development lifecycle including CI/CD
Job Responsibility
Job Responsibility
  • Lead the design and implementation of core AI services, including: Resilient fault-tolerant queues for efficient task distribution
  • Model catalogs for managing and versioning AI models
  • Scheduling mechanisms optimized for cost and performance
  • Architect and scale infrastructure to handle millions of API requests per second
  • Implement robust monitoring and alerting to ensure system health and 24/7 availability
  • Collaborate closely with product management, business strategy, and other engineering teams to define the AI platform roadmap
  • Influence the long-term vision and architectural decisions of the platform
  • Contribute to open-source AI frameworks and actively participate in the AI community
  • Prototype and rapidly iterate on emerging technologies and new features
What we offer
What we offer
  • Restricted Stock Units
  • Health insurance package options that include HDHP and PPO, vision, and dental for you and your dependents
  • Employer contributions to HSA accounts
  • Paid Parental Leave
  • Paid life insurance, short-term and long-term disability
  • Teladoc
  • 401(k) with a 100% match up to 4% of salary
  • Generous paid time off and holiday schedule
  • Cell phone reimbursement
  • Tuition reimbursement
  • Fulltime
Read More
Arrow Right

Software Engineer II and Senior Software Engineer - Performance

The Artificial Intelligence Performance team at Microsoft develops AI software t...
Location
Location
United States , Mountain View
Salary
Salary:
100600.00 - 199000.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Identify and drive improvements to end-to-end inference performance of OpenAI and other state-of-the-art LLMs
  • Measure, benchmark performance on Nvidia/AMD GPUs and first party Microsoft silicon
  • Optimize and monitor performance of LLMs and build SW tooling to enable insights into performance opportunities ranging from the model level to the systems and silicon level to improve customer experience and reduce the footprint of the computing fleet
  • Enable fast time to market of LLMs/models and their deployments at scale by building SW tools that afford velocity in porting models on new Nvidia and AMD GPUs
  • Design, implement, and test functions or components for our AI/DNN/LLM frameworks and tools
  • Speeding up/reducing complexity of key components/pipelines to improve performance and/or efficiency of our systems
  • Communicate and collaborate with our partners both internal and external
  • Embody Microsoft's Culture and Values
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - AI Engineering

RTB House is a global company that provides state-of-the-art marketing technolog...
Location
Location
Salary
Salary:
Not provided
rtbhouse.com Logo
RTB House
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Pragmatic Architect: Proven ability to evaluate third-party tools and vendor solutions against custom-built software to find the most efficient path forward
  • Technical Stack: High proficiency in Python is mandatory. Significant experience with at least one other language (Java, Go, TypeScript, or Scala) is highly preferred
  • AI/ML Implementation: Hands-on experience integrating LLMs into production environments
  • Systems Thinker: Ability to define technical roadmaps for specific features and drive high-level design choices that prioritize maintainability and performance
  • C1 level in English and Polish.
Job Responsibility
Job Responsibility
  • Drive Technical Excellence: Act as a technical pillar within the Lab, implementing high-standard code and sophisticated system designs. You will mentor mid-level peers and lead deep-dive code reviews
  • Architect Multi-Agent Systems: Design and deploy distributed systems and multi-agent architectures that automate complex engineering tasks. You will own the architectural decisions for 'build vs. integrate' strategies
  • Innovate with Agentic AI: Spearhead the evaluation and prototyping of LLMs, Agentic frameworks, and Model Context Protocols (MCPs). You will transform theoretical AI advancements into production-ready tools
  • Own the Full Lifecycle: Take responsibility for the entire development cycle. From initial concept and API integration to production deployment and long-term scalability
  • Influence Product Strategy: Partner with Product and Engineering Managers to ensure the Lab's innovations align with the broader company roadmap and provide measurable ROI to our developers.
What we offer
What we offer
  • Projects focused on extreme performance and high code quality – clean code and solid code reviews are our standard
  • Collaboration within an interdisciplinary, self-sufficient team (including DevOps, database experts, backend developers, product designers, and QA engineers)
  • Access to modern technologies and the opportunity to apply them in large-scale, high-impact projects.
  • Fulltime
Read More
Arrow Right

Principal AI Software Engineer, Senior Vice President

Are you looking for a career move that will put you at the heart of a global fin...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Exceptional Python Expertise: Demonstrated mastery of core Python, including advanced features, performance optimization, and a deep understanding of the FastAPI framework
  • Prior hands-on experience with Generative AI, Large Language Model (LLM) frameworks (e.g. LangChain, LlamaIndex), and their application in enterprise environments is a must. This must be underpinned by a profound understanding of core machine learning principles, algorithms, and data science methodologies
  • Full Lifecycle Ownership: Extensive hands-on experience and technical authority throughout the entire software development lifecycle, from conceptualization and design to implementation, deployment, and operational ownership of enterprise software solutions, involving significant cross-functional collaboration
  • Strategic System Design: Significant hands-on experience in architecting and designing (architecture, design patterns, reliability, scaling) highly complex new and current systems with broad technical impact
  • Hands-on expertise with containerized deployment technologies (e.g. Kubernetes, OpenShift, Docker) and orchestration strategies
  • Hands-on experience and in-depth understanding of C++ is a significant bonus, particularly for complex code analysis, parsing, and integration into knowledge graph structures
Job Responsibility
Job Responsibility
  • Architect and implement cutting-edge software systems, defining the technical design for our AI solutions to ensure scalability, performance, and reliability
  • Drive the hands-on design, implementation, and deployment of sophisticated systems that automate the analysis of data, code, and documentation
  • Apply deep expertise to structure extracted knowledge within a Credit Risk Domain-aware knowledge graph, including advanced strategies for effectively modelling complex codebases, particularly C++, within this graph
  • Act as a critical technical partner with data scientists, business analysts, and other engineering teams to translate challenging business requirements into robust technical solutions and ensure successful, high-quality project delivery
  • Tackle the most complex technical challenges within our AI initiatives, providing solutions that set the standard for engineering excellence
What we offer
What we offer
  • Generous holiday allowance starting at 27 days plus bank holidays
  • increasing with tenure
  • A discretional annual performance related bonus
  • Private medical insurance packages to suit your personal circumstances
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, AI Platform

Everlaw is looking for a Senior Software Engineer, AI Platform with experience b...
Location
Location
United States , Oakland
Salary
Salary:
173000.00 - 251000.00 USD / Year
everlaw.com Logo
Everlaw
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS or MS in Computer Science, or equivalent coursework
  • experience coding in languages such as C, C++, C#, Java, Python, Javascript, Go or Rust
  • good knowledge of algorithms and fundamental computer science concepts, relational databases, API design, and building user interfaces
  • practical experience with AI/ML-powered systems such as retrieval pipelines, semantic search features, agentic pipelines, document classification systems, or LLM-integrated features
  • work experience with AI development tools like Cursor and Claude Code
  • at least 4 years of experience building distributed systems in the cloud with service based architecture, using frontend frameworks to create rich, deep, web applications, and experience with the best practices to test, maintain, and launch cloud based software
  • at least 1 year of experience leading or coordinating multi-developer efforts, including planning and technical breakdown
Job Responsibility
Job Responsibility
  • Build AI platform capabilities that power product experiences such as Deep Dive, predictive coding, multi-modal understanding, agentic workflows, translations, search, review, and more
  • contribute to RAG, semantic retrieval, and agentic orchestration patterns, including indexing pipelines, query flows, tool-calling and planning logic, relevance tuning, benchmarking, and multi-modal workloads where applicable
  • collaborate with Product, Platform, Security, and DevOps partners to build and ship new features in our production environments
  • help with scaling our system to larger datasets with hundreds of millions of documents
  • provide technical mentorship to other engineers
  • be a code reviewer
  • fix defects in our product
  • provide on-call support for the product
  • contribute to documentation
  • do technical interviews
What we offer
What we offer
  • Equity program
  • 401(k) retirement plan with company matching
  • health, dental, and vision
  • Flexible Spending Accounts for health and dependent care expenses
  • paid parental leave and approximately 10 days (80 hours) per year of sick leave
  • seventeen paid vacation days plus 11 federal holidays
  • membership to Modern Health to help employees prioritize mental health and wellness
  • annual allocation for Learning & Development opportunities and applicable professional membership dues
  • company-sponsored life and disability insurance
  • work in Downtown Oakland, just steps from the BART line and dozens of restaurants
  • Fulltime
Read More
Arrow Right

Senior AI Software Engineer

We’re looking for a Senior AI Software Engineer to help build and scale producti...
Location
Location
Poland , Warszawa
Salary
Salary:
Not provided
devire.pl Logo
Devire
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in software engineering, AI engineering, or ML engineering
  • Strong Python skills and experience building production-grade APIs and backend systems
  • Hands-on experience with LLMs, embeddings, and agent-based architectures
  • Experience with cloud platforms (AWS and/or Azure) and modern deployment patterns
  • Solid understanding of CI/CD, testing, and infrastructure as code
  • Experience with monitoring, logging, and system reliability in production environments
  • Good understanding of performance, scalability, and cost trade-offs in AI systems
  • Strong problem-solving skills and ability to work in a cross-functional environment
  • Fluent English
Job Responsibility
Job Responsibility
  • Take AI use cases from prototype to fully productionized solutions, ensuring reliability, scalability, and security
  • Design and build cloud-native APIs and microservices for AI applications using Python (e.g. FastAPI, gRPC)
  • Develop and maintain agent-based systems, including tool integrations and orchestration workflows
  • Build and optimize RAG pipelines and/or text-to-SQL solutions, focusing on performance, cost efficiency, and accuracy
  • Implement CI/CD pipelines and automated testing for AI services
  • Deploy and manage applications in AWS and/or Azure using modern cloud architecture patterns (containers, serverless, etc.)
  • Ensure full observability of AI systems (logging, tracing, monitoring, cost tracking)
  • Introduce and maintain evaluation frameworks, guardrails, and quality controls for AI outputs
  • Collaborate closely with Data, ML, Product, and Engineering teams to deliver robust, end-to-end solutions
  • Contribute to architecture decisions, best practices, and engineering standards across the team
Read More
Arrow Right

Senior Software Engineer - AI Platform

We are looking for an AI Software Engineer with an AI-First mindset, focused on ...
Location
Location
Spain
Salary
Salary:
Not provided
https://feverup.com/fe Logo
Fever
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s and/or Master’s degree in Artificial Intelligence, Data Science, Mathematics, Physics, or Engineering fields
  • Strong proficiency in Python
  • Hands-on experience with integration of LLM into existing or new applications
  • Experience applying Retrieval-Augmented Generation (RAG), prompt engineering, and fine-tuning
  • Experience with Continuous integration and continuous deployment pipelines to take the code from development to production and monitoring
Job Responsibility
Job Responsibility
  • Configure and implement applications using Large Language Models (LLMs) and other Generative AI models (multimodal, AI agents, etc.)
  • Integrate AI APIs and tools from providers like OpenAI, Anthropic, Google, Meta, Hugging Face, Stability AI, and others
  • Continuously improve and experiment with new AI architectures, frameworks, and best practices
  • Collaborate with development teams to incorporate AI-powered functionalities into the engineering processes and tools
What we offer
What we offer
  • 40% discount on all Fever events and experiences
  • Home office friendly anywhere in Spain
  • Relocation package for international candidates
  • Health insurance and other benefits such as Flexible remuneration with a 100% tax exemption through Cobee
  • English Lessons
  • Wellhub Membership
  • Possibility to receive in advance part of your salary by Payflow
  • Attractive compensation package consisting of base salary and the potential to earn a significant bonus for top performance and Stock Options
  • Fulltime
Read More
Arrow Right