CrawlJobs Logo

Principal Software Engineer - CoreAI Model Inference & Serving

https://www.microsoft.com/ Logo

Microsoft Corporation

Location Icon

Location:
United States , Redmond

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

139900.00 - 274800.00 USD / Year
Save Job
Save Icon
Apply Position

Job Description:

Join our team within CoreAI, where we are building the AI data-plane that powers all LLM inferencing workloads across Microsoft and Azure customers—from cutting-edge startups to Fortune 500 enterprises. Our converged AI fabric delivers inference capabilities for all LLMs in Microsoft catalog, including OpenAI, Anthropic, Mistral, Cohere, Llama, and more. As a Principal Software Engineer, you will shape the future of one of the largest and fastest-growing services in Azure, foundational to Microsoft’s AI strategy. Our mission is to serve models at scale—reliably, efficiently, and with ultra-low latency—enabling a rich set of AI-powered product experiences. This is a rapidly evolving space with immense opportunities to learn, innovate, and drive industry-wide impact!

Job Responsibility:

  • Be a hands-on technical leader, designing, coding, and shipping core serving systems, smart routing, and request distribution for a broad portfolio of LLMs, including OpenAI, Mistral, Grok, DeepSeek, and others
  • Build large-scale AI services and platform capabilities that power new products and customer experiences
  • Drive cutting-edge innovation in AI systems alongside world-class engineers and cross-functional partners
  • Lead through architecture, code reviews, mentorship, and technical excellence while staying close to implementation
  • Improve reliability, scalability, observability, efficiency, and performance across mission-critical services

Requirements:

  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, or Java
  • OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter

Nice to have:

  • 4+ years of design and problem-solving experience, with understanding of system performance, scalability, and engineering best practices
  • Understanding of distributed systems specifically in request serving at scale
  • (e.g. inferencing, L7 gateways, high-performance storage, distributed databases across global-scale infrastructure)
  • Demonstrated experience in building high-quality, reliable systems at scale
  • Experience using modern AI-assisted development tools and workflows to move faster, improve quality, and amplify engineering impact
  • Customer-obsessed approach to problem solving, with empathy and a drive to deliver impactful solutions

Additional Information:

Job Posted:
April 03, 2026

Employment Type:
Fulltime
Work Type:
Hybrid work
Icon
Microsoft Corporation - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon
PREMIUM
More languages and countries
+ Unlock 31694 hidden job offers
Languages
English Čeština Deutsch Ελληνικά Español Français +15
Countries
United States United Kingdom India Canada Australia +
See plans
Plans from $2.99 / month

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Principal Software Engineer - CoreAI Model Inference & Serving

Principal Product Manager/Architect - Foundry Inference Platform (CoreAI)

We are seeking a Principal Product Manager/Architect to define and guide the tec...
Location
Location
United States , Redmond
Salary
Salary:
163000.00 - 296400.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 10+ years experience in product/service/program management or software development OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check
Job Responsibility
Job Responsibility
  • 1. Product Reliability: Own the product direction for Microsoft Foundry inference, with a primary mandate to make the platform the most reliable enterprise inferencing service available. This includes defining architectural standards for global serving, multi-region resiliency, automated failover, and platform-managed disaster recovery
  • Drive architectural alignment across global routing, capacity pooling, observability, and control plane abstractions to ensure consistent availability, predictable recovery behavior, and simplified customer operations at scale
  • Partner with engineering, infrastructure, and security leaders to ensure reliability targets, SLAs, SLOs and recovery objectives are designed into the platform by default
  • 2. GPU Fleet Efficiency & Capacity: Set the product direction for GPU fleet efficiency and capacity management, guiding platform-level design decisions that maximize utilization, minimize fragmentation, and accelerate timetomonetization of new hardware and models
  • This includes shaping the architecture for global capacity pooling, intelligent scheduling, fungibility across workloads, automated demand forecasting, and softwaredefined allocation
  • The Product Manager/Architect is expected to influence architectural investments across inference utilization, model serving, and hardware/system performance
  • 3. Strategic Customer & Innovation Engagement: Act as a senior technical advisor and architect for Foundry’s most innovative and strategic customers
  • Engage directly with customers on deep technical challenges, including largescale model migrations, reliabilitysensitive production deployments, and advanced serving architectures
  • Support competitive and strategic initiatives by articulating Foundry’s architectural advantages, turning bespoke requests into scalable features
  • 4. Cross-Company Technical Leadership: Serve as a unifying architectural voice across product management, engineering, infrastructure, and partner teams
  • Fulltime
Read More
Arrow Right
New

Senior Lecturer/Associate Professor in Literacy

As a Senior Lecturer / Associate Professor in Literacy, you will play a key role...
Location
Location
Australia , Albury-Wodonga, Bathurst, Port Macquarie, Wagga Wagga
Salary
Salary:
Not provided
csu.edu.au Logo
Charles Sturt University
Expiration Date
June 08, 2026
Flip Icon
Requirements
Requirements
  • A doctoral qualification relevant to literacy or education, with a recognised teaching qualification
  • A strong record of high-quality teaching and student-centred learning
  • An established or emerging research profile aligned to literacy, curriculum or pedagogy
  • The ability to build productive partnerships and contribute to academic leadership
Job Responsibility
Job Responsibility
  • Lead impactful literacy teaching and research
  • Teach across online and on-campus environments
  • Shape future teachers and education practice
  • Contribute to curriculum innovation
  • Build strong relationships with students and partners
  • Provide academic leadership in literacy education
  • Contribute to the School's research profile
  • Supervise higher degree research students
  • Actively engage with professional, community and government stakeholders
  • At Associate Professor level: significant academic leadership, research impact, and contribution to the broader discipline at national/international level
What we offer
What we offer
  • 17% superannuation
  • Fulltime
Read More
Arrow Right
New

Program Manager - Controls and Avionics Solutions

This position is based in Endicott, New York. New York and on-site work will be ...
Location
Location
United States , Endicott
Salary
Salary:
120874.00 - 205486.00 USD / Year
baesystems.com Logo
Baesystems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in engineering, engineering or manufacturing management, or other discipline
  • Demonstrated ability for building strong customer/ stakeholder relationships
  • Strong communication, negotiation, and presentation skills
  • Ability to interpret data and make data-driven decisions
  • Highly adaptable with strong initiative
  • Demonstrated ability to lead and motivate cross-functional teams
  • Knowledge of the global aviation market and regulatory requirements and/ or military aviation market
Job Responsibility
Job Responsibility
  • Maintaining strong customer relationships and leading a multidisciplinary team to execute complex development programs within schedule and budget
  • Leadership and management oversight of a project team assuring that project’s financials, schedule, and technical objectives are met and that the highest level of customer satisfaction is achieved while meeting all contractual commitments
  • Work effectively and collaboratively with Engineering, Operations, and all Program Office functional leadership to assure deliveries continue to exceed customer commitments and achievement of financial commitments to the company
  • Manages, coordinates, plans, organizes, controls, integrates, and executes projects within the Military Aircraft Systems portfolio
  • Participates in the support of new business and in the development of proposals
What we offer
What we offer
  • Health insurance
  • Dental insurance
  • Vision insurance
  • Health savings accounts
  • 401(k) savings plan
  • Disability coverage
  • Life and accident insurance
  • Employee assistance program
  • Legal plan
  • Discounts on home, auto, and pet insurance
  • Fulltime
Read More
Arrow Right
New

Finance Business Partner (Research)

Full Time, Fixed Term (12 months). Level 7 - $101,421 to $110,819 p.a. (plus 17%...
Location
Location
Australia , Wagga Wagga
Salary
Salary:
101421.00 - 110819.00 AUD / Year
csu.edu.au Logo
Charles Sturt University
Expiration Date
June 02, 2026
Flip Icon
Requirements
Requirements
  • A degree in Accounting or Finance (professional accounting body membership is desirable)
  • Experience in project budgeting, forecasting and financial analysis
  • Background in management accounting or business partnering within complex environments
  • exposure to government funding or higher education is advantageous
  • Excellent stakeholder engagement skills, with the ability to work effectively with academics and researchers
  • Familiarity with business intelligence systems and dashboard reporting
Job Responsibility
Job Responsibility
  • Partner with academics to deliver strategic financial insights that enable research success
  • Directly influence world-class projects and decisions shaping the future of education and innovation
  • Lead initiatives that enhance financial governance, deliver accurate and timely reporting, and support key projects such as cost-pricing systems and research budgeting
  • Help build financial capability across the University, fostering collaboration and continuous improvement
What we offer
What we offer
  • Flexibility with a 35-hour work week
  • Access to hybrid work arrangements
  • 17% superannuation
  • Fulltime
Read More
Arrow Right
New

Associate Lecturer/ Lecturer in Oral Health

Make a real impact by educating future oral health professionals to serve the ur...
Location
Location
Australia , Wagga Wagga
Salary
Salary:
80046.00 - 134965.00 AUD / Year
csu.edu.au Logo
Charles Sturt University
Expiration Date
June 16, 2026
Flip Icon
Requirements
Requirements
  • A qualification relevant to the discipline and appropriate to the level being applied for
  • Full registration (for teaching/research) as a Dentist or Oral Health Therapist with the Australian Health Practitioner Regulation Agency (Ahpra)
  • Excellent understanding of the clinical practice of oral health therapy, supported by a record of teaching and subject coordination relevant to the discipline and appropriate to the level being applied for
  • Evidence of the delivery of high quality student-centred learning and teaching in oral health therapy and/or general dentistry
  • A record of research activity or capability relevant to the discipline and appropriate to the level being applied for, as outlined in the position descriptions, may facilitate the progression of research opportunities
Job Responsibility
Job Responsibility
  • deliver high-quality teaching, clinical supervision and learning experiences in Oral Health
  • work with students in both clinical and preclinical settings while contributing to curriculum development, industry engagement and community partnerships
What we offer
What we offer
  • Generous support provided to assist with relocating to Riverina’s beautiful Wagga Wagga or surrounds
  • 17% superannuation
  • Fulltime
Read More
Arrow Right
New

Change Analyst

As Change Analyst you will provide specialist change management expertise to sup...
Location
Location
Australia , Albury-Wodonga, Bathurst, Dubbo, Orange, Wagga Wagga
Salary
Salary:
101421.00 - 110819.00 AUD / Year
csu.edu.au Logo
Charles Sturt University
Expiration Date
June 03, 2026
Flip Icon
Requirements
Requirements
  • Relevant qualifications and/or equivalent experience in organisational change and transformation
  • Experienced in applying change management frameworks and methodologies to large-scale/complex organisational initiatives
  • Skilled in analysing change impacts and shaping clear, targeted responses in policy-driven environments
  • Strong communication and interpersonal skills
Job Responsibility
Job Responsibility
  • Provide specialist change management expertise to support the successful planning and implementation of the Models of Engagement and Assessment initiative
  • Lead change analysis, stakeholder engagement planning and adoption activities to enable a sustainable transition to new models of course delivery and assessment.
What we offer
What we offer
  • Competitive salary and benefits including 17% super
  • Flexible working arrangements that support a healthy work-life balance
  • Fulltime
Read More
Arrow Right
New

Postdoc / Research Fellow in Digital Agricultural Futures

We are seeking a Research Associate / Postdoctoral Research Fellow (Level A) or ...
Location
Location
Australia , Mildura
Salary
Salary:
80046.00 - 134965.00 AUD / Year
csu.edu.au Logo
Charles Sturt University
Expiration Date
June 10, 2026
Flip Icon
Requirements
Requirements
  • Level A: A relevant postgraduate qualification (Masters or PhD) or equivalent experience in digital agriculture, irrigation, spatial science or related fields
  • Level B: A completed PhD (or equivalent standing) with demonstrated independent research capability
  • Experience in applied or multidisciplinary research environments, ideally connected to agriculture, education, or regional systems
  • Knowledge of, or interest in, education and training frameworks, workforce development or professional learning
  • Strong communication and relationship-building skills, with the ability to work effectively with researchers, industry and community stakeholders
  • The ability to manage priorities, work independently and collaboratively, and contribute to impactful research outcomes
Job Responsibility
Job Responsibility
  • Preparing the future workforce for digital irrigated agriculture
  • Working closely with academics, industry partners, education providers and communities across the Murray–Darling Basin
  • Contributing to research that explores digital literacy, education frameworks and innovative delivery models for contemporary agriculture
  • At Level A: contributing to research delivery under the guidance of senior researchers, supporting data collection, analysis, stakeholder engagement and co-authored outputs
  • At Level B: taking a more independent and substantive leadership position, leading defined research components, cultivating partnerships, and producing high-quality scholarly and industry-focused outputs
  • Regular interstate travel is required, along with strong collaboration across multidisciplinary and industry-linked projects
What we offer
What we offer
  • 17% superannuation
  • Relocation opportunity to Mildura, VIC
  • Flexible/hybrid arrangements considered
  • Fulltime
Read More
Arrow Right
New

Collections Representative

We are seeking a detail-oriented Collections Representative to manage outstandin...
Location
Location
United States , Fairfax
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 1+ years of collections, accounts receivable, or customer service experience
  • Strong communication skills and a dependable, team-oriented work style
  • Strong data entry skills with the ability to maintain speed and accuracy
  • Ability to manage difficult conversations professionally
  • Attention to detail and strong organizational skills
  • Proficiency with Microsoft Office and accounting/ERP systems
Job Responsibility
Job Responsibility
  • Handle a large volume of incoming calls and assist customers with clarity and efficiency
  • Respond to customer inquiries by phone and email, providing timely updates and accurate information
  • Enter and update customer or account information with a high level of accuracy in internal systems
  • Support payment posting activities and help ensure transactions are recorded correctly
  • Complete administrative support work such as scanning documents, preparing mail, and organizing records
  • Place outbound calls when needed to follow up on account matters or clarify customer information
  • Assist different departments with routine support tasks to help maintain smooth daily operations
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • Company 401(k) plan
  • Fulltime
Read More
Arrow Right