CrawlJobs Logo

Senior Software Engineer - Data Platform, AI Infrastructure

United States, Redmond Employment contract 119800.00 - 234700.00 USD / Year · Job Posted May 03, 2026
Apply Position
Job Link Share

Job Description

We are building a large-scale, productized data platform that powers critical insights and systems across Azure-based services for AI Infrastructure. This platform will process terabytes to petabytes of data daily and is designed for reliability, scalability, and long-term evolution. As a Senior Software Engineer - Data Platform, AI Framework you will focus on building and operating the core infrastructure layer of the platform - covering orchestration, APIs, observability, and system reliability. You will work closely with data engineers and partner teams to ensure the platform is robust, standardized, and capable of supporting rapid growth. We are looking for engineers who execute well, own systems end-to-end, and bring structure to complex problems - not just ideas. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Job Responsibility

  • Design, build, and operate core components of a distributed data platform, including: Orchestration systems (e.g., Airflow or equivalent)
  • Backend services and APIs (Python/FastAPI or similar)
  • Monitoring, alerting, and reliability systems
  • Own the end-to-end lifecycle of platform components - from design through deployment, scaling, and maintenance
  • Ensure systems meet requirements for availability, performance, and data reliability at large scale
  • Define and enforce standardized patterns for infrastructure, deployment, and observability across the platform
  • Partner with data engineering teams to enable efficient, reliable data processing workflows
  • Diagnose and resolve complex issues in distributed systems, including performance bottlenecks and failure modes
  • Contribute to infrastructure-as-code and deployment systems to support reproducibility and operational excellence
  • Drive continuous improvements in system robustness, cost efficiency, and operational clarity

Requirements

  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
  • Strong programming experience in Python
  • Experience building and operating large-scale distributed systems
  • Hands-on experience with: Backend services or APIs (e.g., FastAPI, Flask, or similar)
  • Cloud-based infrastructure (Azure, AWS, or GCP)
  • Monitoring and observability systems (metrics, logging, alerting)
  • Experience designing systems with reliability, scalability, and operational clarity in mind
  • Proven ability to own and deliver production systems end-to-end
  • Ability to break down ambiguous problems, ask the right questions, and execute effectively
  • Experience with Azure technologies such as: ADLS Gen2 (Blob Storage)
  • Synapse / Spark
  • Azure Data Explorer (ADX)
  • Experience with orchestration frameworks (e.g., Airflow)
  • Experience with infrastructure-as-code (Bicep, ARM, Terraform, or similar)
  • Familiarity with data platform concepts (data pipelines, schema evolution, data quality, etc.)
  • Experience working on systems handling terabyte to petabyte-scale data
  • Exposure to privacy, compliance, and secure data handling practices

Nice to have

  • Strong programming experience in Python
  • Experience building and operating large-scale distributed systems
  • Hands-on experience with: Backend services or APIs (e.g., FastAPI, Flask, or similar)
  • Cloud-based infrastructure (Azure, AWS, or GCP)
  • Monitoring and observability systems (metrics, logging, alerting)
  • Experience designing systems with reliability, scalability, and operational clarity in mind
  • Proven ability to own and deliver production systems end-to-end
  • Ability to break down ambiguous problems, ask the right questions, and execute effectively
  • Experience with Azure technologies such as: ADLS Gen2 (Blob Storage)
  • Synapse / Spark
  • Azure Data Explorer (ADX)
  • Experience with orchestration frameworks (e.g., Airflow)
  • Experience with infrastructure-as-code (Bicep, ARM, Terraform, or similar)
  • Familiarity with data platform concepts (data pipelines, schema evolution, data quality, etc.)
  • Experience working on systems handling terabyte to petabyte-scale data
  • Exposure to privacy, compliance, and secure data handling practices

What we offer

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Software Engineer - Data Platform, AI Infrastructure

8 matching positions

Senior Software Engineer, AI Data Platform (CoreAI)

Join Microsoft’s CoreAI – AI Platform team in Bay Area/Redmond to build the AI D...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years
Job Responsibility
Job Responsibility
  • Design and build scalable data pipelines and services to automate the dataset lifecycle (ingestion, registration, validation, PII handling, discovery, sharing, lineage), including intelligent agent-driven automation for key stages
  • Develop secure and reliable infrastructure for data access, entitlement management, and operational support across global time zones
  • Implement governance and compliance tooling to ensure data integrity, auditability, and adherence to regulatory standards
  • Create user-facing tools and APIs that make datasets easily discoverable and reusable
  • Contribute to strategic extensions such as continuous feedback loops, human-in-the-loop workflows, and data intelligence services for internal and external stakeholders
  • Collaborate with cross-org partners to align priorities and deliver company-wide impact
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Infrastructure & AI

Fullstory Anywhere is one of Fullstory's three primary product verticals, and it...
Location
Location
United States , Atlanta
Salary
Salary:
160000.00 - 170000.00 USD / Year
fullstory.com Logo
Fullstory
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant experience building and operating high-throughput data pipelines (batch and/or streaming) in a major cloud platform, including work with cloud data warehouses like BigQuery, Snowflake, or Databricks.
  • Proficiency in Go, Python, Java or a similar language.
  • Hands-on experience with data transformation tooling such as dbt, with a strong understanding of data modeling and pipeline observability.
  • Familiarity with LLM integration patterns and evaluation approaches (e.g., LangSmith, Vertex AI, or comparable frameworks), or demonstrated ability to ramp quickly in applied AI.
  • A track record of owning major system areas end-to-end: driving architectural decisions, maintaining production health, and improving reliability over time.
Job Responsibility
Job Responsibility
  • Maintain, extend, and scale Go microservices that transform and deliver Fullstory session data into customer warehouses and power the team's MCP server that enables AI agent integrations.
  • Develop and maintain dbt models and pipeline orchestration to ensure timely, fault-tolerant data migrations across hundreds of customer destinations.
  • Define evaluation frameworks for LLM outputs using tools like Langsmith and Vertex AI, ensuring AI-powered customer agents produce accurate, useful results.
  • Investigate and resolve production incidents across the data pipeline, implementing systemic fixes that prevent entire classes of failure from recurring.
  • Write technical design documents that drive consensus on architectural changes, proactively surfacing scaling bottlenecks, edge cases, and cross-team dependencies.
  • Demonstrate sound technical judgment by de-risking work through spikes, taking on tech debt deliberately, and knowing when to escalate versus dig in.
What we offer
What we offer
  • Flexibility and Connection
  • flexible PTO policy
  • annual company-wide closure
  • Benefits
  • paid parental leave
  • Bereavement leave, including miscarriage/pregnancy loss
  • Learning opportunities
  • annual learning subsidy
  • Productivity support
  • monthly productivity stipend
  • Fulltime
Read More
Arrow Right

Senior Data Engineer - AI Infrastructure

We are building a large-scale data platform that transforms raw system logs into...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Design and implement large-scale data pipelines using PySpark and distributed processing frameworks
  • Build and maintain data models that accurately represent underlying system behavior and business logic
  • Ensure high standards of data correctness, completeness, and consistency across datasets
  • Develop validation, monitoring, and alerting mechanisms to detect data quality issues
  • Partner with data scientists to support experimentation and analytics use cases
  • Collaborate with platform engineers to ensure efficient data ingestion, processing, and storage
  • Optimize pipelines for performance, scalability, and cost efficiency
  • Define and enforce best practices for schema design, data transformations, and pipeline reliability
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, AI Platform

GoodLeap is a technology company delivering best-in-class financing and software...
Location
Location
United States , AUSTIN; SAN FRANCISCO; IRVINE; ROSEVILLE
Salary
Salary:
173000.00 - 200000.00 USD / Year
goodleap.com Logo
GoodLeap
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience building and shipping scalable, robust backend services and APIs
  • Strong proficiency in Python and/or TypeScript
  • Solid understanding of distributed systems, service-oriented architecture, and event-driven patterns (e.g. Kafka, RabbitMQ, SQS)
  • Passion for software development, emerging technologies and culture of innovation
  • A collaborative mindset and interest in mentoring teammates and elevating team practices
  • Excellent communication and interpersonal skills
Job Responsibility
Job Responsibility
  • Build features and extensions to our agentic AI platform using scalable, robust, and AI-first software engineering practices
  • Design tools and infrastructure to enable teams at GoodLeap to easily build and enhance AI agents that empower homeowners, contractors, and operations staff
  • Work alongside a team of AI engineers, product managers, and data scientists to evaluate and improve our agent ecosystem
  • Collaborate with Staff engineers, product, architecture, and design leads to deliver highly-available, fault-tolerant products and services
  • Work on significant and unique technical challenges, evaluate and recommend solutions, and guide decision making by considering technical tradeoffs
  • Grasp both the technical and business perspective so you can help drive innovation
  • Work autonomously and be self-disciplined, requiring minimal supervision or guidance
  • Collaborate with other team members and coach more junior team members to grow both their technical skills and soft skills
What we offer
What we offer
  • May be eligible for a bonus and equity
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data Platform

Help build the future of data by creating the technical "nervous system" of the ...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
multiverse.io Logo
Multiverse
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant experience in commercial software (Python, TypeScript, and Go)
  • Passion for modular, readable code
  • Understanding of how to treat "data as software"
  • Comfortable with cloud-native tools (AWS/Azure, Kubernetes)
  • Ability to explain complex architectural choices to both product managers and engineers
Job Responsibility
Job Responsibility
  • Architect for growth: define and refactor data models to ensure they stay fast and clear as we scale
  • Create a universal data layer: build GraphQL APIs and connectors so anyone can access data safely
  • Enable GenAI: build infrastructure for Vector Databases and AI-driven automation
  • Automate safety: implement "Privacy by Design" by automating security checks
What we offer
What we offer
  • 27 days holiday
  • 5 additional days off: 1 life event day, 2 volunteer days, 2 company-wide wellbeing days
  • 8 bank holidays per year
  • Private medical Insurance with Bupa
  • Medical cashback scheme
  • Life insurance
  • Gym membership & wellness resources through Wellhub
  • Access to Spill - mental health support
  • Work-from-anywhere scheme - up to 10 days per year
  • Kitchen that's always stocked
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, AI Data

We're seeking an exceptional Senior Software Engineer to join our AI Data team. ...
Location
Location
Salary
Salary:
Not provided
assemblyai.com Logo
AssemblyAI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of professional software engineering experience
  • Strong proficiency in Python and SQL with demonstrated ability to write production-quality code
  • Solid understanding of software engineering fundamentals: Data structures and algorithms
  • System design and architectural patterns
  • Testing strategies (unit, integration, end-to-end)
  • Code review practices and technical collaboration
  • Experience with: RESTful APIs and distributed systems concepts
  • Containerization (Docker) and basic cloud infrastructure
  • Track record of delivering high-quality software in a team environment
  • Ability to thrive in a startup environment with changing priorities and rapid iteration
Job Responsibility
Job Responsibility
  • Architect Next-Gen AI Data Infrastructure
  • Design scalable, future-proof data platforms optimized for AI research workloads
  • Build efficient self-serve data processing pipelines leveraging GCP's advanced services
  • Implement cost-effective storage and monitoring solutions for ML at scale
  • Create flexible training resource management with intelligent queuing
  • Optimize resource allocation for maximum training efficiency
  • Participate in on-call rotation to ensure system reliability
  • Advance Technical Excellence
  • Lead adoption of cutting-edge ML tools and frameworks, continuously evaluating and integrating best-in-class solutions
  • Streamline existing workflows while introducing new tooling that further reduces complexity
What we offer
What we offer
  • competitive equity grants
  • 100% employer-paid benefits
  • the flexibility of being fully remote
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - AI Platform (Michelangelo)

This role is part of Uber’s ML Serving team within the AI Platform, responsible ...
Location
Location
United States , Seattle, Washington; Sunnyvale, California
Salary
Salary:
202000.00 - 224000.00 USD / Year
uber.com Logo
Uber
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS or MS in Computer Science or a related technical discipline, or equivalent experience
  • 5+ years of full-time engineering experience
  • Significant experience building production-grade backend or distributed systems using Java, Go, Python, or C++
  • Proven ability to design, ship, and operate scalable services end to end
  • Strong foundation in system design, data structures, and algorithms
Job Responsibility
Job Responsibility
  • Design, build, and own scalable ML serving services and infrastructure components
  • Drive technical design decisions and lead implementation of complex systems
  • Partner with ML engineers and platform teams to productionize ML and generative AI models
  • Improve system reliability, performance, and operational excellence through automation and tooling
  • Mentor junior engineers and contribute to team-wide engineering best practices
What we offer
What we offer
  • Eligible to participate in Uber's bonus program
  • May be offered an equity award & other types of comp
  • All full-time employees are eligible to participate in a 401(k) plan
  • Eligible for various benefits
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, AI Platform and Enablement

We're building a next-generation AI-powered platform and web application for cre...
Location
Location
United States , San Francisco
Salary
Salary:
180000.00 - 286000.00 USD / Year
descript.com Logo
Descript
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience in deploying and managing AI models in production
  • Experience with the tools of large volume data pipelines like spark, flume, dask, etc.
  • Familiarity with cloud platforms (AWS, Google Cloud, Azure) and container technologies (Docker, Kubernetes)
  • Knowledge of DevOps and MLOps best practices
  • Strong problem-solving abilities and excellent communication skills
Job Responsibility
Job Responsibility
  • Build, maintain, and standardize third-party model integrations, including consulting for other engineering teams with AI model integration needs
  • Design, implement, and maintain our AI infrastructure supporting our machine learning life cycle, including data ingestion pipelines, training developer experience and infrastructure, evaluation frameworks, and deployments / GPU infrastructure
  • Collaborate with Product Managers, Research Engineers, and AI Researchers to understand their infrastructure needs and ensure our AI systems are robust, scalable, and efficient
  • Optimize and scale our models and algorithms for efficient inference
  • Deploy, monitor, and manage AI models in production
What we offer
What we offer
  • Generous healthcare package
  • 401k matching program
  • Catered lunches
  • Flexible vacation time
  • Fulltime
Read More
Arrow Right