CrawlJobs Logo

Principal Data Infrastructure Engineer

https://www.microsoft.com/ Logo

Microsoft Corporation

Location Icon

Location:
United States , Redmond

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

139900.00 - 274800.00 USD / Year
Save Job
Save Icon
Apply Position

Job Description:

As Microsoft continues to push the boundaries of AI, we are on the lookout for passionate individuals to work with us on the most interesting and challenging AI questions of our time. Our vision is bold and broad — to build systems that have true artificial intelligence across agents, applications, services, and infrastructure. It’s also inclusive: we aim to make AI accessible to all — consumers, businesses, developers — so that everyone can realize its benefits. We’re looking for a Member of Technical Staff - Principal Data Infrastructure Engineer. This role is a dynamic blend of Platform Engineering, DevOps/SRE, and Big Data Infrastructure Engineering, focused on enabling large-scale data and ML pipelines and intelligent systems. If you’ve architected big data platforms from the ground up and are eager to apply that expertise to consumer AI, we want to hear from you. You’ll bring: Deep technical expertise; A passion for automation and observability; Fluency in distributed systems; Creativity to design scalable solutions; And just as importantly: empathy, collaboration, and a growth mindset

Job Responsibility:

  • Architect and maintain scalable, reliable, and observable Big Data Infrastructure for mission-critical AI applications
  • Champion DevOps and SRE best practices—automated deployments, service monitoring, and incident response
  • Build a self-service big data platform that empowers data and platform engineers and researchers
  • Develop robust CI/CD pipelines and automate infrastructure provisioning using Infrastructure as Code tools (Bicep, Terraform, ARM)
  • Collaborate with Data Engineers, Data Scientists, AI Researchers, and Developers to deliver secure, seamless big data workflows
  • Lead technical design reviews and uphold a clean, secure, and well-documented codebase
  • Proactively identify and resolve bottlenecks in data pipelines and infrastructure
  • Optimize system performance across storage, compute, and analytics layers
  • Partner with Security teams to enhance system security (IAM, OAuth, Kerberos)
  • Embody and promote Microsoft’s values: Respect, Integrity, Accountability, and Inclusion

Requirements:

  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering
  • OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling, or data engineering
  • OR equivalent experience
  • 4+ years in Big Data Infrastructure, DevOps, SRE, or Platform Engineering
  • 3+ years of hands-on experience managing and scaling distributed systems—from bare-metal to cloud-native environments
  • 2+ years deploying containerized applications using Kubernetes and Helm/Kustomize
  • Solid scripting and automation skills using Python, Bash, or PowerShell
  • Proven success in CI/CD pipeline management, release automation, and production troubleshooting
  • Experience working with Databricks for scalable data processing and analytics
  • Familiarity with security practices in infrastructure environments, including IAM, OAuth, and Kerberos administration
  • Proven experience with cloud-native infrastructure across Azure, AWS, or GCP
  • Hands-on expertise with modern data platforms like Databricks
  • Deep understanding of data storage and processing technologies: Relational & NoSQL databases
  • Key-value stores
  • Spark compute engines
  • Distributed file systems (e.g., HDFS, ADLS Gen2)
  • Messaging systems (e.g., Event Hub, Kafka, RabbitMQ)
  • Capacity planning and incident management for large-scale big data systems
  • Solid collaboration history with Data Engineers, Data Scientists, ML Engineers, Networking, and Security teams
  • Familiarity with modern web stacks: TypeScript, Node.js, React, and optionally PHP
  • Exposure to agentic workflows, deep learning, or AI frameworks
  • Practical experience integrating LLMs (e.g., GPT-based models) into daily workflows—automating documentation, code generation, reviews, and operational intelligence
  • Solid grasp of prompt engineering techniques to design, optimize, and evaluate interactions with LLMs
  • Demonstrated ability to troubleshoot and resolve complex performance and scalability issues across infrastructure layers
  • Excellent interpersonal and communication skills, with a solid passion for mentorship and continuous learning
  • Experience applying LLMs to DevOps workflows, enhancing incident response, and streamlining cross-functional collaboration is a solid advantage

Additional Information:

Job Posted:
February 13, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Icon
Microsoft Corporation - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Principal Data Infrastructure Engineer

Principal Data Engineer

Atlassian is looking for a Principal Data Engineer to join our Data Engineering ...
Location
Location
United States , San Francisco
Salary
Salary:
168700.00 - 271100.00 USD / Year
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • You have 12+ years of experience in a Data Engineer role as an individual contributor
  • You have at least 2 years of experience as a tech lead for a Data Engineering team
  • You are an engineer with a track record of driving and delivering large (multi-person or multi-team) and complex efforts
  • You are a great communicator and maintain many of the essential cross-team and cross-functional relationships necessary for the team's success
  • Experience with building streaming pipelines with a micro-services architecture for low-latency analytics
  • Experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, and column stores (e.g. Redshift)
  • Experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools
  • Experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka)
  • Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team
  • Industry experience working with large-scale, high-performance data processing systems (batch and streaming) with a 'Streaming First' mindset to drive Atlassian's business growth and improve the product experience
Job Responsibility
Job Responsibility
  • Own the technical evolution of the data engineering capabilities and be responsible for ensuring solutions are being delivered incrementally, meeting outcomes, and promptly escalating risks and issues
  • Establish a deep understanding of how things work in data engineering, use this to direct and coordinate the technical aspects of work across data engineering, and systematically improve productivity across the teams
  • Maintain a high bar for operational data quality and proactively address performance, scale, complexity and security considerations
  • Drive complex decisions that can impact the work in data engineering. Set the technical direction and balance customer and business needs with long-term maintainability & scale
  • Understand and define the problem space, and architect solutions. Coordinate a team of engineers towards implementing them, unblocking them along the way if necessary
  • Lead a team of data engineers through mentoring and coaching, work closely with the engineering manager, and provide consistent feedback to help them manage and grow the team
  • Work with close counterparts in other departments as part of a multi-functional team, and build this culture in your team
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

Atlassian is looking for a Principal Data Engineer to join our Corporate Data En...
Location
Location
United States , San Francisco; Mountain View
Salary
Salary:
168700.00 - 271100.00 USD / Year
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 12+ years of experience in a Data Engineer role as an individual contributor
  • at least 2 years of experience as a tech lead for a Data Engineering team
  • track record of driving and delivering large and complex efforts
  • great communicator and maintain cross-team and cross-functional relationships
  • experience with building streaming pipelines with a micro-services architecture for low-latency analytics
  • experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, and column stores (e.g. Redshift)
  • experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools
  • experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka)
  • understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team
  • industry experience working with large-scale, high-performance data processing systems (batch and streaming) with a "Streaming First" mindset
Job Responsibility
Job Responsibility
  • help our stakeholder teams ingest data faster into our data lake
  • find ways to make our data pipelines more efficient
  • come up ideas to help instigate self-serve data engineering within the company
  • building micro-services, architecting, designing, and enabling self serve capabilities at scale to help Atlassian grow
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Senior Principal Data Engineer

Atlassian is looking for a Senior Principal Data Engineer to join the Go-To Mark...
Location
Location
United States , San Francisco
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 18+ years of experience in a Data Engineer role as an individual contributor
  • at least 7 years of experience as a tech lead for Data Engineering teams, and delivered complex, cross-team initiatives
  • built durable relationships with executives/senior leaders across Sales, Marketing, Finance, Commerce and related organizations, and understand complexities of data in these organizations
  • a track record of driving and delivering large complex, multi-team efforts
  • a great communicator and maintain many of the essential cross-team and cross-functional relationships necessary for the team's success
  • experience with building streaming pipelines with a micro-services architecture for low-latency analytics
  • experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, dbt, and column stores (e.g. Redshift)
  • experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools
  • experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka)
  • understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team
Job Responsibility
Job Responsibility
  • Help stakeholder teams ingest data faster into our data lake
  • find ways to make data pipelines more efficient
  • come up with ideas to help instigate self-serve data engineering within the company
  • building micro-services, architecting, designing, and enabling self-serve capabilities at scale to help Atlassian grow
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

Atlassian is looking for a Principal Data Engineer to join our Data Engineering ...
Location
Location
United States , San Francisco; Seattle; Austin
Salary
Salary:
168700.00 - 271100.00 USD / Year
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 12+ years of experience in a Data Engineer role as an individual contributor
  • At least 2 years of experience as a tech lead for a Data Engineering team
  • Engineer with a track record of driving and delivering large (multi-person or multi-team) and complex efforts
  • Great communicator and maintain many of the essential cross-team and cross-functional relationships necessary for the team's success
  • Experience with building streaming pipelines with a micro-services architecture for low-latency analytics
  • Experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, and column stores (e.g. Redshift)
  • Experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools
  • Experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka)
  • Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team
  • Industry experience working with large-scale, high-performance data processing systems (batch and streaming) with a "Streaming First" mindset to drive Atlassian's business growth and improve the product experience
Job Responsibility
Job Responsibility
  • Own the technical evolution of the data engineering capabilities and be responsible for ensuring solutions are being delivered incrementally, meeting outcomes, and promptly escalating risks and issues
  • Establish a deep understanding of how things work in data engineering, use this to direct and coordinate the technical aspects of work across data engineering, and systematically improve productivity across the teams
  • Maintain a high bar for operational data quality and proactively address performance, scale, complexity and security considerations
  • Drive complex decisions that can impact the work in data engineering
  • Set the technical direction and balance customer and business needs with long-term maintainability & scale
  • Understand and define the problem space, and architect solutions
  • Coordinate a team of engineers towards implementing them, unblocking them along the way if necessary
  • Lead a team of data engineers through mentoring and coaching, work closely with the engineering manager, and provide consistent feedback to help them manage and grow the team
  • Work with close counterparts in other departments as part of a multi-functional team, and build this culture in your team
What we offer
What we offer
  • Health coverage
  • Paid volunteer days
  • Wellness resources
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

We are on the lookout for a Principal Data Engineer to help define and lead the ...
Location
Location
United Kingdom
Salary
Salary:
Not provided
dotdigital.com Logo
Dotdigital
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Extensive experience delivering python-based projects in the data engineering space
  • Extensive experience working with SQL and NoSQL database technologies (e.g. SQL Server, MongoDB & Cassandra)
  • Proven experience with modern data warehousing and large-scale data processing tools (e.g. Snowflake, DBT, BiqQuery, Clickhouse)
  • Hands on experience with data orchestration tools like Airflow, Dagster or Prefect
  • Experience using cloud environments (e.g. Azure, AWS, GCP) to process, store and surface large scale data
  • Experience using Kafka or similar event-based architectures e.g. (Pub/Sub via AWS SQS, Azure EventHubs, AWS Kinesis)
  • Strong grasp of data architecture and data modelling principles for both OLAP and OLTP workloads
  • Capable in the wider software development lifecycle in terms of agile ways of working and continuous integration/deployment of data solutions
  • Experience as a lead or Principal Engineer on large-scale data initiative or product builds
  • Demonstrated ability to architect data systems and data structures for high volume, high throughput systems
Job Responsibility
Job Responsibility
  • Lead the design and implementation of scalable, secure and resilient data systems across streaming, batch and real-time use cases
  • Architect data pipelines, model and storage solutions that power analytical and product use cases
  • using primarily Python and SQL via orchestration tooling that run workloads in the cloud
  • Leverage AI to automate both data processing and engineering processes
  • Assure and drive best practices relating to data infrastructure, governance, security and observability
  • Work with technologists across multiple teams to deliver coherent features and data outcomes
  • Support the data team to help adopt data engineering principles
  • Identify, validate and promote new tools and technologies that improve the performance and stability of data services
What we offer
What we offer
  • Parental leave
  • Medical benefits
  • Paid sick leave
  • Dotdigital day
  • Share reward
  • Wellbeing reward
  • Wellbeing Days
  • Loyalty reward
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

Our Principal Data Engineers are responsible for leading and delivering strategi...
Location
Location
United Kingdom , Bristol; London; Manchester; Swansea
Salary
Salary:
100000.00 - 115000.00 GBP / Year
madetech.com Logo
Made Tech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Understanding of the issues and challenges that the public sector faces in delivering services that make the best use of data and digital capabilities, transforming legacy infrastructure, and taking an innovative and user-centric approach
  • Ability to innovate and take learnings from the commercial sector, other countries and advances in technology and apply them to UK Public Sector challenges to create tangible solutions for our clients
  • Experience building trusted advisor relationships with senior client stakeholders within the public sector.
  • Experience of building and leading high performing, consulting teams and creating the leveraged engagements to provide a cost-effective, profitable, successful client-facing delivery
  • Leadership of bids and solution shaping to produce compelling proposals that help Made Tech win new business and grow the industry
  • Experience of managing third-party partnerships and suppliers (in conjunction with Made Tech colleagues) to provide a consolidated and seamless delivery team to clients.
  • Experience in delivering complex and difficult engagements that span multiple capabilities for user-facing digital and data services in the public sector
  • Experience in identifying opportunities based on client needs and developing targeted solutions to progress the development of the opportunity
  • Experience of working with sales professionals and commercial responsibility for strategic organisational goals.
  • Experience working directly with customers and users within a technology consultancy
Job Responsibility
Job Responsibility
  • Collaborate with clients to understand their needs, provide solution advice in your role as a trusted advisor and shape solutions that leverage Made Tech's wider capabilities and credentials
  • Assess project performance as a part of the billable delivery team, Quality Assure (QA) the deliverables and outcomes, and ensure client satisfaction. Coach and mentor team members as well as providing direction to enable them to achieve their engagement outcomes and to develop their careers
  • Act as a Technical Authority of the Data & AI capability to provide oversight and ensure alignment with internal and industry best practices. Ensure engagement experience is captured and used to improve standards and contribute to Made Tech knowledge
  • Participate in business development activities, including bids and pre-sales within the account, industry and practice. Coach team members on their contributions and oversee the relevant technical aspects of the proposal submission
  • Undertake people management responsibilities, including performance reviews and professional development of your engagement and practice colleagues
  • Serve as a thought leader within Made Tech, our account engagements and the wider public sector and represent the company at industry events
What we offer
What we offer
  • 30 days of paid annual leave + bank holidays
  • Flexible Parental Leave
  • Remote Working
  • Paid counselling as well as financial and legal advice
  • Flexible benefit platform which includes a Smart Tech scheme, Cycle to work scheme, and an individual benefits allowance which you can invest in a Health care cash plan or Pension plan
  • Optional social and wellbeing calendar of events
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

A VC-backed conversational AI scale-up is expanding its engineering team and is ...
Location
Location
United States
Salary
Salary:
Not provided
weareorbis.com Logo
Orbis Consultants
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years in software development and data engineering with ownership of production-grade systems
  • Proven expertise in Spark/PySpark
  • Strong knowledge of distributed computing and modern data modeling approaches
  • Solid programming skills in Python, with an emphasis on clean, maintainable code
  • Hands-on experience with SQL and NoSQL databases (e.g., PostgreSQL, DynamoDB, Cassandra)
  • Excellent communicator who can influence and partner across teams
Job Responsibility
Job Responsibility
  • Design and evolve distributed, cloud-based data infrastructure that supports both real-time and batch processing at scale
  • Build high-performance data pipelines that power analytics, AI/ML workloads, and integrations with third-party platforms
  • Champion data reliability, quality, and observability, introducing automation and monitoring across pipelines
  • Collaborate closely with engineering, product, and AI teams to deliver data solutions for business-critical initiatives
What we offer
What we offer
  • Fully remote
  • great equity
  • the chance to join a rocketship
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

The Principal Data Engineer will lead the design, development, and optimization ...
Location
Location
United States , Washington DC; Philadelphia PA; Wilmington DE
Salary
Salary:
113200.00 - 146664.00 USD / Year
amtrak.com Logo
AMTRAK
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s Degree or equivalent combination of education, training and/or relevant experience
  • Plus 7 years of relevant work experience
  • Proficiency and hands on experience designing and implementing end-to-end data solutions in AWS (e.g., S3, EMR, Glue, Redshift, Kenesis) and/or Azure (e.g. Azure Data Factory, Synaps Analytics, Azure Data Lakre Storage)
  • Experience with some of the following technologies, Databricks, Python, Apche Sparks, IDMC, Talend, CI/CD pipelines, Jenkins, Gitlab, Power BI, SQL and NoSQL
Job Responsibility
Job Responsibility
  • Architect and Design: Define and implement the architectural strategy for our enterprise data platform, focusing on scalability, security, and performance. Design and build robust, high-volume, and performant data pipelines using cloud-native services like AWS Data Pipelines (Glue, EMR, S3, Redshift, etc.) or Azure Data Factory/Synapse Analytics
  • Technical Leadership: Act as a subject matter expert and mentor for junior and mid-level data engineers, setting best practices for code quality, testing, and deployment
  • Data Governance & Management: Utilize tools like IDMC (Informatica Data Management Cloud) for data integration, quality, governance, and cataloging across the enterprise
  • Data Processing and Analysis: Develop, optimize, and manage large-scale data processing jobs using Databricks(Spark/Delta Lake) for ETL/ELT workflows and advanced analytics
  • Coding and Scripting: Write high-quality, efficient, and well-documented code primarily in Python for data manipulation, automation, and pipeline orchestration
  • Deployment and Automation: Implement and maintain robust CI/CD pipelines and infrastructure-as-code (e.g., Terraform/CloudFormation) for automated deployment and management of data solutions
  • Business Intelligence: Ensure data readiness for reporting and analytics and possess working knowledge/experience with BI tools like Tableau and PowerBI to facilitate data consumption
  • Performance and Optimization: Monitor, troubleshoot, and tune existing data infrastructure and pipelines to ensure optimal performance and cost efficiency
What we offer
What we offer
  • health, dental, and vision plans
  • health savings accounts
  • wellness programs
  • flexible spending accounts
  • 401K retirement plan with employer match
  • life insurance
  • short and long term disability insurance
  • paid time off
  • back-up care
  • adoption assistance
  • Fulltime
Read More
Arrow Right
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Canada Jobs Australia Jobs Germany Jobs Ireland Jobs Poland Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
CrawlJobs Group
ITFlashcards AllDevBlogs
© 2026 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy