CrawlJobs Logo
Mastercard Logo Mastercard · IT - Software Development

Lead Data Engineer - Data Pipelines

Czechia, Prague · Job Posted March 20, 2026
Apply Position
Job Link Share

Job Description

Economic Intelligence is a Mastercard Services program within Economic & Business Intelligence (EBI). The program delivers macro-to-micro economic and market insights powered by Mastercard data, analytics, and research. Our goal is to help customers understand economic trends, benchmark performance, forecast outcomes, and make informed strategic decisions across geographies and industries. As an engineering team, we analyze billions of transactions globally and build platforms that deliver insights into historical consumer spending, forecasts of future spending, and expert commentary on the current economic climate. Our customers span financial services, retail, and government sectors worldwide. We are actively expanding our platform capabilities and work closely with Data Scientists and Product Managers to deliver reliable, flexible, and scalable solutions.

Job Responsibility

  • Lead technical execution to deliver end-to-end capabilities from data processing through publishing and customer consumption
  • Design, build, and evolve scalable, maintainable data pipelines that deliver insights from economic data
  • Drive performance, reliability, observability, and readability in the data platform codebase
  • Own and unify the data layer across multiple products, establishing shared patterns, reusable components, and clear interfaces
  • Translate business needs into durable technical designs
  • Tackle technical debt strategically
  • Set engineering direction and standards for pipeline development and cross-team integration
  • Write, test, and review code
  • Provide technical leadership through mentorship, rigorous code reviews, and support for other engineers’ growth and delivery
  • Continuously innovate, evaluating new approaches, tools, and technologies

Requirements

  • Strong professional experience with Python building production-grade systems
  • Strong SQL skills, including writing readable and well-tuned queries for large-scale datasets
  • Experience leading design and delivery of data pipelines and/or data platform components, with attention to scalability, reliability, and maintainability
  • Familiarity with Databricks (or similar Spark-based data platforms)
  • Comfortable using LLM‑based coding tools
  • Strong foundation in object-oriented software design and engineering best practices
  • Comfortable with Linux command line and production troubleshooting
  • Clear communicator with strong written and verbal English
  • Ownership mindset: motivation, creativity, self-direction, and ability to thrive in small, collaborative teams while influencing across boundaries
  • Passion for analytical/quantitative problem solving, quality, and continuous improvement of both platform and team processes

Nice to have

  • Experience with C# or another strongly-typed object-oriented language, and ability to collaborate effectively with .NET/React engineers on integration points between pipelines and publishing/UI systems
  • Experience with workflow orchestration (Airflow)
Mastercard - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Lead Data Engineer - Data Pipelines

8 matching positions

Lead Data Engineer - Data Pipelines

Economic Intelligence is a Mastercard Services program within Economic & Busines...
Location
Location
Czechia , Prague
Salary
Salary:
Not provided
mastercard.com Logo
Mastercard
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong professional experience with Python building production-grade systems
  • Strong SQL skills, including writing readable and well-tuned queries for large-scale datasets
  • Experience leading design and delivery of data pipelines and/or data platform components, with attention to scalability, reliability, and maintainability
  • Familiarity with Databricks (or similar Spark-based data platforms)
  • Comfortable using LLM‑based coding tools, with the discipline to validate, test, and take full ownership of the generated code
  • Strong foundation in object-oriented software design and engineering best practices
  • Comfortable with Linux command line and production troubleshooting
  • Clear communicator with strong written and verbal English
  • Ownership mindset: motivation, creativity, self-direction, and ability to thrive in small, collaborative teams while influencing across boundaries
  • Passion for analytical/quantitative problem solving, quality, and continuous improvement of both platform and team processes
Job Responsibility
Job Responsibility
  • Lead technical execution to deliver end-to-end capabilities from data processing through publishing and customer consumption
  • Design, build, and evolve scalable, maintainable data pipelines that deliver insights from economic data
  • Drive performance, reliability, observability, and readability in the data platform codebase
  • Own and unify the data layer across multiple products, establishing shared patterns, reusable components, and clear interfaces
  • Translate business needs into durable technical designs
  • Tackle technical debt strategically
  • Set engineering direction and standards for pipeline development and cross-team integration
  • Write, test, and review code
  • Provide technical leadership through mentorship, rigorous code reviews, and support for other engineers’ growth and delivery
  • Continuously innovate, evaluating new approaches, tools, and technologies
  • Fulltime
Read More
Arrow Right

Data Engineer Lead (OT Data)

Data Engineer (OT Data) (Category - Engineer) Sector: Oil and Gas Location: Doha...
Location
Location
Qatar , Doha
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's in engineering, Information Systems, or a related quantitative field
  • 5+ years of proven experience in a data engineering role
  • Experience within oil and gas industry is highly preferred
  • Demonstrable experience building and operationalizing large-scale data pipelines and applications
Job Responsibility
Job Responsibility
  • Architect & Build Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines
  • Integrate Diverse Data Sources: Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians (like OSIsoft PI, Aspen InfoPlus.21), DCS, PLC, and IoT sensors
  • Cloud Data Platform Development: Implement and manage data solutions on the Microsoft Azure cloud platform, Leveraging services like Azure IoT Hub, Azure Event Hubs, and Azure Stream Analytics for real-time ingestion and processing of operational technology (OT) data
  • Data Modelling & Warehousing: Design and implement data models optimized for time-series data from industrial assets, supporting operational dashboards and real-time analytics
  • Enable Advanced AI: Build the data infrastructure to support AI/ML models for predictive maintenance, operational anomaly detection, and process optimization using real-time OT data
  • Champion Master Data Management (MDM): Design and implement MDM strategies and solutions to create a single, authoritative source of truth for critical data domains such as wells, equipment, and assets, ensuring data consistency across the enterprise
  • Ensure Data Quality & Governance: Implement robust data quality checks, validation rules, and monitoring to ensure the accuracy, consistency, and reliability of our data. Adhere to and help shape our data governance policies
  • Embrace Industry Standards: Champion and implement industry-specific data standards and models, such as the OSDU™ Data Platform, to ensure interoperability and a unified data view across the upstream lifecycle
  • Collaborate & Innovate: Work closely with a cross-functional team of geoscientists, drilling engineers, data scientists, and business analysts to understand their data needs and deliver effective solutions
  • Automate & Optimize: Identify opportunities for process automation and infrastructure optimization to improve data delivery, scalability, and cost-effectiveness
  • Fulltime
Read More
Arrow Right

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...
Location
Location
United Arab Emirates , Dubai
Salary
Salary:
Not provided
northbaysolutions.com Logo
NorthBay
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in data engineering and data platform development
  • Strong hands-on experience with: AWS Glue
  • Amazon EMR (Spark)
  • AWS Lambda
  • Apache Airflow (MWAA)
  • Amazon EC2
  • Amazon CloudWatch
  • Amazon Redshift
  • Amazon DynamoDB
  • AWS DataZone
Job Responsibility
Job Responsibility
  • Design, develop, and optimize scalable data pipelines using AWS native services
  • Lead the implementation of batch and near-real-time data processing solutions
  • Architect and manage data ingestion, transformation, and storage layers
  • Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
  • Orchestrate complex data workflows using Apache Airflow (MWAA)
  • Develop and manage serverless data processing using AWS Lambda
  • Design and optimize data warehouses using Amazon Redshift
  • Implement and manage NoSQL data models using Amazon DynamoDB
  • Utilize AWS DataZone for data governance, cataloging, and access management
  • Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch
  • Fulltime
Read More
Arrow Right
New

Lead, Data Engineer

The Lead Data Engineer provides technical leadership for the design, development...
Location
Location
Canada , Vancouver
Salary
Salary:
120000.00 - 140000.00 USD / Year
canfor.com Logo
Canfor
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Information Systems, Mathematics, or Statistics
  • 7–10+ years of experience in enterprise data warehousing, analytics, and data platform engineering
  • 7+ years of demonstrated experience designing and delivering data engineering, ELT, and streaming solutions, including technical leadership across complex initiatives
  • Proven track record designing and evolving scalable, secure, and cost-effective cloud data architectures in an enterprise environment
  • Proven experience in database design, data schema design and data modeling
  • Experience defining and governing enterprise data architecture standards, engineering frameworks, and reference patterns
  • Experience delivering ETL/ELT solutions for importing data from a wide variety of sources
  • Experience working with Microsoft Fabric or Azure Products: Microsoft Fabric, Azure Synapse, Azure Data Factory, Azure Purview, Azure DevOps, etc.
  • Experience designing and building cloud data architecture to facilitate data pipelines and analysis workflows within a cloud environment (preferably Microsoft Fabric)
  • Experience with Enterprise Resource Planning systems (preferably Oracle) is an asset
Job Responsibility
Job Responsibility
  • Lead the design and implementation of batch, real-time, and streaming data pipelines in Microsoft Fabric, while establishing engineering standards, reusable patterns, and delivery best practices
  • Design and build event-driven and near real-time data processing solutions that enable reliable, scalable data movement
  • Own the evolution of the data platform, including scalability, reliability, observability, resiliency, and cost optimization
  • Enable data platforms that support AI/ML and Generative AI use cases
  • Establish data observability, monitoring, and data quality SLAs to ensure trusted, resilient data solutions across the analytics platform
  • Establish and enforce processes and controls that maintain the security, quality, accountability, availability, and compliance of enterprise data assets
  • Partner with business and IT leaders to shape roadmaps, lead delivery planning, manage technical tradeoffs, and execute projects within time and budget constraints
  • Establish and maintain architecture, engineering, operational, and support documentation standards for the data platform
  • Translate business requirements into scalable data platform designs and guide solution implementation to support analytics and reporting delivery
  • Partner with analytics and data science teams to operationalize advanced analytics solutions
What we offer
What we offer
  • performance-based incentive plans
  • recognition programs
  • benefits
  • paid leaves
  • pension plans with base and matching contributions
  • savings options
  • robust health & well-being initiatives
  • Fulltime
Read More
Arrow Right
New

Lead Data Engineer

Lead Data Engineer | Databricks | Azure | PySpark. You are an experienced Data E...
Location
Location
Netherlands , Amsterdam
Salary
Salary:
Not provided
levy-professionals.com Logo
Levy Professionals
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Around 10 years of experience in Data Engineering
  • Extensive hands-on experience with Databricks, PySpark, and Python
  • Strong experience with Azure Data Factory, Azure DevOps, CI/CD practices
  • Automated testing frameworks
  • Software engineering best practices
  • Ability to mentor engineers
Job Responsibility
Job Responsibility
  • Design, build, and optimise scalable data pipelines using PySpark and Databricks
  • Develop and maintain high-quality Python code
  • Contribute to automated testing frameworks using pytest
  • Support and enhance CI/CD pipelines within Azure DevOps
  • Collaborate with cross-functional teams
  • Mentor junior and medior Data Engineers
  • Fulltime
Read More
Arrow Right

Data Engineer – Lead

Data Engineer – Lead
Location
Location
India , Bengaluru Urban
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on experience with Microsoft Fabric, including Lakehouse, Warehouse, OneLake, Pipelines, Dataflows Gen2, Notebooks, and Power BI integration
  • Expertise in ETL/ELT, data pipelines, distributed data processing, and cloud-scale data engineering
  • Strong SQL, Python, PySpark, and data modeling skills
  • Experience with Lakehouse, Warehouse, and Medallion Architecture
  • Understanding of Delta tables, dimensional modeling, star schema, facts, dimensions, and curated analytical datasets
  • Experience integrating structured, semi-structured, file-based, API-based, enterprise application, and cloud data sources
  • Experience with data quality, reconciliation, logging, monitoring, and error-handling frameworks
  • Experience leading technical teams and coordinating onshore/offshore delivery
  • Experience with Git, CI/CD, Azure DevOps, branching, code reviews, and release management
  • Good to Have: Experience with Azure Data Factory, Synapse, Databricks, ADLS Gen2, Azure SQL, Microsoft Purview, or related Azure services
Job Responsibility
Job Responsibility
  • Lead the design and implementation of scalable data pipelines and data processing frameworks in Microsoft Fabric
  • Define data engineering standards, development practices, naming conventions, coding guidelines, and reusable technical patterns
  • Lead implementation of Bronze, Silver, and Gold layers in the Medallion Architecture
  • Oversee ingestion, transformation, orchestration, validation, and publication of data from multiple enterprise, clinical, operational, and cloud-based sources
  • Guide development of Fabric Pipelines, Dataflows Gen2, Notebooks, Lakehouse tables, Warehouse objects, and curated datasets
  • Ensure scalability, performance, reliability, maintainability, security, monitoring, and optimization of data solutions
  • Define standards for data quality, reconciliation, logging, error handling, auditability, and lineage
  • Conduct technical design reviews, code reviews, performance reviews, and deployment readiness reviews
  • Mentor and guide data engineering teams across onshore/offshore locations
  • Collaborate with architects, platform engineers, BI teams, QA teams, AI/ML teams, functional consultants, and stakeholders
Read More
Arrow Right

Lead Data Engineer

We’re partnering with a market-leading, highly regulated organisation to recruit...
Location
Location
United Kingdom , Hampshire
Salary
Salary:
Not provided
eutopiaonline.com Logo
Eutopia
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong Data Engineering experience with ownership of data platforms or warehouses
  • Advanced SQL and Python skills
  • Experience working within Azure environments
  • Previous team leadership or mentoring experience
  • Strong understanding of data architecture, engineering best practices, and data quality
  • Comfortable working with legacy source systems and improving data reliability
  • Able to communicate effectively with both technical and business stakeholders
Job Responsibility
Job Responsibility
  • Lead and develop a team of Data Engineers and Data Analysts
  • Own the data engineering function, platform roadmap, and delivery priorities
  • Design, build, and optimise data pipelines, integrations, and warehouse solutions
  • Drive best practice across SQL, Python, data modelling, and engineering standards
  • Improve platform reliability, performance, and scalability
  • Lead data quality, monitoring, and observability initiatives
  • Work closely with senior stakeholders across the business to deliver data-driven outcomes
  • Evaluate new tools, technologies, and future platform requirements
  • Fulltime
Read More
Arrow Right

Lead Data Engineer - AI Search

We are looking for a Lead Data Engineer — AI Search (all genders) to join Valtec...
Location
Location
France , Paris
Salary
Salary:
Not provided
valtech.com Logo
Valtech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of professional software engineering experience in enterprise environments
  • Strong expertise in Google Cloud Platform (GCP) and AI ecosystems
  • Experience with AI Search, conversational AI, and generative AI integrations
  • Strong application architecture, API design, and enterprise integration experience
  • Solid understanding of cloud-native, microservices, and event-driven architectures
  • Experience with CI/CD pipelines, software quality, observability, and cloud security best practices
  • Experience working with Kubernetes and Infrastructure as Code
  • Technical leadership experience, including mentoring engineers and guiding technical decisions
  • Ability to collaborate across architecture, engineering, cloud, and data teams
  • Strong communication and stakeholder management skills
Job Responsibility
Job Responsibility
  • Define detailed technical design, components, interfaces, and integration patterns
  • Lead development teams and enforce engineering standards and best practices
  • Coordinate architecture, cloud/platform, data, QA, and business stakeholders
  • Ensure delivery quality through testing, observability, and application security practices
  • Guide teams on GCP and AI Search implementation patterns
  • Lead technical discovery, estimation, and solution design workshops
  • Identify technical risks and propose mitigation strategies early
  • Ensure scalable, reliable, and production-ready AI solutions
  • Contribute hands-on in complex and high-impact technical areas
  • Support industrialization and operationalization of AI and search use cases
What we offer
What we offer
  • Flexibility, with remote and hybrid work options (country-dependent)
  • Career advancement, with international mobility and professional development programs
  • Learning and development, with access to cutting-edge tools, training and industry experts
  • Fulltime
Read More
Arrow Right