CrawlJobs Logo

Lead Data Engineer – AI & Foundation Models

Ireland, Dublin 18 · Job Posted July 03, 2026
Apply Position
Job Link Share

Job Description

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Lead Data Engineer – AI & Foundation Models Overview Mastercard is seeking a Lead Data Engineer to design, build, and operate the data foundations that power a strategic AI program within the AI & Data organization. This role is responsible for ensuring that high‑quality, well‑governed, and scalable data is available to support foundation models, AI platforms, and downstream use cases. As a technical lead, you will own end‑to‑end data engineering delivery across the program—partnering closely with AI engineers, software engineers, and product teams to ensure data pipelines, feature assets, and analytical datasets are production‑ready, reliable, and aligned with enterprise standards. Role In this role, you will lead the development and operation of data pipelines and data products that enable AI model training, inference, and evaluation. Key responsibilities include: Lead the design and implementation of scalable data pipelines supporting AI model training, inference, and experimentation Own data ingestion, transformation, and aggregation patterns across batch and streaming workloads Partner with AI engineers to enable feature engineering, feature stores, and training datasets aligned to model requirements Ensure data pipelines meet enterprise standards for quality, availability, lineage, and governance Drive best practices for data modeling, schema management, partitioning, and performance optimization Implement robust data quality checks, validation, and monitoring to ensure trust in downstream AI systems Collaborate with platform and infrastructure teams to build pipelines on cloud‑native and distributed data processing platforms Support secure data access patterns, including environment isolation, access controls, and auditability Lead code reviews and design reviews for data engineering deliverables across the program Mentor and guide senior and mid‑level data engineers, providing technical direction and delivery oversight Contribute to program‑level planning by estimating effort, identifying dependencies, and managing delivery risks related to data availability All About You Strong experience designing and building production‑grade data pipelines in large‑scale environments Deep expertise with distributed data processing frameworks (e.g. Spark or equivalent) and SQL‑based analytics Experience working with cloud data platforms and storage technologies (AWS, Azure, or GCP) Solid understanding of data modeling, performance tuning, and cost‑efficient data architecture Experience supporting machine learning and AI workloads, including training datasets, feature engineering, and inference data flows Familiarity with data governance concepts, including lineage, data quality, access control, and auditability Strong software engineering fundamentals, including version control, testing, CI/CD, and code quality standards Ability to translate AI and product requirements into practical, scalable data solutions Experience leading technical delivery and mentoring engineers, without formal line‑management responsibility Clear, concise communicator able to collaborate effectively with engineers, data scientists, product managers, and stakeholders Bachelor’s degree or equivalent practical experience in computer science, engineering, or a related field Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must: Abide by Mastercard’s security policies and practices; Ensure the confidentiality and integrity of the information being accessed; Report any suspected information security violation or breach, and Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.

Job Responsibility

  • Lead the design and implementation of scalable data pipelines supporting AI model training, inference, and experimentation
  • Own data ingestion, transformation, and aggregation patterns across batch and streaming workloads
  • Partner with AI engineers to enable feature engineering, feature stores, and training datasets aligned to model requirements
  • Ensure data pipelines meet enterprise standards for quality, availability, lineage, and governance
  • Drive best practices for data modeling, schema management, partitioning, and performance optimization
  • Implement robust data quality checks, validation, and monitoring to ensure trust in downstream AI systems
  • Collaborate with platform and infrastructure teams to build pipelines on cloud‑native and distributed data processing platforms
  • Support secure data access patterns, including environment isolation, access controls, and auditability
  • Lead code reviews and design reviews for data engineering deliverables across the program
  • Mentor and guide senior and mid‑level data engineers, providing technical direction and delivery oversight
  • Contribute to program‑level planning by estimating effort, identifying dependencies, and managing delivery risks related to data availability

Requirements

  • Strong experience designing and building production‑grade data pipelines in large‑scale environments
  • Deep expertise with distributed data processing frameworks (e.g. Spark or equivalent) and SQL‑based analytics
  • Experience working with cloud data platforms and storage technologies (AWS, Azure, or GCP)
  • Solid understanding of data modeling, performance tuning, and cost‑efficient data architecture
  • Experience supporting machine learning and AI workloads, including training datasets, feature engineering, and inference data flows
  • Familiarity with data governance concepts, including lineage, data quality, access control, and auditability
  • Strong software engineering fundamentals, including version control, testing, CI/CD, and code quality standards
  • Ability to translate AI and product requirements into practical, scalable data solutions
  • Experience leading technical delivery and mentoring engineers, without formal line‑management responsibility
  • Clear, concise communicator able to collaborate effectively with engineers, data scientists, product managers, and stakeholders
  • Bachelor’s degree or equivalent practical experience in computer science, engineering, or a related field

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Lead Data Engineer – AI & Foundation Models

8 matching positions

Lead AI Engineer

The CNPF Data & AI organisation is looking for a Lead AI Engineering Engineer to...
Location
Location
Ireland , Dublin 18
Salary
Salary:
Not provided
mastercard.com Logo
Mastercard
Expiration Date
October 10, 2026
Flip Icon
Requirements
Requirements
  • Strong experience as a hands-on AI engineer, ML engineer, or senior software engineer working on production AI systems
  • Solid foundations in software engineering, system design, and distributed systems
  • Proven experience productionising machine learning models and operating them at scale
  • Comfortable working across data engineering, ML engineering, and applied data science tasks
  • Experience with large-scale data platforms and modern ML/AI tooling
  • Strong problem-solving skills and ability to work with ambiguous requirements
  • Ability to influence technical direction without formal people management responsibility
  • Clear communication skills and comfort collaborating across functions
  • You have built and operated AI or agentic applications that run in real production environments
  • Hands-on experience implementing agent-based or LLM-powered systems beyond simple POCs
Job Responsibility
Job Responsibility
  • Lead hands-on development of AI and agentic systems from design through production deployment
  • Build and operate ML/AI services, pipelines, and APIs using strong software engineering practices
  • Design and implement ML engineering capabilities such as model serving, monitoring, evaluation, and retraining
  • Partner with data scientists to productionise models and experiments efficiently
  • Contribute directly to data preparation, feature engineering, experimentation, and modelling when required
  • Drive technical design reviews and provide mentorship to engineers and data scientists
  • Ensure AI solutions meet Mastercard standards for performance, reliability, security, and governance
  • Collaborate closely with platform, security, and infrastructure teams to ship responsibly at scale
  • Fulltime
Read More
Arrow Right

Data engineer AI and Analytics - Vice president

We are seeking a highly skilled and forward-thinking Vice President to lead our ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of technology experience, with a significant track record in wealth management technology at leading financial institutions
  • Deep, hands-on experience managing large-scale, high-throughput OLTP databases (e.g., Oracle, PostgreSQL, SQL Server)
  • Extensive experience with analytical data platforms, data warehousing, and data lakes (e.g., Snowflake, BigQuery, Redshift, Hadoop ecosystem)
  • Hands-on experience with the data engineering aspects of AI/ML, including feature engineering, data preparation for model training, and deploying models into production pipelines. Familiarity with MLOps principles
  • Proven track record of designing, building, and operating mission-critical, real-time RESTful APIs at scale
  • Strong data modeling skills for both transactional and analytical use cases
  • Expertise with performance measurement and monitoring tools (e.g., AppDynamics, Splunk, Prometheus, Grafana)
  • Proven ability to lead, mentor, and manage a high-performing engineering team
  • A strong, demonstrable work ethic and a deep sense of ownership, with a history of seeing complex systems through from conception to long-term operational stability
  • The ability to maintain a holistic, end-to-end view of a complex data ecosystem, understanding the flow of data from real-time transaction to analytical insight
Job Responsibility
Job Responsibility
  • Unified Data Platform Ownership:Lead a team of skilled engineers responsible for the design, development, and 24/7 operation of the unified client and account data platform, encompassing both high-throughput OLTP systems and large-scale OLAP analytical stores
  • AI & Advanced Analytics Enablement:Pioneer the application of AI/ML models on core client and account data to generate actionable insights, identify patterns, and create intelligent features for advisor and client applications. Partner with data science teams to provide them with high-quality, curated data
  • Real-Time API Management:Oversee the entire lifecycle of the high-throughput, low-latency APIs that provide real-time data to advisor workstations, client-facing portals, and other critical applications, ensuring they are performant, reliable, and secure
  • Data Integrity & Scale:Take ultimate responsibility for the integrity, availability, and performance of massive volumes of data across both transactional and analytical systems
  • Measurement & Analytics:Champion a culture of measurement. Define, monitor, and report on key performance indicators (KPIs) across the entire ecosystem, from database performance and API latency to data quality and the efficacy of AI models
  • End-to-End Ecosystem View:Serve as the subject matter expert for the entire client and account data ecosystem, from the foundational OLTP databases and real-time APIs to the analytical data warehouses/lakes and the AI/ML models they feed
  • Technical Strategy:Partner with global architects and business leaders to define the technical roadmap for the core data platforms, including strategies for data replication, ETL/ELT processes, and the integration of AI/ML capabilities
  • Fulltime
Read More
Arrow Right

Lead Agentic Ai Engineer

Citi's Wholesale Technology organization is seeking an exceptional, hands-on Lea...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6–10 years of relevant experience in an AI/ML development role, Applications Development, or Systems Analysis, with a substantial and demonstrated focus on Python technologies
  • Minimum 2+ years of professional experience in software development with a focus on AI, prompt engineering, machine learning, and/or agentic AI systems
  • Proven track record as a lead developer for agentic flow design, prompt design, and testing of autonomous AI systems with deep expertise in Google ADK
  • Subject Matter Expert (SME) in at least one area of Applications Development, particularly Python application development (Django, Flask, FastAPI)
  • Python (expert-level): FastAPI, Django, Flask, asyncio, PySpark — strong fundamentals in algorithms, data structures, concurrency, and design patterns
  • Proficient in Java (Spring Boot, Spring Cloud), JavaScript/TypeScript (React, Next.js, Node.js), and SQL/data modeling
  • Experience across AWS, Azure, and GCP with Docker, Kubernetes, and CI/CD pipelines
  • Proficient in MLOps practices including model versioning, deployment, and lifecycle management
  • Strong foundation in secure API design, microservices, event-driven architecture, and distributed systems with expertise in testing, Git workflows, and performance optimization
  • Deep expertise in LLMs (OpenAI GPT, Gemini, Claude, Llama) with hands-on experience in LangChain, LangGraph, LlamaIndex, AutoGen, CrewAI, and Google ADK
Job Responsibility
Job Responsibility
  • Lead end-to-end design, development, and deployment of large-scale agentic AI solutions using Google Agent Development Kit (ADK) and frameworks such as LangChain, LangGraph
  • Architect advanced multi-agent systems (perception, reasoning, planning, execution) integrating multiple LLM providers (OpenAI, Anthropic, Google Gemini)
  • Build AI-powered capabilities using Google Gemini, Vertex AI, Agent Development Kit (ADK), Google A2UI, vector databases, RAG pipelines, semantic search, and advanced prompt and context management
  • Engineer autonomous agents incorporating planning, tool usage, memory management, and multi-step reasoning patterns
  • Develop scalable, high-performance backend services in Python (FastAPI, asyncio) with resilient APIs, event-driven designs, and microservices architectures
  • Build and maintain robust data pipelines working with SQL (Oracle, PostgreSQL) and NoSQL (MongoDB) databases
  • Implement secure REST APIs and agent interfaces with strong authentication, authorization (OAuth), and encryption best practices
  • Optimize AI agent performance, latency, and cost through prompt optimization, caching strategies, and vector index tuning
  • Provide architectural guidance for Next-Generation AI (NGAI) initiatives, ensuring adherence to CTO guidelines and platform standards
  • Develop and maintain a strategic roadmap for generative AI adoption, evaluating new models, techniques, and platforms
  • Fulltime
Read More
Arrow Right

Lead, Data Governance & AI Readiness

We are looking for a senior leader to own the Data Trust and Governance on Custo...
Location
Location
United States , New York
Salary
Salary:
185000.00 - 205500.00 USD / Year
uber.com Logo
Uber
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in data governance, data quality, metadata management, master data management, data stewardship, or related disciplines
  • 10+ years of overall professional experience across data, analytics, data platforms, data engineering, or closely related domains
  • Master's degree in computer science, engineering, information systems, or a similar field
  • Proven track record of driving cross-functional alignment across engineering, analytics, data science, product, operations, and business teams
  • Deep understanding of modern data models, critical data elements, metrics, lineage, metadata, context, semantic, and data lifecycle management
  • Experience defining and operationalizing data quality controls, observability requirements, service levels, issue management, and trust scorecards
  • Strong understanding of how data must be structured and governed for both analytics use cases and AI or agentic system consumption
  • Ability to operate independently and bring structure to ambiguous, complex problem spaces
  • Strong communication skills, with the ability to work effectively with both technical and non-technical stakeholders
Job Responsibility
Job Responsibility
  • Establish and Enforce Data Quality and Trust
  • Govern the Shared Operational Foundation for Analytics and AI
  • Data Observability, Timeliness, and Reliability
  • Master Data, Metadata, and Business Semantics
  • Metric Definition, Lineage, and Grounding to Source Facts
  • Agentic AI and Analytics Readiness Governance
  • Operational Governance Alignment and Adoption
  • Drive Data Modernization and Shift-Left Governance
  • Govern Cost, Security, and Lifecycle Management
What we offer
What we offer
  • Bonus program
  • Equity award
  • 401(k) plan
  • Various benefits (see link https://jobs.uber.com/en/benefits)
  • Fulltime
Read More
Arrow Right

Lead Agentic AI & Java Integration Engineer

We are currently seeking a Lead Agentic AI & Java Integration Engineer to join o...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of progressive software engineering experience, with a core foundation in enterprise Java and Spring Boot
  • Deep, hands-on expertise in Agentic AI frameworks (LangChain, LangGraph) and prompt engineering (structured outputs, function/tool calling)
  • Proven experience building and scaling RAG pipelines and working intimately with Vector Databases
  • Extensive experience implementing AI evaluation frameworks (e.g., DeepEval) and LLM observability metrics
  • Strong background in distributed systems, event-driven architectures, and building resilient API ecosystems
  • Demonstrated ability to translate highly ambiguous, complex business problems into secure, reliable, and compliant production-ready systems
Job Responsibility
Job Responsibility
  • Agentic System Orchestration: Design and orchestrate sophisticated AI agents using LangChain and LangGraph. Manage complex control flows, tool calling, memory management, and multi-step reasoning workflows
  • Enterprise RAG & Search: Architect robust GenAI patterns, including Retrieval-Augmented Generation (RAG), semantic search, vector database integration, and end-to-end embeddings lifecycle management
  • Java/Spring Boot Integration: Seamlessly integrate GenAI capabilities and agent workflows into large-scale, distributed, event-driven enterprise applications built on Java, Spring Boot, and microservices architectures
  • AI Guardrails & Security: Engineer robust enterprise guardrails encompassing prompt injection defenses, data privacy filters, safety constraints, explainability, and Human-in-the-Loop (HITL) approval flows
  • Evaluation & Operationalization: Define and automate rigorous GenAI evaluation pipelines using frameworks like DeepEval. Continuously monitor models for accuracy, contextual relevance, hallucinations, conceptual drift, latency, and operational cost
  • Technical Leadership & Mentorship: Serve as the domain expert for GenAI engineering standards. Mentor development teams on architectural decision-making, specifically determining when to utilize agent-based systems versus conventional deterministic services
Read More
Arrow Right

Lead Agentic AI & Java Integration Engineer

We are currently seeking a Lead Agentic AI & Java Integration Engineer to join o...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of progressive software engineering experience, with a core foundation in enterprise Java and Spring Boot
  • Deep, hands-on expertise in Agentic AI frameworks (LangChain, LangGraph) and prompt engineering (structured outputs, function/tool calling)
  • Proven experience building and scaling RAG pipelines and working intimately with Vector Databases
  • Extensive experience implementing AI evaluation frameworks (e.g., DeepEval) and LLM observability metrics
  • Strong background in distributed systems, event-driven architectures, and building resilient API ecosystems
  • Demonstrated ability to translate highly ambiguous, complex business problems into secure, reliable, and compliant production-ready systems
Job Responsibility
Job Responsibility
  • Agentic System Orchestration: Design and orchestrate sophisticated AI agents using LangChain and LangGraph. Manage complex control flows, tool calling, memory management, and multi-step reasoning workflows
  • Enterprise RAG & Search: Architect robust GenAI patterns, including Retrieval-Augmented Generation (RAG), semantic search, vector database integration, and end-to-end embeddings lifecycle management
  • Java/Spring Boot Integration: Seamlessly integrate GenAI capabilities and agent workflows into large-scale, distributed, event-driven enterprise applications built on Java, Spring Boot, and microservices architectures
  • AI Guardrails & Security: Engineer robust enterprise guardrails encompassing prompt injection defenses, data privacy filters, safety constraints, explainability, and Human-in-the-Loop (HITL) approval flows
  • Evaluation & Operationalization: Define and automate rigorous GenAI evaluation pipelines using frameworks like DeepEval. Continuously monitor models for accuracy, contextual relevance, hallucinations, conceptual drift, latency, and operational cost
  • Technical Leadership & Mentorship: Serve as the domain expert for GenAI engineering standards. Mentor development teams on architectural decision-making, specifically determining when to utilize agent-based systems versus conventional deterministic services
Read More
Arrow Right
New

Software Engineer, AI Native Development

Meta is seeking a Staff Software Engineer to help define and lead the next gener...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
meta.com Logo
Meta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of software engineering experience, including experience building developer tooling, platform infrastructure, or AI-integrated systems
  • Experience designing and shipping production systems that incorporate large language models, code generation models, or other generative AI technologies into software engineering workflows
  • Experience leading major technical initiatives end-to-end, including architecture design, cross-team coordination, staged rollout, and post-launch reliability ownership
  • Experience communicating technical decisions and trade-offs in writing to both engineering and non-engineering stakeholders, including design documents and technical proposals
  • Experience applying AI tools fluently within a software development context, with demonstrated judgment on appropriate use cases, limitations, and quality validation
Job Responsibility
Job Responsibility
  • Design and build AI-native developer tooling and automation frameworks that integrate large language models into core engineering workflows such as code generation, code review, test synthesis, and incident response
  • Lead the architecture and implementation of AI-accelerated systems that reduce iteration cycles, eliminate manual toil, and scale engineering output across product teams
  • Identify opportunities to apply generative AI and foundation models to complex software engineering problems, and drive adoption of these solutions across the broader engineering organization
  • Establish and evangelize best practices for responsible and effective AI use in software development, including guidelines for when to apply AI versus deep human expertise
  • Partner with product, infrastructure, and platform teams to embed AI-native workflows into existing development pipelines, CI/CD systems, and experimentation frameworks
  • Own the technical design and end-to-end delivery of major AI tooling initiatives, including defining service level objectives, monitoring strategies, and rollout plans
  • Instrument and analyze AI workflow performance to identify bottlenecks, measure productivity impact, and drive data-informed improvements to developer experience
  • Mentor other engineers on AI-native development patterns, judgment in AI tool selection, and techniques for building reliable AI-assisted systems
  • Contribute to engineering programs that advance the organization's AI fluency, including onboarding guides, internal knowledge sharing, and cross-team working groups
  • Proactively incorporate privacy, security, and integrity principles into AI-integrated systems, partnering with cross-functional stakeholders to ensure responsible deployment
  • Fulltime
Read More
Arrow Right
New

Ai Engineer - Automation/Robotics

Join a pioneering technology leader at the forefront of innovation, dedicated to...
Location
Location
United States , Redmond
Salary
Salary:
60.00 - 65.00 USD / Hour
aquent.com Logo
Aquent
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in a technical field such as computer science, computer engineering, or a related discipline
  • 2-4 years of professional experience in AI/ML engineering or a related field
  • A strong foundation in computer science, including robust competencies in data structures, algorithms, and software design
  • Practical experience utilizing simulation environments or working with robotic hardware
  • Demonstrated ability in in-depth troubleshooting and unit testing for both new and existing production systems
  • Proficiency in programming, coupled with expertise in problem diagnosis and resolution
  • Background in leading machine learning frameworks such as TensorFlow or PyTorch
Job Responsibility
Job Responsibility
  • Fine-tune and enhance a variety of sophisticated software implementation projects
  • Architect, design, and implement robust test automation infrastructure
  • Develop advanced training and evaluation pipelines for machine learning models
  • Analyze and refine training datasets specifically for robotic and AI applications
  • Create tools for rapid deployment of novel algorithms in both simulation and on physical hardware
  • Integrate robotic control and perception stacks within various simulation frameworks
What we offer
What we offer
  • Subsidized health, vision, and dental plans
  • paid sick leave
  • retirement plans with a match
  • free online training through Aquent Gymnasium
  • Fulltime
Read More
Arrow Right