CrawlJobs Logo
Whitehall Resources Ltd Logo Whitehall Resources Ltd · IT - Software Development

Senior Data Engineer – Informatica

United Kingdom, London · Job Posted May 29, 2026
Apply Position
Job Link Share

Job Description

Whitehall Resources currently require an experienced Data Engineer to work with a key client based in London. **Please note this role falls INSIDE IR35**

Job Responsibility

  • Engineer production grade data pipelines on AWS (EMR, S3, Lambda), using PySpark/Python and SQL, with a focus on performance, resilience, testing, and observability
  • Migrate and modernise legacy workloads (e.g., ETL jobs and reporting feeds) onto cloud native services, creating reusable components and shared frameworks
  • Support reporting & MI use cases, including transformations and data models that feed downstream tools (e.g., Power BI)
  • Own CI/CD and version control practices (e.g., Git/GitLab), review code, and enforce engineering standards
  • Coach and mentor engineers, provide technical guidance/code reviews, and contribute to architectural decisions across squads
  • Work in Agile delivery, collaborating across product, data, and platform teams using Jira/Confluence
  • translate requirements into robust engineering tasks
  • Embed security and compliance by design, aligning with BPSS/SC constraints and department data handling policies

Requirements

  • Hands on expertise in AWS & Spark: Amazon EMR, S3, Lambda
  • strong PySpark/Python and SQL for large scale batch processing
  • Data engineering at scale in government or similarly complex domains, including performance tuning and data quality management
  • CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance
  • Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows
  • APIs & integration: building/consuming data services to move and expose data safely and reliably
  • Agile ways of working with Jira/Confluence
  • clear stakeholder communication and concise technical documentation
  • Security clearance: BPSS (minimum) and SC cleared or SC clearable for UK government work.

Nice to have

  • Data warehousing & modeling (e.g., Redshift
  • dimensional modeling
  • dbt)
  • Basic Power BI familiarity to partner with BI developers and validate end to end data flows
  • AWS ecosystem depth (Athena, Redshift, EC2, CloudWatch, IAM) and event driven patterns
  • AWS Certified Cloud Practitioner (or higher), Azure AI Fundamentals (awareness of ML/AI services)
Whitehall Resources Ltd - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Data Engineer – Informatica

8 matching positions

Senior Data Engineer (Informatica PowerCenter)

Who we're looking for: We are looking for a Senior Data Engineer who brings han...
Location
Location
Sri Lanka
Salary
Salary:
Not provided
iqzsystems.com Logo
IQZ Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of hands-on Data Engineering with production pipelines on at least one major cloud
  • Proficiency in SQL and at least one scripting language: Python, Shell, or PowerShell
  • Experience with Azure Data Factory or a comparable orchestration tool (Airflow, dbt)
  • Hands-on with Databricks or Spark for large-scale transformations
  • 3+ years of Informatica PowerCenter development and administration
  • Strong understanding of data warehousing: star schema, Data Vault, incremental loads, SCD handling
Job Responsibility
Job Responsibility
  • Design, build, and maintain batch and near-real-time ETL/ELT pipelines handling high-volume data with 99%+ SLA adherence
  • Build ingestion and transformation layers across structured, semi-structured, and unstructured sources (relational DBs, flat files, REST APIs, streaming feeds)
  • Implement medallion architecture (bronze/silver/gold) for lakehouse environments, ensuring clean separation of raw, conformed, and curated data zones
  • Develop transformation logic using Spark (or equivalent) with proper testing, documentation, and lineage tracking baked in
  • Build and manage CI/CD pipelines for data workflows: automated testing, environment promotion, and deployment standardization
  • Work across one or more major cloud platforms: Azure: Data Factory, Synapse Analytics, ADLS Gen2, Databricks, Stream Analytics
  • AWS: Glue, S3, Redshift, Lambda
  • GCP: BigQuery, Dataflow, Cloud Composer
  • Optimize cloud resource usage, tune pipeline performance, and implement cost-aware design patterns
  • Apply enterprise data modeling techniques: Kimball dimensional modeling, Data Vault, and normalized schemas suited to the use case
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

Senior Data Engineer - The Senior Data Engineer is responsible for executing the...
Location
Location
Canada , Toronto
Salary
Salary:
110000.00 - 130000.00 USD / Year
fourseasons.com Logo
Four Seasons
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 5-8 years experience Data Engineering
  • with experience in Azure using Databricks
  • Bachelor’s degree Required
  • Expert in Data Engineering in Microsoft Azure Environment + Toolsets (ADF, Synapse, Data Bricks, Cosmos DB, FunctionApps, Logic Apps etc)
  • Lakehouse experience using Databricks
  • Azure platform administration
  • Data Governance Software experience
  • Microsoft Power BI delivery and support
  • Strong SQL query skills
  • Strong history in Python, Pyspark, or Scala
Job Responsibility
Job Responsibility
  • Leading the delivery of finance domain data migration projects with clear milestones, quality controls, and operational readiness
  • Designing and maintaining reusable data ingestion and transformation frameworks aligned with enterprise engineering standards
  • Ensuring data consistency, lineage, and trustworthiness across finance and enterprise data domains
  • Collaborating with business and technical stakeholders to enable analytics, forecasting, and AI readiness for finance data
  • Driving operational excellence across engineering practices, optimizing for cost, performance, and maintainability
  • Design, develop, and maintain robust batch and streaming data pipelines across Azure Data Factory, Databricks, Informatica, and SQL environments
  • Closely work with vendors, contractors and other FS teams to operationalize the data migration process for the hotels
  • Implement and oversee CI/CD workflows, Infrastructure as Code (IaC), and automated testing frameworks
  • Operationalize data quality by integrating Informatica IDQ into finance workflows
  • Apply governance standards by embedding metadata, cataloging, and lineage practices
  • Fulltime
Read More
Arrow Right

Senior Data Engineer – SC Cleared

Whitehall Resources are currently looking for a Data Engineer on a remote basis ...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
whitehallresources.com Logo
Whitehall Resources Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • MUST BE SC CLEARED
  • INSIDE IR35
  • Data engineering at scale in government or similarly complex domains, including performance tuning and data‑quality management
  • Hands‑on expertise in AWS & Spark: Amazon EMR, S3, Lambda
  • strong PySpark/Python and SQL for large‑scale batch processing
  • CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance
  • Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows
  • APIs & integration: building/consuming data services to move and expose data safely and reliably
  • Agile ways of working with Jira/Confluence
  • clear stakeholder communication and concise technical documentation
Job Responsibility
Job Responsibility
  • Data engineering at scale in government or similarly complex domains, including performance tuning and data‑quality management
  • Hands‑on expertise in AWS & Spark: Amazon EMR, S3, Lambda
  • strong PySpark/Python and SQL for large‑scale batch processing
  • CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance
  • Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows
  • APIs & integration: building/consuming data services to move and expose data safely and reliably
  • Agile ways of working with Jira/Confluence
  • clear stakeholder communication and concise technical documentation
  • Data warehousing & modelling (e.g., Redshift
  • dimensional modelling
  • Fulltime
Read More
Arrow Right

Senior Data Engineer (Informatica/Snowflake)

We’re looking for a seasoned Data Engineer to join our Data & Analytics Competen...
Location
Location
Poland
Salary
Salary:
Not provided
n-ix.com Logo
N-iX
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 4 years of exp as a Data Engineer
  • You know IICS inside and out and consider yourself an SQL expert
  • You understand cloud ecosystems (AWS, Azure, or GCP) and know your way around data modeling and warehousing concepts
  • You are currently living in Poland and can communicate effectively in English.
Job Responsibility
Job Responsibility
  • Building & Scaling: Using IICS to design and manage ETL/ELT pipelines that don't just work, but scale
  • Integrating Everything: Pulling data from APIs, cloud platforms, and various databases into a cohesive Snowflake environment
  • Problem Solving: Tuning pipelines for peak performance and setting up the kind of monitoring and error-handling that lets you sleep at night
  • Collaborating: Working closely with business and analytics teams to make sure the data we deliver is actually what they need for high-level decision-making
  • Environment Ownership: Managing the full lifecycle across dev, test, and prod, while keeping our technical docs sharp.
What we offer
What we offer
  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits.
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

The Data Engineer is accountable for developing high quality data products to su...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • First Class Degree in Engineering/Technology (4-year graduate course)
  • 9 to 11 years’ experience implementing data-intensive solutions using agile methodologies
  • Experience of relational databases and using SQL for data querying, transformation and manipulation
  • Experience of modelling data for analytical consumers
  • Ability to automate and streamline the build, test and deployment of data pipelines
  • Experience in cloud native technologies and patterns
  • A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
  • Excellent communication and problem-solving skills
  • An inclination to mentor
  • an ability to lead and deliver medium sized components independently
Job Responsibility
Job Responsibility
  • Developing and supporting scalable, extensible, and highly available data solutions
  • Deliver on critical business priorities while ensuring alignment with the wider architectural vision
  • Identify and help address potential risks in the data supply chain
  • Follow and contribute to technical standards
  • Design and develop analytical data models
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

Capstone IT is helping our client find a Senior Data Engineer who will design, b...
Location
Location
United States
Salary
Salary:
Not provided
capstonec.com Logo
Capstone IT Staffing
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of strong SQL experience (complex queries and performance tuning)
  • 3+ years of hands-on Snowflake in production
  • 3+ years of ETL experience (DataStage preferred, Informatica acceptable)
  • 1–3 years of dbt Core or dbt Cloud experience
  • 2+ years of Python focused on data processing and automation
  • 3+ years of cloud platform experience (AWS strongly preferred)
Job Responsibility
Job Responsibility
  • Design and develop complex SQL queries and optimize performance
  • Build and maintain data pipelines in Snowflake
  • Develop and manage transformations using dbt (Core or Cloud)
  • Create and maintain ETL processes using DataStage or Informatica
  • Support cloud-based data architecture (AWS preferred)
  • Use Python for data processing, scripting, and automation
  • Ensure high-quality, reliable data delivery for analytics and reporting
  • Collaborate with stakeholders to meet accelerated project timelines
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

In Cyclad, we work with top international IT companies in order to boost their p...
Location
Location
Poland , Wrocław
Salary
Salary:
140.00 - 160.00 PLN / Hour
cyclad.pl Logo
Cyclad Sp. z o.o.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Approximately 8 years of experience in data engineering or related roles
  • Proficiency with traditional data warehouse and ETL solutions, with Informatica, is considered a strong advantage
  • Hands-on experience with Python and PySpark for building and optimizing data pipelines
  • Practical exposure to Databricks for modern data engineering solutions
  • Familiarity with cloud platforms, preferably Microsoft Azure
  • Proven ability to design, build, and maintain scalable and reliable data pipelines
  • Ability to collaborate with cross-functional teams, including business stakeholders, DevOps, and infrastructure teams
  • Strong analytical and problem-solving skills with attention to detail
  • Excellent communication skills and ability to work in complex, distribute environments
  • Very good English skills
Job Responsibility
Job Responsibility
  • Lead the migration of legacy ETL solutions (primarily on-premise tools such as Informatica) to a modern Azure-based data platform
  • Analyze existing data workflows and plan migration activities to ensure performance, scalability, and minimal disruption
  • Collaborate closely with business stakeholders, DevOps, and infrastructure teams to align requirements and execution
  • Design, build, and optimize modern data pipelines using cloud-native tools and frameworks
  • Contribute to the innovation and modernization of the overall data engineering landscape
  • Ensure best practices in data reliability, performance, and maintainability
What we offer
What we offer
  • Private medical care with dental care (covering 70% of costs) + rehabilitation package. Family package option possible
  • Multisport card (also for an accompanying person)
  • Life insurance
  • Fulltime
Read More
Arrow Right

Senior Engineer, Data Warehouse

The Senior Engineer, Data Warehouse is responsible for designing, developing, te...
Location
Location
United States , Miami
Salary
Salary:
Not provided
carnival.com Logo
Carnival Cruises
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree/Master’s degree preferred in Information Technology, Computer Science, or related field or equivalent work experience
  • 5+ years Experience as a Software Engineer (Developer) required
  • 5+ years Experience serving as technical resource throughout the full software development lifecycle, from conception, architecture definition, detailed design, scoping, planning, implementation, testing to documentation, delivery, and maintenance required
  • 5+ years Experience in production support, troubleshooting complex issues, developing RCAs, and meeting SLAs required
  • 5+ years Experience with compliance and security principles and developing secure applications that meet legal and organizational requirements. (i.e. PCI, SOX, PII, etc.)
  • 5+ years Experience with monitoring and logging tools to report on application and infrastructure health required
  • 5+ years Experience with DevOps and CI/CD required
  • 5+ years Experience developing with role-based access control (RBAC), IAM (Identity and Access Management)
  • 5+ years Experience developing and implementing of third-party APIs and library integrations
  • 5+ years Experience with cloud platforms (AWS, Azure, or Google Cloud) and containerization
Job Responsibility
Job Responsibility
  • Software Development: Collaborate with team members and stakeholders to understand project goals, provide technical input, and ensure successful delivery
  • Participate in architectural design decisions to ensure scalability, performance, and reliability
  • Collaborate with Business Analyst and Software Architects to analyze user requirements and application design to create software while ensuring separate elements of the application works well in the larger program
  • Write, debug, and optimize code to implement software solutions based on technical and business requirements
  • Leverage best practices and industry standards to ensure quality code is developed according to specs and user requirements
  • Ensure solutions can be easily expanded with new feature sets and functionality on frameworks that are easily extendible
  • Create frameworks which promote ease of adoption, sharing, re-use, and interfacing with other application platforms
  • Develop unit tests, perform code reviews, and ensure the software meets quality standards
  • Provide ROM to development leads for planning purposes
  • Collaborates with Quality Engineer during all phases of testing to ensure defects /deficiencies are addressed
What we offer
What we offer
  • Health Benefits: Cost-effective medical, dental and vision plans
  • Employee Assistance Program and other mental health resources
  • Additional programs include company paid term life insurance and disability coverage
  • Financial Benefits: 401(k) plan that includes a company match
  • Employee Stock Purchase plan
  • Paid Time Off
  • Holidays – All full-time and part-time with benefits employees receive days off for 8 company-wide holidays, plus 2 additional floating holidays to be taken at the employee’s discretion
  • Vacation Time – All full-time employees at the manager and below level start with 14 days/year
  • director and above level start with 19 days/year
  • Part-time with benefits employees receive time off based on the number of hours they work, with a minimum of 84 hours/year
  • Fulltime
Read More
Arrow Right