CrawlJobs Logo
Whitehall Resources Ltd Logo Whitehall Resources Ltd · IT - Software Development

Senior Data Engineer – Informatica

United Kingdom, London · Job Posted May 29, 2026
Apply Position
Job Link Share

Job Description

Whitehall Resources currently require an experienced Data Engineer to work with a key client based in London. **Please note this role falls INSIDE IR35**

Job Responsibility

  • Engineer production grade data pipelines on AWS (EMR, S3, Lambda), using PySpark/Python and SQL, with a focus on performance, resilience, testing, and observability
  • Migrate and modernise legacy workloads (e.g., ETL jobs and reporting feeds) onto cloud native services, creating reusable components and shared frameworks
  • Support reporting & MI use cases, including transformations and data models that feed downstream tools (e.g., Power BI)
  • Own CI/CD and version control practices (e.g., Git/GitLab), review code, and enforce engineering standards
  • Coach and mentor engineers, provide technical guidance/code reviews, and contribute to architectural decisions across squads
  • Work in Agile delivery, collaborating across product, data, and platform teams using Jira/Confluence
  • translate requirements into robust engineering tasks
  • Embed security and compliance by design, aligning with BPSS/SC constraints and department data handling policies

Requirements

  • Hands on expertise in AWS & Spark: Amazon EMR, S3, Lambda
  • strong PySpark/Python and SQL for large scale batch processing
  • Data engineering at scale in government or similarly complex domains, including performance tuning and data quality management
  • CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance
  • Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows
  • APIs & integration: building/consuming data services to move and expose data safely and reliably
  • Agile ways of working with Jira/Confluence
  • clear stakeholder communication and concise technical documentation
  • Security clearance: BPSS (minimum) and SC cleared or SC clearable for UK government work.

Nice to have

  • Data warehousing & modeling (e.g., Redshift
  • dimensional modeling
  • dbt)
  • Basic Power BI familiarity to partner with BI developers and validate end to end data flows
  • AWS ecosystem depth (Athena, Redshift, EC2, CloudWatch, IAM) and event driven patterns
  • AWS Certified Cloud Practitioner (or higher), Azure AI Fundamentals (awareness of ML/AI services)
Whitehall Resources Ltd - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Data Engineer – Informatica

8 matching positions

Senior ETL Developer

Embark on a transformative journey as a Senior ETL Developer at Barclays, where ...
Location
Location
United States , Whippany
Salary
Salary:
120000.00 - 175000.00 USD / Year
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Designing and building large-scale, highly optimized data warehouses and data marts
  • Python, SQL, Informatica, and shell scripting
  • Writing and optimizing multi-layered SQL queries for performance and scalability
  • Overseeing large-scale batch processing and job scheduling using tools like Autosys
  • Supporting production environments and navigating the change management lifecycle
Job Responsibility
Job Responsibility
  • Development and delivery of high-quality software solutions by using industry aligned programming languages, frameworks, and tools. Ensuring that code is scalable, maintainable, and optimized for performance
  • Cross-functional collaboration with product managers, designers, and other engineers to define software requirements, devise solution strategies, and ensure seamless integration and alignment with business objectives
  • Collaboration with peers, participate in code reviews, and promote a culture of code quality and knowledge sharing
  • Stay informed of industry technology trends and innovations and actively contribute to the organization’s technology communities to foster a culture of technical excellence and growth
  • Adherence to secure coding practices to mitigate vulnerabilities, protect sensitive data, and ensure secure software solutions
  • Implementation of effective unit testing practices to ensure proper code design, readability, and reliability
What we offer
What we offer
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
  • Medical, dental and vision coverage
  • 401(k)
  • life insurance
  • other paid leave for qualifying circumstances
  • incentive award eligibility
  • Fulltime
Read More
Arrow Right

Senior Dataops/Cloud Data Engineer

Our client is looking for a Senior DataOps/Cloud Data Engineer for a 15 month co...
Location
Location
Canada , North York
Salary
Salary:
533.22 - 569.16 CAD / Day
https://www.randstad.com Logo
Randstad
Expiration Date
July 11, 2026
Flip Icon
Requirements
Requirements
  • Data engineering (pandas, pyspark)
  • Data lakehouse solutions such as Databricks (Delta Lake)
  • Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data
  • Experience using AWS Services such as Glue, StepFunctions, Lambda, S3
  • Extract/Transform/Load data using tools such as Informatica IDMC
Job Responsibility
Job Responsibility
  • Design, develop, test, implement, and troubleshoot: data pipelines using various tools such as Databricks, Azure Data Factory, AWS Glue
  • complex data transformation procedures
  • data models for data warehouses and lakehouses
What we offer
What we offer
  • Earn a competitive rate within the industry
  • Potential for extension
  • Fulltime
Read More
Arrow Right

Senior Principal Data Engineer

Your success is a train ride away! As we move America’s workforce toward the fut...
Location
Location
United States , Washington
Salary
Salary:
175427.00 - 215100.00 USD / Year
amtrak.com Logo
AMTRAK
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree or equivalent in Computer Science, Computer Information Systems, or a related field
  • 7 years of progressive, post-baccalaureate experience as a Senior Principal Data Engineer or any occupation related to Software Development
  • Experience building solutions using Informatica Data Quality, Informatica MDM, Erwin, and Informatica Power Center
  • Experience administering applications in UNIX, LINUX, and Windows environments
  • Experience writing and executing scripts in Perl
  • Experience implementing solutions on AWS leveraging identity resolutions, data lake, metadata, and data governance
  • Experience performing data migrations, cleansing, integration, and normalization for data warehouses
  • Experience creating customized Master Data Management (MDM) and IDD solutions
  • All positions require pre-employment background check verification and a pre-employment drug screen
Job Responsibility
Job Responsibility
  • Lead the technological advancement and architecture of data-driven solutions across multiple departments including Operations, Finance, Safety, Marketing, and IT
  • Spearhead the design, development, and delivery of end-to-end solutions on contemporary technology platforms
  • Oversee designing data ingestion, transfer, and consumption processes that are cost-effective and performance efficient
  • Act as a Subject Matter Expert (SME) for key and critical source systems to offer essential guidance and mentorship to the Development, Testing, Implementation, and Support teams
  • Adapt and enhance solutions to meet evolving requirements, ensuring alignment with business objectives
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

In Cyclad, we work with top international IT companies in order to boost their p...
Location
Location
Poland , Wrocław
Salary
Salary:
140.00 - 160.00 PLN / Hour
cyclad.pl Logo
Cyclad Sp. z o.o.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Approximately 8 years of experience in data engineering or related roles
  • Proficiency with traditional data warehouse and ETL solutions, with Informatica, is considered a strong advantage
  • Hands-on experience with Python and PySpark for building and optimizing data pipelines
  • Practical exposure to Databricks for modern data engineering solutions
  • Familiarity with cloud platforms, preferably Microsoft Azure
  • Proven ability to design, build, and maintain scalable and reliable data pipelines
  • Ability to collaborate with cross-functional teams, including business stakeholders, DevOps, and infrastructure teams
  • Strong analytical and problem-solving skills with attention to detail
  • Excellent communication skills and ability to work in complex, distribute environments
  • Very good English skills
Job Responsibility
Job Responsibility
  • Lead the migration of legacy ETL solutions (primarily on-premise tools such as Informatica) to a modern Azure-based data platform
  • Analyze existing data workflows and plan migration activities to ensure performance, scalability, and minimal disruption
  • Collaborate closely with business stakeholders, DevOps, and infrastructure teams to align requirements and execution
  • Design, build, and optimize modern data pipelines using cloud-native tools and frameworks
  • Contribute to the innovation and modernization of the overall data engineering landscape
  • Ensure best practices in data reliability, performance, and maintainability
What we offer
What we offer
  • Private medical care with dental care (covering 70% of costs) + rehabilitation package. Family package option possible
  • Multisport card (also for an accompanying person)
  • Life insurance
  • Fulltime
Read More
Arrow Right

Senior Data Engineer (Informatica PowerCenter)

Who we're looking for: We are looking for a Senior Data Engineer who brings han...
Location
Location
Sri Lanka
Salary
Salary:
Not provided
iqzsystems.com Logo
IQZ Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of hands-on Data Engineering with production pipelines on at least one major cloud
  • Proficiency in SQL and at least one scripting language: Python, Shell, or PowerShell
  • Experience with Azure Data Factory or a comparable orchestration tool (Airflow, dbt)
  • Hands-on with Databricks or Spark for large-scale transformations
  • 3+ years of Informatica PowerCenter development and administration
  • Strong understanding of data warehousing: star schema, Data Vault, incremental loads, SCD handling
Job Responsibility
Job Responsibility
  • Design, build, and maintain batch and near-real-time ETL/ELT pipelines handling high-volume data with 99%+ SLA adherence
  • Build ingestion and transformation layers across structured, semi-structured, and unstructured sources (relational DBs, flat files, REST APIs, streaming feeds)
  • Implement medallion architecture (bronze/silver/gold) for lakehouse environments, ensuring clean separation of raw, conformed, and curated data zones
  • Develop transformation logic using Spark (or equivalent) with proper testing, documentation, and lineage tracking baked in
  • Build and manage CI/CD pipelines for data workflows: automated testing, environment promotion, and deployment standardization
  • Work across one or more major cloud platforms: Azure: Data Factory, Synapse Analytics, ADLS Gen2, Databricks, Stream Analytics
  • AWS: Glue, S3, Redshift, Lambda
  • GCP: BigQuery, Dataflow, Cloud Composer
  • Optimize cloud resource usage, tune pipeline performance, and implement cost-aware design patterns
  • Apply enterprise data modeling techniques: Kimball dimensional modeling, Data Vault, and normalized schemas suited to the use case
  • Fulltime
Read More
Arrow Right

Senior Data Architect

The Senior Data Architect role requires over 10 years of experience in enterpris...
Location
Location
United States , New York
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 10+ years of enterprise-level experience in data architecture, data modeling, data transformation, data storage, data archival, and performance engineering
  • Advanced hands-on experience with Informatica or Talend for ETL/ELT development (minimum 5+ years)
  • Strong expertise with relational and cloud data platforms such as PostgreSQL, Oracle, and Snowflake (minimum 7+ years combined)
  • Extensive experience writing and tuning Advanced SQL for large-scale datasets (minimum 8+ years)
  • 5+ years of experience creating and optimizing Power BI dashboards, semantic models, and DAX expressions
  • Strong proficiency in Unix Shell Scripting for automation and data workflows (minimum 5+ years)
  • Deep understanding of enterprise data governance, data quality, metadata management, and lineage tracking
  • Excellent communication, documentation, and analytical skills required to support large banking/financial organizations
  • Bachelor’s in Computer Science, Information Systems, Engineering, or equivalent work experience
Job Responsibility
Job Responsibility
  • Lead the design and implementation of enterprise‑level data architecture for large-scale banking and financial systems
  • Develop comprehensive data models (conceptual, logical, physical) to support transactional, operational, and analytical workloads
  • Architect and optimize data transformation, data integration, and ETL/ELT pipelines using Informatica or Talend
  • Design scalable data storage, archival, retention, and retrieval solutions across PostgreSQL, Oracle, and Snowflake
  • Perform data performance engineering, including SQL optimization, indexing, and partitioning for high-volume systems
  • Create, refine, and validate complex SQL queries, scripts, and stored procedures supporting business analytics and reporting
  • Build and support enterprise dashboards by enabling Power BI semantic models and performance tuning
  • Develop and automate Unix Shell Scripts for data ingestion, scheduling, and workflow orchestration
  • Collaborate with cross-functional teams to support business, regulatory, risk, fraud, and compliance reporting needs
  • Maintain and enhance data quality, metadata, lineage, and governance standards across enterprise platforms
Read More
Arrow Right

Senior Data Engineer

DefineX is a next-generation consulting house and venture builder, helping finan...
Location
Location
Turkey , Istanbul
Salary
Salary:
Not provided
definex.com Logo
DefineX
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS, MS, or PhD in Computer Science/Engineering, Mathematics, or a related field
  • 5+ years of hands-on experience in Data Engineering, ETL development, or backend data systems
  • Strong expertise in data modeling, data architecture, and ETL design principles
  • Solid experience preparing datasets for machine learning and advanced analytics use cases
  • Proficiency in handling large-scale datasets, including data preprocessing, validation, and feature preparation
  • Hands-on experience with ETL and data integration tools (ODI and Informatica preffered)
  • Experience with Apache ecosystem technologies (Hadoop, Kafka, Hive, NiFi, etc.)
  • Experience with Apache Spark (Scala or PySpark preferred)
  • Good understanding of data structures, distributed systems, and service-oriented architectures
  • Ability to translate data into actionable insights and communicate effectively with technical and non-technical stakeholders
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable data engineering and ETL pipelines that feed machine learning use cases in banking environments
  • Develop robust data ingestion, transformation, and enrichment processes to prepare high-quality, analytics-ready datasets for ML model training, validation, and production
  • Collaborate closely with data scientists, ML engineers, and business teams to enable end-to-end ML workflows, ensuring reliable data availability and consistency
  • Build and operate batch and streaming data pipelines for processing large-scale structured and unstructured data
  • Ensure data quality, lineage, and governance across analytical and ML data layers
  • Contribute as a hands-on data developer within self-organized agile teams
  • Continuously improve data platforms and pipelines to support evolving analytical and AI-driven use cases
What we offer
What we offer
  • Growth and Development: Be part of a growing global team of professionals with training and support to help you grow
  • Every DefineXer has a Growth Coach to accelerate their growth through feedback
  • Independence and Ownership: Blur in creative and challenging business and technology transformation projects
  • Time Off: 20 vacation days per annum
  • We love to Give Back: You will get certain hours a year to volunteer and organize office volunteer programs with local NGOs
  • Health and Wellness: Competitive private health and life insurance coverage
  • Fulltime
Read More
Arrow Right

Senior Engineer - Big Data and Business Intelligence

Ready to build the future with AI? At Genpact, we don’t just keep up with techno...
Location
Location
United States , New York
Salary
Salary:
189592.00 - 199071.00 USD / Year
nytimes.com Logo
The New York Times
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master’s degree in an Engineering (all), Computer Science, Sciences, Mathematics, or a related field and 2 years of experience in the job offered, a related software engineering, programmer, or systems analyst position, or related occupation. As an alternative, a Bachelor’s degree in an Engineering (all), Computer Science, Sciences, Mathematics, or a related field and 5 years of progressively responsible post-Bachelor's experience in the job offered, a related software engineering, programmer, or systems analyst position, or related occupation is acceptable.
  • 2 years of experience with the following: Informatica suite (PowerCenter, Intelligent Cloud Services, Big Data Management)
  • Amazon Cloud Services (S3, EC2, EMR)
  • Big Data tools (MapReduce, Spark or similar)
  • Relational Databases (Oracle Exadata)
  • Programming Languages (SQL, PL/SQL)
  • NoSQL Databases (MongoDB or similar)
  • and Data Visualization Tools (Business Objects, Tableau or similar)
Job Responsibility
Job Responsibility
  • Analyze petabytes of structured and unstructured data hosted on cloud, flat files, and a variety of databases (traditional, NoSQL, columnar and in-memory) and create technology road map for data integration
  • Prepare technical design outlining detailed instructions and logical steps required for coding
  • Develop, optimize, automate, and maintain software programs, algorithms, and prototypes to handle everyday big data problems
  • Collect, parse, transform, store, and visualize large sets of data to help end customers unlock the power of data insights and data driven decision-making
  • Research and recommend software and tools that are pivotal for building impactful data products
  • Supervise, coach, and mentor junior data engineers
  • Fulltime
Read More
Arrow Right