Data Engineer Job at Whitehall Resources Ltd (London)

Data Engineer

We are looking for a fully remote Data Engineer to support and enhance a modern ...

Location

United States , Nashville

Salary:

Not provided

Robert Half

Expiration Date

Until further notice

Requirements

Extensive experience in data engineering with strong hands-on use of Python in production environments
Solid background in SQL and data warehouse support, including writing and optimizing complex queries
Practical experience with Apache Spark and Azure Databricks for distributed data processing
Understanding of data modeling concepts and the ability to structure datasets for analytical use
Familiarity with cloud-based data integration tools, including exposure to Azure Data Factory
Working knowledge of visualization platforms such as Power BI, Tableau, or Looker
Experience with Microsoft SQL technologies and comfort operating within Azure-based data ecosystems
Exposure to C# is beneficial but not required

Job Responsibility

Oversee the daily performance, stability, and upkeep of the data warehouse to ensure dependable access to trusted data
Design, build, and refine data pipelines that move and transform information efficiently across multiple platforms
Develop engineering solutions using Python, Spark, and Databricks to support large-scale data processing and analytics workloads
Create and maintain SQL-based data structures and workflows that improve data quality, usability, and consistency
Apply sound data modeling practices to organize datasets for reporting, analysis, and downstream application use
Partner with cross-functional teams to understand integration needs and deliver scalable data solutions aligned with business priorities
Support cloud-based data movement and orchestration processes, including work involving Azure Data Factory where needed
Contribute to reporting and visualization readiness by preparing well-structured datasets for tools such as Power BI, Tableau, or Looker

What we offer

Medical, vision, dental, and life and disability insurance
401(k) plan
Free online training

Data Engineer

Join our team and you'll be responsible for building, maintaining, and optimizin...

Location

Germany , Düsseldorf

Salary:

Not provided

StepStone

Expiration Date

Until further notice

Requirements

3+ years of experience in Data Engineering or a similar role
Solid computer science fundamentals — including data structures, algorithms, distributed systems, and networking
Strong Python skills for writing clean, efficient, and maintainable data pipelines
Deep understanding of database internals, query optimization, and storage engines — with advanced SQL skills across OLAP and OLTP patterns and familiarity with NoSQL data models
Hands-on experience with cloud services and distributed platforms such as AWS Glue, AWS Lambda, Spark, or equivalent
Strong grasp of data modeling and modern architectures including Data Lakes and Data Warehouses
Practical experience with orchestration tools like Airflow or Dagster, alongside Terraform for infrastructure-as-code, CI/CD pipelines, and Git

Job Responsibility

Design, build, and continuously optimize ETL/ELT workflows that power our data landscape — from ingestion to delivery — with reliability and scale in mind
Partner with cross-functional stakeholders to shape datasets that drive informed decisions, leading data modeling, migration planning, and optimization from conception to production
Proactively monitor, validate, and improve data integrity across our pipelines using modern tooling
Own the technical documentation, data specifications, and metadata that keep our datasets accessible and understandable
Build and maintain robust integrations with marketing platforms — including LinkedIn, Google Ads, and Meta

What we offer

30 days of holidays + 1 Cultural Heritage Day + 2 extra days for Christmas and New Year's Eve
Hybrid working model
Company pension scheme
24/7 Employee Assistance Programme
Life Assurance Cover
Volunteering days
Job bike
Discounted parking or job ticket
In-house gym
In-house barista

Fulltime

Data Engineer

Join us as a Data Engineer, who is responsible for delivering the technology sta...

Location

India , Bengaluru; Pune

Salary:

Not provided

Barclays

Expiration Date

Until further notice

Requirements

Experience in ETL / data processing / data warehousing / data product management
Preferred AWS cloud development, with a strong understanding of AWS services and Python programming language
Experience in leading the high performing teams
Delivering data products on AWS
Understanding of CI/CD tools and pipelines (Jenkins, GitLab CI, etc.) for automation and deployment
Experience in Data Processing and ETL in On-prem and Cloud based systems
Strong communication skills and ability to work in a collaborative team environment
Understanding of data architecture, data governance, data security and production support

Job Responsibility

delivering the technology stack, using strong analytical and problem solving skills to understand the business requirements and deliver quality solutions coming up with the ETL process design of the Foundational / Consolidated and Business data Products for the Tesco Tenancy on EDP
To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure
Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data
Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures
Development of processing and analysis algorithms fit for the intended data complexity and volumes
Collaboration with data scientist to build and deploy machine learning models

What we offer

Competitive holiday allowance
Life assurance
Private medical care
Pension contribution

Fulltime

Data Engineer

Let’s do this. Let’s change the world. We are looking for a talented Data Engine...

Location

India , Hyderabad

Salary:

Not provided

Amgen

Expiration Date

Until further notice

Requirements

Bachelor's degree in Computer Science, Information Technology, or a related field with 5-9 years of relevant experience
Advanced Python, PySpark, and SQL (window functions, complex joins, MERGE patterns, optimization)
Hands-on Databricks and Airflow experience in enterprise environments
Experience with cloud data platforms (AWS), object storage, and secure secret handling
Strong data quality engineering, monitoring, and troubleshooting in regulated data contexts
Solid understanding of ETL orchestration, dependency management, and SLA-driven delivery

Job Responsibility

Build and operate large-scale healthcare data pipelines across batch workflows, metadata-driven ingestion, and data service publishing
Own end-to-end engineering from source ingestion to conformed data products, with strong focus on reliability, data quality, and operational observability
Partner with analytics, business, and platform teams to deliver trusted datasets for sales, claims, activity, patient, and rare disease use cases
Design and maintain PySpark/SQL pipelines in Databricks for landing, unified, unstitched, and published data layers
Build and support Airflow DAGs for scheduling, dependencies, retries, and production operations
Implement metadata/config-driven frameworks for ingestion, transformation, and rule-based processing
Develop robust data quality controls, DQ summaries, failure handling, and alerting workflows
Manage batch/process audit logs, run status tracking, release flags, and operational reporting
Integrate multi-source data (files, APIs, cloud storage, and relational systems) into governed Delta/Spark tables
Optimize pipeline performance using partitioning, parallelization, and query tuning

Fulltime

Data Engineer

Campbell Property Group (CPG), one of Australia’s leading residential constructi...

Location

Australia , Norwest, New South Wales 2153

Salary:

Not provided

DOMAINE HOMES

Expiration Date

Until further notice

Requirements

Demonstrated experience leading a data and analytics function or team (construction or property industry highly desirable)
5–10 years’ experience across data management, business intelligence, analytics and data engineering
Proven delivery of analytics, data platform and/or AI initiatives that produced measurable business value
Experience establishing data governance, quality and ethical-use practices
Strong knowledge of data management, governance and master data management practices
Data engineering, pipelines/ETL, data modelling and warehousing concepts Cloud data platform strategy and delivery (e.g. Azure or AWS) spanning data storage, pipelines, analytics and AI services
specific platform direction to be confirmed
Microsoft SQL Server is a core requirement
strong SQL and relational database skills
Machine learning, AI agents and automation concepts and their responsible application

Job Responsibility

Designing and building data pipelines, integrations and data structures
Identifying and resolving data quality, performance and integration issues
Recommending improvements to data engineering processes and tooling
Design, build, test and maintain data pipelines and ETL processes to acquire and prepare data
Monitor and optimise pipeline performance, reliability and scalability
Build integration components to consume vendor APIs and database extracts (e.g. construction, finance and CRM systems)
Implement improvements to integration processes and tools
Develop and maintain logical and physical data models for warehouse and reporting purposes
Maintain and support databases (primarily Microsoft SQL Server) that store and serve data
Identify, escalate and remediate data-quality issues

What we offer

Secure undercover parking
A welcome gift when you start
Competitive pay and flexible work options
A focus on your health and wellbeing – with multiple gym and health insurance discounts
Great retail deals – including travel, groceries, and more
Big savings on home builds, trades, suppliers and utilities
Ongoing learning through the CPG Academy, with mentoring and unlimited training
Opportunities to give back through our charity partners
Generous leave – including parental, volunteer, reservist and emergency services leave
Free wellbeing support for you and your family, including EAP and health coaching

Fulltime

Data Engineer

This role will focus on designing, developing, and supporting Databricks-based p...

Location

United States , Austin; Mountain View; Warren

Salary:

138700.00 - 206950.00 USD / Year

General Motors

Expiration Date

Until further notice

Requirements

Bachelor's degree in Computer Science, Information Systems, Data Engineering, Data Science, Engineering, or a related field
or equivalent experience
5+ years of experience as a data engineer, ETL developer, or integration engineer building production-grade data pipelines and data products
Hands-on experience with Databricks for data engineering and analytics enablement, including: Strong SQL skills in Databricks
Experience building and supporting ETL/ELT pipelines in Databricks
Experience developing pipelines using Python, notebooks, DataStage, and scalable data transformation patterns
Experience with workflow orchestration, dependency management, scheduling, monitoring, and operational support of production pipelines
Proven experience designing and implementing dimensional, layered, or medallion-style data models for analytics and operational use cases
Strong knowledge of data warehousing and ETL/ELT concepts, including how upstream design impacts downstream performance, usability, and trust in data products
Experience integrating data from enterprise applications, especially operational platforms such as ServiceNow

Job Responsibility

Design, build, and maintain scalable data pipelines in Databricks to ingest, transform, validate, and publish trusted data products for analytics, reporting, and AI use cases
Develop and support end-to-end ETL/ELT workflows using Databricks notebooks, Python, Spark, and SQL, including orchestration, parameterization, error handling, restartability, and performance optimization
Build and maintain Bronze, Silver, and Gold data products that are reusable, governed, and aligned to business and downstream consumption needs
Build and support integrations both into Databricks and between enterprise applications, including legacy and modern integration patterns such as DataStage-based workflows
Implement pipeline logic for ingestion, standardization, cleansing, enrichment, joins, aggregations, and publishing of curated data assets for downstream use
Partner with product owners, architects, source-system teams, report and analytics teams, and data consumers to translate business needs into well-defined technical solutions and trusted data products
Define and implement data transformations, semantic structures, and curated data assets that improve usability, consistency, downstream performance, and trust in the data
Apply strong data quality, validation, reconciliation, and documentation practices to ensure data products are accurate, discoverable, reliable, and production-ready
Use GitHub-based development practices for version control, code review, collaboration, and promotion of pipeline changes across environments
Support secure and compliant data delivery by implementing access controls, permissions, and governance requirements in alignment with GM policies

What we offer

health and wellbeing benefit programs
medical
dental
vision
Health Savings Account
Flexible Spending Accounts
retirement savings plan
sickness and accident benefits
life insurance
paid vacation & holidays

Fulltime

Data Engineer

We are currently seeking a Data Engineer to join our team in Bangalore, Karnātak...

Location

India , Bangalore

Salary:

Not provided

NTT DATA

Expiration Date

Until further notice

Requirements

9+ years in Data Engineering / Big Data / Platform Engineering with end to end delivery of large enterprise programs
Multi tenant database and data modeling expertise—designing tenant aware schemas and delivering efficient, robust database code and designs for websites and products
Advanced SQL across Oracle, Azure SQL Managed Instance (SQL MI), MySQL, and Snowflake
strong performance tuning and query optimization
Hands on with cloud data platforms: Azure (preferred), including big data services and integrations
Deep experience building ETL/ELT pipelines using Informatica, SnapLogic, Azure Functions, Python targeting Snowflake and other cloud data warehouses
Proven track record designing, developing, and deploying batch data processing pipelines and feature generation platforms using Azure, Python, and SQL
Strong DataOps/CI/CD for data and code: GitHub based workflows, automated database releases, and source control best practices across the SDLC
Experience with event/streaming: Azure Event Hubs, Kafka, or Amazon Kinesis
Azure platform proficiency, including: Azure Data Factory, Azure Databricks, AKS, Azure Service Bus (ASB), API Management, Storage Accounts, Event Hub/Event Grid, Redis Cache

Fulltime

New

Data Engineer

As a Data Engineer, you will join Genus and work within the Data Platform Engine...

Location

India , Bengaluru

Salary:

Not provided

Randstad

Expiration Date

September 03, 2026

Requirements

Travel Occasional Travel will be required
Education Bachelors Degree or Equivilent in Computing related subject
Licenses/Certifications Desirable: Databricks Certified Professions
Dbt Certified
Microsoft Certified
SQL / T-SQL and analytical data warehouse development - 8+ years
Apache Spark experience using Python, PySpark and SparkSQL at scale - 5+ years
Expertise on performance optimisation and production stability of the data workloads - 3+ years
Expertise on building metadata-driven, reusable data pipelines using Databricks and ADF - 3+ years
Hands-on experience working with Delta-based datasets, multi-layer Lakehouse architectures, and evolving schemas - 3+ years

Job Responsibility

Design, develop, test, and deploy advanced data engineering solutions using Azure SQL Server, Python, PySpark, Databricks, and Azure Synapse
Optimize data models and queries for performance and resource efficiency
Build scalable data processing pipelines using Python and PySpark for large-scale data transformation and analytics
Implement and manage Databricks and Azure Synapse environments for big data processing and analytics
Develop and maintain data transformation models in dbt and Azure Data Factory
Ensure proper source control and deployment practices using Git and CI/CD tools (Azure DevOps, GitHub)
Collaborate with stakeholders to understand requirements and deliver technical solutions aligned with business needs
Guarantee data accuracy and integrity through data quality checks and cleansing processes
Performance tuning of SQL queries and data processing scripts
Manage and optimize the data platform for cost and performance, including proactive monitoring and self-healing design patterns

Fulltime

Select Country

Data Engineer

Job Description

Requirements

Looking for more opportunities?

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Our AI answers in your language