Data Engineer Job at Whitehall Resources Ltd (Shropshire)

Senior Data Engineer – Data Engineering & AI Platforms

We are looking for a highly skilled Senior Data Engineer (L2) who can design, bu...

Location

India , Chennai, Madurai, Coimbatore

Salary:

Not provided

OptiSol Business Solutions

Expiration Date

Until further notice

Requirements

Strong hands-on expertise in cloud ecosystems (Azure / AWS / GCP)
Excellent Python programming skills with data engineering libraries and frameworks
Advanced SQL capabilities including window functions, CTEs, and performance tuning
Solid understanding of distributed processing using Spark/PySpark
Experience designing and implementing scalable ETL/ELT workflows
Good understanding of data modeling concepts (dimensional, star, snowflake)
Familiarity with GenAI/LLM-based integration for data workflows
Experience working with Git, CI/CD, and Agile delivery frameworks
Strong communication skills for interacting with clients, stakeholders, and internal teams

Job Responsibility

Design, build, and maintain scalable ETL/ELT pipelines across cloud and big data platforms
Contribute to architectural discussions by translating business needs into data solutions spanning ingestion, transformation, and consumption layers
Work closely with solutioning and pre-sales teams for technical evaluations and client-facing discussions
Lead squads of L0/L1 engineers—ensuring delivery quality, mentoring, and guiding career growth
Develop cloud-native data engineering solutions using Python, SQL, PySpark, and modern data frameworks
Ensure data reliability, performance, and maintainability across the pipeline lifecycle—from development to deployment
Support long-term ODC/T&M projects by demonstrating expertise during technical discussions and interviews
Integrate emerging GenAI tools where applicable to enhance data enrichment, automation, and transformations

What we offer

Opportunity to work at the intersection of Data Engineering, Cloud, and Generative AI
Hands-on exposure to modern data stacks and emerging AI technologies
Collaboration with experts across Data, AI/ML, and cloud practices
Access to structured learning, certifications, and leadership mentoring
Competitive compensation with fast-track career growth and visibility

Fulltime

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...

Location

United Arab Emirates , Dubai

Salary:

Not provided

NorthBay

Expiration Date

Until further notice

Requirements

8+ years of experience in data engineering and data platform development
Strong hands-on experience with: AWS Glue
Amazon EMR (Spark)
AWS Lambda
Apache Airflow (MWAA)
Amazon EC2
Amazon CloudWatch
Amazon Redshift
Amazon DynamoDB
AWS DataZone

Job Responsibility

Design, develop, and optimize scalable data pipelines using AWS native services
Lead the implementation of batch and near-real-time data processing solutions
Architect and manage data ingestion, transformation, and storage layers
Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
Orchestrate complex data workflows using Apache Airflow (MWAA)
Develop and manage serverless data processing using AWS Lambda
Design and optimize data warehouses using Amazon Redshift
Implement and manage NoSQL data models using Amazon DynamoDB
Utilize AWS DataZone for data governance, cataloging, and access management
Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch

Fulltime

Software Engineer (Data Engineering)

We are seeking a Software Engineer (Data Engineering) who can seamlessly integra...

Location

India , Hyderabad

Salary:

Not provided

NStarX

Expiration Date

Until further notice

Requirements

4+ years in Data Engineering and AI/ML roles
Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field
Python, SQL, Bash, PySpark, Spark SQL, boto3, pandas
Apache Spark on EMR (driver/executor model, sizing, dynamic allocation)
Amazon S3 (Parquet) with lifecycle management to Glacier
AWS Glue Catalog and Crawlers
AWS Step Functions, AWS Lambda, Amazon EventBridge
CloudWatch Logs and Metrics, Kinesis Data Firehose (or Kafka/MSK)
Amazon Redshift and Redshift Spectrum
IAM (least privilege), Secrets Manager, SSM

Job Responsibility

Design, build, and maintain scalable ETL and ELT pipelines for large-scale data processing
Develop and optimize data architectures supporting analytics and ML workflows
Ensure data integrity, security, and compliance with organizational and industry standards
Collaborate with DevOps teams to deploy and monitor data pipelines in production environments
Build predictive and prescriptive models leveraging AI and ML techniques
Develop and deploy machine learning and deep learning models using TensorFlow, PyTorch, or Scikit-learn
Perform feature engineering, statistical analysis, and data preprocessing
Continuously monitor and optimize models for accuracy and scalability
Integrate AI-driven insights into business processes and strategies
Serve as the technical liaison between NStarX and client teams

What we offer

Competitive salary and performance-based incentives
Opportunity to work on cutting-edge AI and ML projects
Exposure to global clients and international project delivery
Continuous learning and professional development opportunities
Competitive base + commission
Fast growth into leadership roles

Fulltime

Senior Software Engineer, Data Engineering

Join us in building the future of finance. Our mission is to democratize finance...

Location

United States , Menlo Park

Salary:

146000.00 - 198000.00 USD / Year

Robinhood

Expiration Date

Until further notice

Requirements

5+ years of professional experience building end-to-end data pipelines
Hands-on software engineering experience, with the ability to write production-level code in Python for user-facing applications, services, or systems (not just data scripting or automation)
Expert at building and maintaining large-scale data pipelines using open source frameworks (Spark, Flink, etc)
Strong SQL (Presto, Spark SQL, etc) skills
Experience solving problems across the data stack (Data Infrastructure, Analytics and Visualization platforms)
Expert collaborator with the ability to democratize data through actionable insights and solutions

Job Responsibility

Help define and build key datasets across all Robinhood product areas. Lead the evolution of these datasets as use cases grow
Build scalable data pipelines using Python, Spark and Airflow to move data from different applications into our data lake
Partner with upstream engineering teams to enhance data generation patterns
Partner with data consumers across Robinhood to understand consumption patterns and design intuitive data models
Ideate and contribute to shared data engineering tooling and standards
Define and promote data engineering best practices across the company

What we offer

Market competitive and pay equity-focused compensation structure
100% paid health insurance for employees with 90% coverage for dependents
Annual lifestyle wallet for personal wellness, learning and development, and more
Lifetime maximum benefit for family forming and fertility benefits
Dedicated mental health support for employees and eligible dependents
Generous time away including company holidays, paid time off, sick time, parental leave, and more
Lively office environment with catered meals, fully stocked kitchens, and geo-specific commuter benefits
Bonus opportunities
Equity

Fulltime

Software Engineer, Data Engineering

Join us in building the future of finance. Our mission is to democratize finance...

Location

Canada , Toronto

Salary:

124000.00 - 145000.00 CAD / Year

Robinhood

Expiration Date

Until further notice

Requirements

3+ years of professional experience building end-to-end data pipelines
Hands-on software engineering experience, with the ability to write production-level code in Python for user-facing applications, services, or systems (not just data scripting or automation)
Expert at building and maintaining large-scale data pipelines using open source frameworks (Spark, Flink, etc)
Strong SQL (Presto, Spark SQL, etc) skills
Experience solving problems across the data stack (Data Infrastructure, Analytics and Visualization platforms)
Expert collaborator with the ability to democratize data through actionable insights and solutions

Job Responsibility

Help define and build key datasets across all Robinhood product areas. Lead the evolution of these datasets as use cases grow
Build scalable data pipelines using Python, Spark and Airflow to move data from different applications into our data lake
Partner with upstream engineering teams to enhance data generation patterns
Partner with data consumers across Robinhood to understand consumption patterns and design intuitive data models
Ideate and contribute to shared data engineering tooling and standards
Define and promote data engineering best practices across the company

What we offer

bonus opportunities
equity
benefits

Fulltime

Senior Data Engineer

Atlassian is looking for a Senior Data Engineer to join their product DE team. T...

Location

United States , Seattle; San Francisco; Mountain View

Salary:

135600.00 - 217800.00 USD / Year

Atlassian

Expiration Date

Until further notice

Requirements

Partner across engineering teams to tackle company-wide initiatives
Mentor junior members of the team
Partner with leadership, engineers, program managers and data scientists to understand data needs
Apply expertise and build scalable data solution
Develop and launch efficient & reliable data pipelines to move and transform data (both large and small amounts)
Intelligently design data models for storage and retrieval
Deploy data quality checks to ensure high quality of data
Ownership of the end-to-end data engineering component of the solution
Support on-call shift to support the team
Design and develop new systems in partnership with software engineers to enable quick and easy consumption of data

Job Responsibility

Build top-notch data solutions and data architecture to inform our most critical strategic and real-time decisions
Help translate business needs into data requirements and identify efficiency opportunities

What we offer

Health coverage
Paid volunteer days
Wellness resources

Fulltime

Senior Data Engineer

We are looking for a Senior Data Engineer to join our product DE team and report...

Location

United States , Seattle; San Francisco; Mountain View

Salary:

135600.00 - 217800.00 USD / Year

Atlassian

Expiration Date

Until further notice

Requirements

Partner across engineering teams to tackle company-wide initiatives
Mentor junior members of the team
Partner with leadership, engineers, program managers and data scientists to understand data needs
Apply expertise and build scalable data solution
Develop and launch efficient & reliable data pipelines to move and transform data (both large and small amounts)
Intelligently design data models for storage and retrieval
Deploy data quality checks to ensure high quality of data
Ownership of the end-to-end data engineering component of the solution
Support on-call shift as needed to support the team
Design and develop new systems in partnership with software engineers to enable quick and easy consumption of data

Job Responsibility

Build top-notch data solutions and data architecture to inform our most critical strategic and real-time decisions
Help translate business needs into data requirements and identify efficiency opportunities

What we offer

Health coverage
Paid volunteer days
Wellness resources

Fulltime

Software Engineer - Data Engineering

Akuna Capital is a leading proprietary trading firm specializing in options mark...

Location

United States , Chicago

Salary:

130000.00 USD / Year

AKUNA CAPITAL

Expiration Date

Until further notice

Requirements

BS/MS/PhD in Computer Science, Engineering, Physics, Math, or equivalent technical field
5+ years of professional experience developing software applications
Java/Scala experience required
Highly motivated and willing to take ownership of high-impact projects upon arrival
Prior hands-on experience with data platforms and technologies such as Delta Lake, Spark, Kubernetes, Kafka, ClickHouse, and/or Presto/Trino
Experience building large-scale batch and streaming pipelines with strict SLA and data quality requirements
Must possess excellent communication, analytical, and problem-solving skills
Recent hands-on experience with AWS Cloud development, deployment and monitoring necessary
Demonstrated experience working on an Agile team employing software engineering best practices, such as GitOps and CI/CD, to deliver complex software projects
The ability to react quickly and accurately to rapidly changing market conditions, including the ability to quickly and accurately respond and/or solve math and coding problems are essential functions of the role

Job Responsibility

Work within a growing Data Engineering division supporting the strategic role of data at Akuna
Drive the ongoing design and expansion of our data platform across a wide variety of data sources, supporting an array of streaming, operational and research workflows
Work closely with Trading, Quant, Technology & Business Operations teams throughout the firm to identify how data is produced and consumed, helping to define and deliver high impact projects
Build and deploy batch and streaming pipelines to collect and transform our rapidly growing Big Data set within our hybrid cloud architecture utilizing Kubernetes/EKS, Kafka/MSK and Databricks/Spark
Mentor junior engineers in software and data engineering best practices
Produce clean, well-tested, and documented code with a clear design to support mission critical applications
Build automated data validation test suites that ensure that data is processed and published in accordance with well-defined Service Level Agreements (SLA’s) pertaining to data quality, data availability and data correctness
Challenge the status quo and help push our organization forward, as we grow beyond the limits of our current tech stack

What we offer

Discretionary performance bonus
Comprehensive benefits package that may encompass employer-paid medical, dental, vision, retirement contributions, paid time off, and other benefits

Fulltime

Data Engineer

Whitehall Resources Ltd

Location:
United Kingdom , Shropshire

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

Job Description:

Requirements:

Nice to have:

Additional Information:

Job Posted:
May 16, 2026

Looking for more opportunities? Search for other job offers that match your skills and interests.

Similar Jobs for Data Engineer

Senior Data Engineer – Data Engineering & AI Platforms