CrawlJobs Logo
360 Resourcing Solutions Logo 360 Resourcing Solutions · IT - Software Development

Data Engineer

United Kingdom Employment contract · Job Posted May 20, 2026
Apply Position
Job Link Share

Job Description

At UK Tote Group, we're on a mission to reimagine the future of pool betting — building a modern, data-driven betting experience for millions of racing fans. Our technology powers real-time insights, supports responsible gaming, and helps us deliver trusted, customer-first products across the UK and international markets. As a Data Engineer, you'll play a key role in designing, building, and optimising the Databricks-based Lakehouse that drives real-time data, analytics, and reporting across the Tote.

Job Responsibility

  • Design, build, and optimise data pipelines using Databricks, Spark Structured Streaming, and Delta Live Tables
  • Apply the Medallion Architecture to develop Bronze, Silver, and Gold Delta tables
  • Integrate data from Kafka (MSK), AWS S3, and external APIs
  • Work closely with BI teams to enable high-performance Power BI dashboards through Databricks SQL Warehouses
  • Ensure data is governed, discoverable, and secure under Unity Catalog
  • Implement and maintain CI/CD pipelines for Databricks jobs, notebooks, and DLT workflows
  • Monitor, tune, and troubleshoot pipeline performance using Databricks metrics, CloudWatch, and AWS Cost Explorer
  • Document data models, schemas, and lineage
  • Help ensure the platform remains fully compliant with GDPR and Gambling Commission regulations
  • Champion best practices in data platform design, observability, and cost management

Requirements

  • Proven expertise in building pipelines in Databricks
  • Strong grasp of Apache Spark, whether in PySpark or Scala, including Structured Streaming
  • Experience with Kafka (MSK) and real-time data ingestion
  • Deep understanding of Delta Lake, Delta Live Tables, and the Medallion Architecture
  • Strong AWS background, particularly with services such as S3, Glue, Lambda, Batch, and IAM
  • Proficient in Python and SQL for data engineering and analytics
  • Comfortable implementing CI/CD pipelines using tools such as GitHub Actions, Azure DevOps, or Jenkins
  • Solid experience with Git, version control, and Spark performance tuning
  • Collaborative, proactive attitude
  • Ability to balance platform reliability with the pace of delivery

Nice to have

  • Experience working with streaming architectures or data governance frameworks like Unity Catalog
  • Familiarity with Power BI, Looker, or Tableau
  • Exposure to Databricks REST APIs, Airflow, or Databricks Workflows
  • Knowledge of infrastructure-as-code tools such as Terraform
  • AWS networking fundamentals
  • Cost management techniques using Photon and DBU monitoring

What we offer

  • Competitive Basic Salary
  • Discretionary Bonus Scheme
  • Company Shares Option Plan
  • Contributory pension scheme
  • Life insurance (4 x basic salary)
  • Simply Health Cash Plan
  • Holiday entitlement (33 days inclusive of bank holidays)
  • Study Support and opportunity for progression and development
  • Confidential 24/7 365 employee assistance helpline
  • Agile and collaborative office environment with free parking, fruit, biscuits, and drinks
  • Regular social events, charity events and volunteering opportunities
360 Resourcing Solutions - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Data Engineer

8 matching positions

Data Engineer

Engineer will be part of the datastore-migration Factory team that will be respo...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to troubleshoot (SQL) and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices & K8s deployment experience
  • Sophisticated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, Architectural Theory
  • Experience with Kafka, ANSI SQL, FTP, Apache Spark
  • Data formats JSON, Avro, Parquet
  • Platforms Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ
Job Responsibility
Job Responsibility
  • Pipeline migration
  • Logic & Scheduling refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Data Transfer executing the physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement acting as a technical liaison to internal clients, facilitating handoff and sign-off with data owners
  • Code Conversion translating and optimizing legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg
  • Usage analysis to deliver required data products
  • Data Reconciliation & Quality working with reconciliation frameworks to build confidence that migrated data is functionally equivalent
  • Working with other internal data management platform
  • Fulltime
Read More
Arrow Right

Data Engineer

The Data Engineer will play a crucial role in migrating data from on-prem DataLa...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to troubleshoot SQL and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices and K8s deployment experience
  • Sophisticated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, Architectural Theory
  • Kafka, ANSI SQL, FTP, Apache Spark
  • JSON, Avro, Parquet
  • Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ
Job Responsibility
Job Responsibility
  • Perform end-to-end datastore migration from on-prem DataLake to AWS LakeHouse
  • Pipeline Migration including refactoring and migrating extraction logic and job scheduling
  • Data Transfer executing physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement acting as technical liaison to internal clients
  • Consumption Pattern Migration including code conversion translating and optimizing legacy SQL and Spark-based consumption patterns for Snowflake and Iceberg
  • Usage analysis to understand usage patterns and deliver required data products
  • Data Reconciliation and Quality working with reconciliation frameworks to ensure data equivalence
  • Work with internal data management platform and learn new workflows
  • Fulltime
Read More
Arrow Right

Data Engineer

Goldman Sachs is seeking a Data Engineer to join their datastore-migration Facto...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to trouble shoot (SQL) and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices & K8s deployment experience
  • Demonstrated understanding of Temporal Data Modeling (e.g., SCD Type 2)
  • Expertise in Schema Evolution (Ref: Iceberg Apache) and enforcement strategies
  • Advanced knowledge of data partitioning and clustering
  • Balancing Normalization vs. Denormalization and the strategic use of Natural vs. Surrogate Keys
  • Good conduct and ethical decision-making
Job Responsibility
Job Responsibility
  • Engineer will be part of the datastore-migration Factory team that will be responsible to perform for the end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse
  • Pipeline Migration - Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Data Transfer - Executing the physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement - Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners to ensure migrated assets meet business requirements
  • Consumption Pattern Migration - Code Conversion: Translating and optimizing legacy SQL and Spark-based consumption patterns (raw and modeled) for compatibility with Snowflake and Iceberg
  • Usage analysis: Understand usage patterns to deliver the required data products
  • Stakeholder Engagement - Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners to ensure migrated assets meet business requirements
  • Data Reconciliation & Quality - A rigorous approach to data validation is required
  • Candidates must work with reconciliation frameworks to build confidence that migrated data is functionally equivalent to that already used within production flows
  • Engineer will also need to work with our other internal data management platform, and must have an aptitude for learning new workflows and language constructs as necessary
  • Fulltime
Read More
Arrow Right

Data Engineer

We are looking for a Data Engineer with 8-12 years of experience to design and b...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8–12 years of experience in Data Engineering
  • Strong hands on expertise in Python and SQL (Snowflake preferred)
  • Experience with REST APIs / data service integrations
  • Working knowledge of AI/ML systems, Agentic AI frameworks
Job Responsibility
Job Responsibility
  • Build and maintain scalable data pipelines using Python and SQL
  • Enable AI/ML and GenAI use cases, including data preparation and feature pipelines
  • Ensure data quality, reliability, security, and observability
  • Fulltime
Read More
Arrow Right

Data Engineer

As our new Data Engineer, you will join a collaborative and skilled organization...
Location
Location
Poland , Katowice
Salary
Salary:
Not provided
vattenfall.com Logo
Vattenfall
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong Python and SQL skills
  • Practical experience with cloud data platforms (Azure, Databricks) and with different data storage solutions, including data lakes, lake houses and analytical databases
  • Hands-on experience with streaming systems (Kafka, Spark or Flink)
  • Familiarity with Airflow, CI/CD, DevOps practices
  • Experience with Docker, Kubernetes, and Infrastructure-as-Code
  • Understanding of data contracts, time-series data, and event-driven systems
  • Fluency in English and interest in the energy domain
Job Responsibility
Job Responsibility
  • Design, build, and operate batch and real-time data pipelines using Azure, Databricks, and Python
  • Ingest and process external and internal data such as weather data, market prices, grid telemetry, and event streams
  • Work with streaming technologies including Kafka, Spark Structured Streaming, and Flink
  • Ensure high data quality through validation, monitoring, and lineage
  • Develop and improve data workflows, standards, and reliability in production systems
What we offer
What we offer
  • Good remuneration
  • challenging and international work environment
  • possibility to work with some of the best in the field
  • opportunities for personal and professional development
  • Fulltime
Read More
Arrow Right

Data Engineer

We are looking for a Data Engineer to join a team focused on building reliable, ...
Location
Location
United States , Camden
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hands-on experience with Azure Data Factory, Azure Databricks, and Azure Data Lake in a data engineering environment
  • Strong programming ability in Python and PySpark for large-scale data processing and transformation
  • Proficiency in SQL, including writing and optimizing queries for analytics and data integration workloads
  • Experience building and maintaining ETL or ELT pipelines that combine data from multiple structured and semi-structured sources
  • Familiarity with data modeling concepts, curated dataset design, and preparation of data for BI or analytics consumption
  • Understanding of data governance, lineage, and security practices within modern cloud data platforms
  • Experience using GitHub or similar tools for source control, code review, and deployment automation
  • Ability to communicate effectively with business partners and translate functional needs into scalable technical solutions
Job Responsibility
Job Responsibility
  • Create and support scalable data ingestion and transformation workflows using Azure Data Factory, Databricks, and PySpark
  • Connect and consolidate data from enterprise platforms, operational databases, telematics feeds, APIs, and other internal or external sources
  • Structure and manage data within Azure Data Lake and lakehouse environments to support performance, accessibility, and long-term maintainability
  • Design curated datasets, data models, and schemas that improve usability for analytics, business intelligence, and downstream reporting
  • Apply governance and lineage practices through Unity Catalog while promoting strong data quality, consistency, and security standards
  • Work with business stakeholders and cross-functional teams to gather requirements, define technical specifications, and deliver data solutions aligned with operational needs
  • Improve pipeline stability and efficiency by troubleshooting failures, resolving performance issues, and refining storage and query strategies
  • Support Power BI reporting by preparing datasets, assisting with model improvements, and helping maintain reporting standards and governance practices
  • Use GitHub-based development practices for version control, peer review, CI/CD, and disciplined deployment processes
  • Mentor less-experienced engineers and contribute to a collaborative environment focused on continuous improvement and dependable delivery
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • enrollment in company 401(k) plan
  • Fulltime
Read More
Arrow Right

Data Engineer

We are looking for an experienced Data Engineer to design and support data excha...
Location
Location
United States , Asheboro
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience working in a manufacturing environment
  • 7+ years of experience in data engineering, data integration, or ETL-focused roles
  • Proven background working with data transformation, analytics, and system-to-system integrations using APIs, EDI, X12, XML, or similar structured formats
  • Hands-on experience with SQL in support of data movement, validation, and troubleshooting
  • Familiarity with database platforms such as SQL Server and Oracle, including querying and supporting downstream BI solutions
  • Understanding of supply chain operations within a manufacturing environment, with the ability to map business events into technical data processes
  • Experience using Microsoft Excel for data analysis, validation, or reconciliation tasks
Job Responsibility
Job Responsibility
  • Build and maintain business-to-business data interfaces that onboard new partner organizations and align incoming data with internal database structures
  • Develop automated workflows that ingest, transform, validate, and deliver data using file-based exchanges, APIs, and structured transaction formats such as EDI and X12
  • Configure and manage end-to-end integration processes across system interfaces, including flat-file handling, file sharing, and reporting-related data movement
  • Lead data transformation efforts through the full lifecycle by designing solutions, testing functionality, deploying processes, and stabilizing production performance
  • Investigate integration failures or data quality issues, identify root causes, and implement corrective actions to restore reliable processing
  • Partner with business intelligence and reporting teams to provide access to accurate, usable data sources that support analysis and operational decision-making
  • Apply manufacturing and supply chain process knowledge to structure data flows that support purchasing, components, orders, and assembly-related transactions
  • Use available tools and platforms to execute integration projects independently, including extracting data from enterprise applications and translating it into usable formats
  • Create scalable data pipelines that enable customer and order transactions to move through systems with minimal manual intervention
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • enrollment in company 401(k) plan
  • Fulltime
Read More
Arrow Right

Data Engineer

Join us as a Data Engineer at Barclays, where you will spearhead the evolution o...
Location
Location
India , Pune
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • AWS hands on with S3, EC2, Lambda, RDS, Glue, Athena, data pipelines, step function and knowledge of Analytical services
  • Experience of distributed computing architecture, Core Hadoop components (HDFS, Spark, Yarn, HIVE, Impala)
  • Expertise in SQL, advanced SQL skills
  • Expertise in shell scripting and programming (Scala/Python)
  • Experience of Ab Initio implementation on Hadoop (Optional)
  • Experience of Technical Design for ETL/Hadoop/AWS components
Job Responsibility
Job Responsibility
  • Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data
  • Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures
  • Development of processing and analysis algorithms fit for the intended data complexity and volumes
  • Collaboration with data scientist to build and deploy machine learning models
What we offer
What we offer
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
  • Fulltime
Read More
Arrow Right