CrawlJobs Logo

Python Data Engineer

United Kingdom, London · Job Posted January 18, 2026
Apply Position
Job Link Share

Job Description

We are currently looking for an Data Engineer to join our fast-paced, data-driven tech team within a global digital media environment. You’ll play a crucial role in shaping how the business collects, models, and activates data across multiple commercial and editorial functions. In this hands-on role, you will architect and build scalable data pipelines, optimise infrastructure, and deliver high-value insight tools that empower decision makers. The Data Engineer will work closely with commercial, product, and analytics teams, making this position vital to the organisation’s long-term data strategy. As an Data Engineer, you’ll combine engineering, analytics, and product thinking to create a reliable, high-performing data ecosystem. You’ll have the opportunity to work with modern cloud technologies, large-scale datasets, and a business that truly values data as a strategic asset. This role stands out because it sits at the intersection of engineering and commercial impact, your work directly supports revenue optimisation, global reporting, and critical business decisions.

Job Responsibility

  • Building, operating, and optimising end-to-end ETL/ELT data pipelines using APIs, SFTP, and containerised orchestration tools
  • Developing scalable and well-structured data models that support commercial, programmatic, and affiliate revenue functions
  • Managing and improving complex data infrastructure that processes high-volume, multi-source Big Data
  • Creating, maintaining, and enhancing interactive dashboards that drive KPI-focused decision-making
  • Owning data quality, ensuring accuracy, consistency, and reliability across all core datasets
  • Analysing campaign, monetisation, and platform performance and providing actionable insights
  • Collaborating with Operations, Sales, Marketing, Finance, and Senior Analytics teams
  • Supporting strategic projects with advanced data modelling and insight generation

Requirements

  • Strong Python and or Pyspark
  • Experience with cloud technologies such as GCP (BigQuery, Compute Engine, Kubernetes) and AWS (Redshift, EC2)
  • Experience building ETL/ELT pipelines and working with APIs or SFTP integrations
  • Understanding of data modelling, warehousing, and Big Data environments
  • Strong analytical and creative problem-solving skills
  • Ability to manage projects and collaborate effectively in a team
  • Experience creating util packages in Python

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Python Data Engineer

8 matching positions

Python Data Engineer

We are seeking a highly skilled Data Engineer specialising in Python and Google ...
Location
Location
India , Pune
Salary
Salary:
Not provided
vodafone.com Logo
Vodafone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficient Python developer with hands‑on experience in building scalable data processing solutions
  • Skilled in GCP services such as BigQuery, Cloud Storage, Pub/Sub, Cloud Functions, and Cloud Composer
  • Experienced in semantic modelling, workflow orchestration, and Dataform‑based transformations
  • Knowledgeable in SQL, data validation techniques, API-driven microservices (Flask/FastAPI), and automation frameworks
  • Strong collaborator with excellent communication skills and the ability to work effectively with diverse teams
  • Focused, detail‑oriented, and driven to create reliable, high-performance data solutions
Job Responsibility
Job Responsibility
  • Design and maintain ETL/ELT pipelines to ingest, transform, and load large datasets into GCP-based data platforms such as BigQuery and Cloud Storage
  • Develop and optimise scalable back-end components and modular Python code for data processing, workflow orchestration, API integrations, and automation
  • Build and operationalise semantic data layers to support standardised metrics and improved data accessibility
  • Utilise GCP services including Cloud Composer, Dataflow, Pub/Sub, and Cloud Functions to create automated and reliable workflows
  • Develop transformation workflows using Dataform, including SQLX transformations, automated tests, CI/CD integrations, and documentation
  • Collaborate with data scientists, analysts, and business partners to convert data needs into efficient technical solutions
  • Implement data quality checks to ensure accuracy, integrity, and performance in all data workflows
  • Integrate semantic layers into BI tools and leverage metadata and lineage tools for improved governance
What we offer
What we offer
  • Opportunities to work on modern cloud-native data engineering projects with cutting-edge GCP services
  • Exposure to advanced semantic modelling, orchestration, and automation frameworks
  • Collaboration with experts across engineering, data science, and analytics teams
  • Ability to influence large-scale, high-impact data programmes within a global organisation
  • Continuous professional development through hands-on technical challenges
Read More
Arrow Right

Python Data Engineer

We are seeking a highly motivated and intuitive Python Developer to join our dyn...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4-7 years of relevant experience in the Financial Service industry
  • Strong Proficiency in Python: Excellent command of Python programming, including object-oriented principles, data structures, and algorithms
  • PySpark Experience: Demonstrated experience with PySpark for big data processing and analysis
  • Database Expertise: Proven experience working with relational databases, specifically Oracle, and connecting applications using JDBC
  • SQL Mastery: Advanced SQL querying skills for complex data extraction, manipulation, and optimization
  • Big Data Handling: Experience in working with and processing large datasets efficiently
  • Data Streaming: Familiarity with data streaming concepts and technologies (e.g., Kafka, Spark Streaming) for processing continuous data flows
  • Data Analysis Libraries: Proficient in using data analysis libraries such as Pandas for data manipulation and exploration
  • Software Engineering Principles: Solid understanding of software engineering best practices, including version control (Git), testing, and code review
  • Problem-Solving: Intuitive problem-solver with a self-starter mindset and the ability to work independently and as part of a team
Job Responsibility
Job Responsibility
  • Develop, test, and deploy high-quality Python code for data migration, data profiling, and data processing
  • Design and implement scalable solutions for working with large and complex datasets, ensuring data integrity and performance
  • Utilize PySpark for distributed data processing and analytics on large-scale data platforms
  • Develop and optimize SQL queries for various database systems, including Oracle, to extract, transform, and load data efficiently
  • Integrate Python applications with JDBC-compliant databases (e.g., Oracle) for seamless data interaction
  • Implement data streaming solutions to process real-time or near real-time data efficiently
  • Perform in-depth data analysis using Python libraries, especially Pandas, to understand data characteristics, identify anomalies, and support profiling efforts
  • Collaborate with data architects, data engineers, and business stakeholders to understand requirements and translate them into technical specifications
  • Contribute to the design and architecture of data solutions, ensuring best practices in data management and engineering
  • Troubleshoot and resolve technical issues related to data pipelines, performance, and data quality
  • Fulltime
Read More
Arrow Right

Python Data Engineer

Arthur Lawrence is looking for a Python Data Engineer one of our clients in Hous...
Location
Location
United States , Houston
Salary
Salary:
Not provided
arthurlawrence.net Logo
Arthur Lawrence
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of professional Python development
  • Strong knowledge of OOP, design patterns, and SOA
  • Hands-on experience in data engineering, data pipeline development, and web scraping (Requests, BeautifulSoup, Selenium)
  • Oracle/PL SQL expertise, stored procedures
  • Bachelor’s degree in Computer Science, MIS, or related field
  • Agile/Scrum experience
  • Fulltime
Read More
Arrow Right

Data Engineer - Big Data, Python, Databricks

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4 - 6 years of relevant experience in the Financial Service industry
  • Intermediate level experience in Applications Development role
  • Proficient in Big DataTechnologies - Cloudera, Hive, Python, Java/PySpark, Data Bricks
  • Proficient in Data analysis and data modelling
  • Good understanding of ETL concepts
  • Consistently demonstrates clear and concise written and verbal communication
  • Demonstrated problem-solving and decision-making skills
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor's degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Data Engineer (Big Data, Python, Databricks) - Assistant Vice President

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Chennai, Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of relevant handson experience in Big Data technologies like Cloudera, Python, HQL, Java/PySpark
  • Knowledge on Machine Learning, AI would be added advantage
  • Experience in systems analysis, data analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...
Location
Location
United Arab Emirates , Dubai
Salary
Salary:
Not provided
northbaysolutions.com Logo
NorthBay
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in data engineering and data platform development
  • Strong hands-on experience with: AWS Glue
  • Amazon EMR (Spark)
  • AWS Lambda
  • Apache Airflow (MWAA)
  • Amazon EC2
  • Amazon CloudWatch
  • Amazon Redshift
  • Amazon DynamoDB
  • AWS DataZone
Job Responsibility
Job Responsibility
  • Design, develop, and optimize scalable data pipelines using AWS native services
  • Lead the implementation of batch and near-real-time data processing solutions
  • Architect and manage data ingestion, transformation, and storage layers
  • Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
  • Orchestrate complex data workflows using Apache Airflow (MWAA)
  • Develop and manage serverless data processing using AWS Lambda
  • Design and optimize data warehouses using Amazon Redshift
  • Implement and manage NoSQL data models using Amazon DynamoDB
  • Utilize AWS DataZone for data governance, cataloging, and access management
  • Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch
  • Fulltime
Read More
Arrow Right
New

Lead Data Engineer (Python, AWS, SQL, GenAI) (Enterprise Platforms Technology)

Do you love building and pioneering in the technology space? Do you enjoy solvin...
Location
Location
United States , McLean, Virginia; New York, New York
Salary
Salary:
197300.00 - 245600.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree
  • At least 4 years of experience in application development (Internship experience does not apply)
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Job Responsibility
Job Responsibility
  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
  • Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems
  • Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance
What we offer
What we offer
  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right

Lead Data Engineer (Python, AWS, SQL, GenAI) (Enterprise Platforms Technology)

Do you love building and pioneering in the technology space? Do you enjoy solvin...
Location
Location
United States , McLean, Virginia; Chicago, Illinois
Salary
Salary:
179400.00 - 225100.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s Degree
  • At least 4 years of experience in application development (Internship experience does not apply)
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Job Responsibility
Job Responsibility
  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies
  • Work, as a lead developer, with a team of developers with deep experience in data movement, distributed computing, and full stack systems
  • Utilize programming languages like Python, SQL and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Snowflake, Databricks
  • Optimize information system for end-user and downstream application consumers by using sound data design practices
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance
What we offer
What we offer
  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • Comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right