Python Data Engineer Job at Arthur Lawrence (Houston)

Python Data Engineer

We are seeking a highly skilled Data Engineer specialising in Python and Google ...

Location

India , Pune

Salary:

Not provided

Vodafone

Expiration Date

Until further notice

Requirements

Proficient Python developer with hands‑on experience in building scalable data processing solutions
Skilled in GCP services such as BigQuery, Cloud Storage, Pub/Sub, Cloud Functions, and Cloud Composer
Experienced in semantic modelling, workflow orchestration, and Dataform‑based transformations
Knowledgeable in SQL, data validation techniques, API-driven microservices (Flask/FastAPI), and automation frameworks
Strong collaborator with excellent communication skills and the ability to work effectively with diverse teams
Focused, detail‑oriented, and driven to create reliable, high-performance data solutions

Job Responsibility

Design and maintain ETL/ELT pipelines to ingest, transform, and load large datasets into GCP-based data platforms such as BigQuery and Cloud Storage
Develop and optimise scalable back-end components and modular Python code for data processing, workflow orchestration, API integrations, and automation
Build and operationalise semantic data layers to support standardised metrics and improved data accessibility
Utilise GCP services including Cloud Composer, Dataflow, Pub/Sub, and Cloud Functions to create automated and reliable workflows
Develop transformation workflows using Dataform, including SQLX transformations, automated tests, CI/CD integrations, and documentation
Collaborate with data scientists, analysts, and business partners to convert data needs into efficient technical solutions
Implement data quality checks to ensure accuracy, integrity, and performance in all data workflows
Integrate semantic layers into BI tools and leverage metadata and lineage tools for improved governance

What we offer

Opportunities to work on modern cloud-native data engineering projects with cutting-edge GCP services
Exposure to advanced semantic modelling, orchestration, and automation frameworks
Collaboration with experts across engineering, data science, and analytics teams
Ability to influence large-scale, high-impact data programmes within a global organisation
Continuous professional development through hands-on technical challenges

Python Data Engineer

We are seeking a highly motivated and intuitive Python Developer to join our dyn...

Location

India , Chennai

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

4-7 years of relevant experience in the Financial Service industry
Strong Proficiency in Python: Excellent command of Python programming, including object-oriented principles, data structures, and algorithms
PySpark Experience: Demonstrated experience with PySpark for big data processing and analysis
Database Expertise: Proven experience working with relational databases, specifically Oracle, and connecting applications using JDBC
SQL Mastery: Advanced SQL querying skills for complex data extraction, manipulation, and optimization
Big Data Handling: Experience in working with and processing large datasets efficiently
Data Streaming: Familiarity with data streaming concepts and technologies (e.g., Kafka, Spark Streaming) for processing continuous data flows
Data Analysis Libraries: Proficient in using data analysis libraries such as Pandas for data manipulation and exploration
Software Engineering Principles: Solid understanding of software engineering best practices, including version control (Git), testing, and code review
Problem-Solving: Intuitive problem-solver with a self-starter mindset and the ability to work independently and as part of a team

Job Responsibility

Develop, test, and deploy high-quality Python code for data migration, data profiling, and data processing
Design and implement scalable solutions for working with large and complex datasets, ensuring data integrity and performance
Utilize PySpark for distributed data processing and analytics on large-scale data platforms
Develop and optimize SQL queries for various database systems, including Oracle, to extract, transform, and load data efficiently
Integrate Python applications with JDBC-compliant databases (e.g., Oracle) for seamless data interaction
Implement data streaming solutions to process real-time or near real-time data efficiently
Perform in-depth data analysis using Python libraries, especially Pandas, to understand data characteristics, identify anomalies, and support profiling efforts
Collaborate with data architects, data engineers, and business stakeholders to understand requirements and translate them into technical specifications
Contribute to the design and architecture of data solutions, ensuring best practices in data management and engineering
Troubleshoot and resolve technical issues related to data pipelines, performance, and data quality

Fulltime

Python Data Engineer

We are currently looking for an Data Engineer to join our fast-paced, data-drive...

Location

United Kingdom , London

Salary:

Not provided

Data Idols

Expiration Date

Until further notice

Requirements

Strong Python and or Pyspark
Experience with cloud technologies such as GCP (BigQuery, Compute Engine, Kubernetes) and AWS (Redshift, EC2)
Experience building ETL/ELT pipelines and working with APIs or SFTP integrations
Understanding of data modelling, warehousing, and Big Data environments
Strong analytical and creative problem-solving skills
Ability to manage projects and collaborate effectively in a team
Experience creating util packages in Python

Job Responsibility

Building, operating, and optimising end-to-end ETL/ELT data pipelines using APIs, SFTP, and containerised orchestration tools
Developing scalable and well-structured data models that support commercial, programmatic, and affiliate revenue functions
Managing and improving complex data infrastructure that processes high-volume, multi-source Big Data
Creating, maintaining, and enhancing interactive dashboards that drive KPI-focused decision-making
Owning data quality, ensuring accuracy, consistency, and reliability across all core datasets
Analysing campaign, monetisation, and platform performance and providing actionable insights
Collaborating with Operations, Sales, Marketing, Finance, and Senior Analytics teams
Supporting strategic projects with advanced data modelling and insight generation

Data Engineer - Big Data, Python, Databricks

The Applications Development Intermediate Programmer Analyst is an intermediate ...

Location

India , Chennai

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

4 - 6 years of relevant experience in the Financial Service industry
Intermediate level experience in Applications Development role
Proficient in Big DataTechnologies - Cloudera, Hive, Python, Java/PySpark, Data Bricks
Proficient in Data analysis and data modelling
Good understanding of ETL concepts
Consistently demonstrates clear and concise written and verbal communication
Demonstrated problem-solving and decision-making skills
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Bachelor's degree/University degree or equivalent experience

Job Responsibility

Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
Apply fundamental knowledge of programming languages for design specifications
Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
Serve as advisor or coach to new or lower level analysts
Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Data Engineer (Big Data, Python, Databricks) - Assistant Vice President

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

India , Chennai, Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

5-8 years of relevant handson experience in Big Data technologies like Cloudera, Python, HQL, Java/PySpark
Knowledge on Machine Learning, AI would be added advantage
Experience in systems analysis, data analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Bachelor’s degree/University degree or equivalent experience

Job Responsibility

Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
Ensure essential procedures are followed and help define operating standards and processes
Serve as advisor or coach to new or lower level analysts
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Fabric Data Engineer / Azure Data Engineer

Location

Salary:

Not provided

Myticas Consulting

Expiration Date

Until further notice

Requirements

Hands-on Data Engineer with strong Microsoft Fabric experience, including Lakehouse, OneLake, Data Factory/Pipelines, Warehousing, and Fabric ecosystem services
Strong Python development skills for data engineering, data transformation, automation, and pipeline development
Experience working with Microsoft Fabric Lakehouse architecture, including data ingestion, transformation, storage, and analytics enablement
Strong understanding of data engineering fundamentals, including data modeling, partitioning, optimization, and performance tuning
Experience supporting Azure Data Platform technologies, including Azure Data Factory, Azure Synapse Analytics, Azure Storage, and related services
Ability to work within large-scale enterprise data environments supporting multiple business units and data domains
Knowledge of Medallion Architecture (Bronze, Silver, Gold) and practical implementation of modern data lakehouse solutions
Experience integrating and transforming structured and semi-structured data sources including APIs, databases, JSON, CSV, and cloud-based systems
Understanding of Fabric operational monitoring, capacity consumption monitoring, and platform governance best practices
Ability to clearly explain and demonstrate the distinction between Fabric Files, Tables, Lakehouses, Warehouses, and OneLake storage concepts

Job Responsibility

Design, build, and support scalable ETL/ELT data pipelines across enterprise Azure environments
Ability to support integration across multiple systems, applications, databases, and cloud platforms

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...

Location

United Arab Emirates , Dubai

Salary:

Not provided

NorthBay

Expiration Date

Until further notice

Requirements

8+ years of experience in data engineering and data platform development
Strong hands-on experience with: AWS Glue
Amazon EMR (Spark)
AWS Lambda
Apache Airflow (MWAA)
Amazon EC2
Amazon CloudWatch
Amazon Redshift
Amazon DynamoDB
AWS DataZone

Job Responsibility

Design, develop, and optimize scalable data pipelines using AWS native services
Lead the implementation of batch and near-real-time data processing solutions
Architect and manage data ingestion, transformation, and storage layers
Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
Orchestrate complex data workflows using Apache Airflow (MWAA)
Develop and manage serverless data processing using AWS Lambda
Design and optimize data warehouses using Amazon Redshift
Implement and manage NoSQL data models using Amazon DynamoDB
Utilize AWS DataZone for data governance, cataloging, and access management
Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch

Fulltime

New

Senior Python Engineer (Data Engineering & AI Agents)

Our client is a leading global investment management company headquartered in Lo...

Location

Poland; Spain; United Kingdom

Salary:

Not provided

Intellias

Expiration Date

Until further notice

Requirements

6+ years building production software in Python, with strong engineering fundamentals (testing, performance, clean design)
Solid data engineering: SQL, columnar formats (e.g. Parquet), pipeline design, and handling datasets large enough that naive approaches don't scale
Hands-on experience with at least one analytical or query engine (e.g. DuckDB, Trino, Spark, ClickHouse)
Real experience building LLM / agent applications: retrieval (RAG), vector databases, and tool / function calling
A working understanding of data governance: cataloguing, metadata, lineage, and access control (RBAC / ABAC)
An instinct for data quality and trustworthy 'golden' sources

Job Responsibility

Build production-grade Python services and data pipelines over large data stores (columnar / time-series and relational), and the queries that join across them
Select and implement the right query or analytical engine for each workload, rather than defaulting to one
Build catalogue, metadata, lineage and semantic layers that make data discoverable and consistently understood across teams
Implement access control that travels with the data: fusing sensitivity and licensing scope, enforced at the point of use, including for AI agents
Build agent-facing data access: retrieval (RAG), vector search, and APIs / MCP servers, with permissions applied before context reaches the model
Apply LLMs pragmatically to data work (metadata generation, classification, entity resolution) with humans in the loop and evaluate the quality of what the agents produce
Help keep data trustworthy: establish golden sources, deduplication and data-quality checks at the source
Contribute to discovery and solutioning: assessing current state, weighing build-vs-adopt, and shaping pragmatic, costed plans

Select Country

Python Data Engineer

Job Description

Requirements

Nice to have

Looking for more opportunities?

Python Data Engineer

Python Data Engineer

Python Data Engineer

Python Data Engineer

Data Engineer - Big Data, Python, Databricks

Data Engineer (Big Data, Python, Databricks) - Assistant Vice President

Fabric Data Engineer / Azure Data Engineer

Senior AWS Data Engineer / Data Platform Engineer

Senior Python Engineer (Data Engineering & AI Agents)

Our AI answers in your language