CrawlJobs Logo

Senior Data Engineer

India, Bangalore · Job Posted April 23, 2026
Apply Position
Job Link Share

Job Description

The Senior Data Engineer role requires a strong background in Python and PySpark, with at least 6 years of experience in data engineering. The candidate will design scalable data processing pipelines and work with Google Cloud Platform services. Proficiency in SQL and experience with AI/ML workflows are essential. This position is based in Bangalore, requiring in-office presence three days a week.

Job Responsibility

  • Design scalable data processing pipelines
  • Work with Google Cloud Platform services

Requirements

  • 6+ years as a Data Engineer
  • Strong Python programming skills, including data manipulation with libraries like Pandas, NumPy, and scikit-learn
  • Strong experience applying Data Engineering techniques and backend development in a production environment
  • Demonstrated expertise in building scalable data processing pipelines using PySpark and distributed computing frameworks
  • Hands-on, practical experience with Google Cloud Platform (GCP). GCP-specific services include Cloud Composer, Cloud Run, Cloud Functions, BigQuery, Dataform, Pub/Sub, etc
  • Significant hands-on experience with CI/CD pipelines, DevOps tooling, and modern engineering practices
  • SQL skills and strong proficiency in Python and PySpark
  • Experience supporting AI/ML workflows
  • Experience working with large-scale data processing, ETL/ELT pipelines, and data transformation using PySpark
  • Backend Python experience with Flask, Django, or FastAPI

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Data Engineer

8 matching positions

Senior Data Engineer

*** Senior Data Engineer - 10 Months+ Contract - Hybrid -Vilnius, Lithuania*** R...
Location
Location
Lithuania , Vilnius
Salary
Salary:
Not provided
redglobal.com Logo
RED Commerce - The Global SAP Solutions Provider
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience as a Data Engineer
  • 7+ years of experience in the overall IT area
  • Strong experience with PySpark and Databricks (AWS environment)
  • Proven experience building data pipelines and implementing Medallion architecture from scratch
  • Excellent communication skills with fluent English (spoken and written)
  • Strong stakeholder management skills with the ability to gather business requirements and translate into technical solutions
  • Able to join within 2-4 weeks of offer
Read More
Arrow Right
New

Senior Data Engineer

We’re looking for a Senior Data Engineer to help scale and evolve our data platf...
Location
Location
United States
Salary
Salary:
150000.00 - 180000.00 USD / Year
helpcare.ai Logo
Helpcare AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A min of 5+ years of software engineering experience with the last 2-3 years in data engineering, ideally including early-stage or 0→1 environments
  • Strong Python programming experience
  • Advanced proficiency in SQL
  • Proven experience building ETL/ELT pipelines end-to-end
  • Experience with orchestration tools like Airflow
  • Deep understanding of the data lifecycle: ingestion → storage → processing → transformation → serving
  • Experience with cloud platforms (AWS, GCP, or Azure)
  • Experience supporting BI tools (Looker, Tableau, etc.)
  • Familiarity with modern data warehouses (Snowflake, BigQuery, Redshift)
Job Responsibility
Job Responsibility
  • Design and scale our data platform including pipelines, models, and orchestration frameworks
  • Develop scalable ETL/ELT pipelines for ingesting data from APIs, databases, and event streams
  • Define and implement systems for data ingestion, storage, processing, and transformation
  • Build and manage workflow orchestration using tools like Airflow
  • Build semantic layer as well as dashboards
  • Establish best practices for data modeling, testing, and quality
  • Partner with stakeholders to shape data requirements and enable BI and analytics use cases
  • Optimize systems for performance, scalability, and cost from day one
  • Apply software engineering principles (testing, CI/CD, modular design) to data infrastructure
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

About this role: Wells Fargo is seeking a Senior Data Engineer. In this role, ...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
https://www.wellsfargo.com/ Logo
Wells Fargo
Expiration Date
June 12, 2026
Flip Icon
Requirements
Requirements
  • 4+ years of Data Engineering experience
  • Bachelor's degree in Computer Science, Engineering, or a related field
  • 6-10 years of professional experience in data engineering or related roles
  • Proven experience handling large-scale, high-volume data systems
  • Apache Spark (Core, SQL, DataFrames/Datasets)
  • ETL tools – Preferably Informatica
  • Databases: Oracle
  • Streaming Technologies: Apache Kafka and Apache Flink
  • Cloud Storage: Amazon S3
  • NoSQL: Working knowledge of MongoDB
Job Responsibility
Job Responsibility
  • Lead moderately complex initiatives within Technology and contribute to large scale data processing framework initiatives related to enterprise strategy deliverables
  • Build and maintain optimized and highly available data pipelines that facilitate deeper analysis and reporting
  • Review and analyze moderately complex business, operational or technical challenges that require an in-depth evaluation of variable factors
  • Oversee the data integration work, including developing a data model, maintaining a data warehouse and analytics environment, and writing scripts for data integration and analysis
  • Resolve moderately complex issues and lead teams to meet data engineering deliverables while leveraging solid understanding of data information policies, procedures and compliance requirements
  • Collaborate and consult with colleagues and managers to resolve data engineering issues and achieve strategic goals
  • Design, develop, and maintain scalable and reliable data pipelines using Apache Spark for batch and near-real-time processing
  • Build, enhance, and support ETL workflows, preferably using Informatica, ensuring data quality, performance, and reliability
  • Develop and maintain integrations with Oracle databases, including data modeling, performance tuning, and query optimization
  • Implement and manage streaming data pipelines using Apache Kafka and Apache Flink for real-time data ingestion and processing
  • Fulltime
!
Read More
Arrow Right
New

Senior Data Engineer

Senior Data Engineer. Must Haves: Strong hands-on experience building and suppor...
Location
Location
United States , Los Angeles
Salary
Salary:
Not provided
bhsg.com Logo
Beacon Hill
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on experience building and supporting production-grade data pipelines using PySpark and Apache Spark
  • Extensive experience working within Databricks environments
  • Experience with Databricks Unity Catalog migrations and modernization initiatives
  • Experience with Databricks Unity Catalog governance and security models
  • Strong understanding of ETL/ELT design, data modeling, and incremental data processing
  • Experience modernizing legacy data platforms and data migration projects
  • Experience working with large-scale datasets in cloud-based data platforms
  • Experience with workflow orchestration, scheduling, and pipeline automation
  • Knowledge of cloud data platforms such as Snowflake and/or Google Cloud
  • Experience implementing data quality checks, monitoring, logging, and alerting
Job Responsibility
Job Responsibility
  • Migrate existing data assets and pipelines to Databricks Unity Catalog
  • Refactor and optimize existing pipelines using modern engineering standards and best practices
  • Build and maintain scalable PySpark-based data pipelines
  • Develop new source-of-truth datasets and modernize legacy data workflows
  • Design and implement workflow orchestration using Databricks Jobs
  • Deploy and manage pipelines using Databricks Asset Bundles
  • Ensure data pipelines are reliable, observable, and production-ready
  • Implement data quality controls, monitoring, and operational alerting
  • Support platform migrations, testing, validation, and production cutovers
  • Partner with engineering teams to improve platform stability, scalability, and data reliability
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

We are seeking an experienced Senior Data Engineer to join a leading automotive ...
Location
Location
Sweden , Gothenburg
Salary
Salary:
53333.00 - 66667.00 SEK / Month
amaris.com Logo
Amaris Consulting
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on experience with PySpark and Databricks in production environments
  • Advanced Python programming skills
  • Experience with Git/Gerrit and CI/CD pipelines (GitHub Actions, Jenkins, GitLab, or similar)
  • Solid knowledge of Docker and workflow orchestration tools such as Airflow or Databricks Jobs
  • Expertise in designing scalable, privacy-compliant data architectures and pipelines
  • Strong SQL skills and experience handling large-scale datasets
  • Experience with Azure cloud services and Snowflake (or equivalent cloud data warehouse technologies)
  • Knowledge of monitoring, observability, automated testing, and cost optimization practices
  • Familiarity with Power BI or similar BI and reporting tools
  • Understanding of automotive UDS diagnostics and vehicle signal data
Job Responsibility
Job Responsibility
  • Design and develop scalable data pipelines using PySpark and Databricks
  • Build and maintain raw and refined data products related to vehicle signals, software versions, drive cycles, distance measurements, and driver feedback
  • Optimize data processing performance, reliability, and cost efficiency
  • Implement monitoring, observability, automated testing, and usage tracking for data products
  • Collaborate with architects, engineers, Product Owners, and business stakeholders to deliver robust data solutions
  • Support documentation, knowledge transfer, and operational handovers
  • Contribute to technical leadership and best practices within the data engineering domain
What we offer
What we offer
  • An international community, bringing together 110+ different nationalities
  • An environment where trust has a central place: 70% of our key leaders started their careers at the first level of responsibilities
  • A robust training system with our internal Academy and 250+ available modules
  • A vibrant workplace that frequently gathers for internal events (afterworks, team buildings, etc.)
  • Opportunity to turn ideas into action and make a tangible impact through ESG commitments
  • WeCare Together program empowering employees to design and lead projects that create social or environmental impact
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

Senior Data Engineer - Media Org (Hybrid NYC or LA). I’m working on a Senior Dat...
Location
Location
United States , New York City or Los Angeles
Salary
Salary:
Not provided
lawrenceharvey.com Logo
Lawrence Harvey
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong Data Engineering experience building production data pipelines
  • Strong PySpark and SQL expertise
  • Experience with Databricks or similar Spark-based cloud platforms
  • Experience building batch and streaming pipelines
  • Experience with orchestration tools such as Airflow or similar
  • Familiarity with AWS environments, data lakes, and modern cloud data architecture
  • Experience with modern Lakehouse design patterns, including Medallion Architecture (Bronze, Silver / Gold layers) for structuring scalable and modular data pipelines
Job Responsibility
Job Responsibility
  • Designing batch and real-time data pipelines
  • Working heavily within a Databricks environment
  • Helping shape modern lakehouse architecture
  • Partnering closely with analytics, product, and data science teams
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

At AO, we’re all about using data to make smarter decisions and create brilliant...
Location
Location
United Kingdom , Bolton
Salary
Salary:
Not provided
ao.com Logo
AO
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Solid experience in data engineering, building and maintaining data pipelines
  • Strong hands-on knowledge of AWS services like Redshift, Glue, Spectrum and Lambda
  • Confidence working with Python and TSQL
  • A good understanding of data modelling, ETL processes and data warehousing
  • Experience (or exposure to) Infrastructure as Code tools such as Terraform or CloudFormation
  • Familiarity with CI/CD pipelines and automated deployment practices
  • A natural problem solver who enjoys improving and optimising the way things work
  • Comfortable collaborating with different teams and bringing ideas to life together
  • Someone who’s keen to share knowledge and support others’ development
Job Responsibility
Job Responsibility
  • designing and building robust data pipelines that keep our data flowing smoothly and reliably, using AWS tools like Redshift, Glue, Spectrum and Lambda
  • Working closely with teams across AO, you’ll turn business needs into practical, scalable solutions that help unlock the value of our data
  • continuously improve performance, reliability and cost-efficiency across our data systems, while making sure everything is secure and compliant
  • setting up automated deployments using Infrastructure as Code and CI/CD pipelines, to troubleshooting issues and documenting your work, you’ll help keep things running seamlessly
  • share your knowledge with others in the team, helping to coach and support colleagues as we grow
What we offer
What we offer
  • 25 days holidays, plus bank holidays (increasing to 27 days after 2 years with us)
  • Pension
  • Contribute 5% of your annual salary and we'll do the same
  • Be a VIP at the AO Arena
  • discounted gym membership
  • an onsite spa
  • Help @ Hand scheme giving you access to virtual GP’s, Mental Health support and much more
  • exclusive discounts across our product range
  • Enhanced Maternity, Paternity and Adoption leave
  • 2 fully paid days a year to donate your time to any charity of your choice
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

Are you a Senior Data professional who enjoys mentoring talent and designing hig...
Location
Location
Greece , Athens
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
July 10, 2026
Flip Icon
Requirements
Requirements
  • University degree in Computer Science, Statistics, Mathematics, or a related Quantitative field.
  • 5+ years of professional experience in Data Engineering with a mastery of MS SQL Server (T-SQL, SSIS, Performance Tuning).
  • Proven experience in mentoring or leading technical team members.
  • Exposure to VBM methodologies or a strong understanding of financial/business data modeling.
  • Hands-on experience with QlikSense or Qlikview (or similar BI visualization tools like Tableau/Power BI).
  • Excellent command of the English language.
  • A proactive leader with an analytical mindset and a passion for knowledge sharing.
Job Responsibility
Job Responsibility
  • Lead the design and implementation of robust data architectures using on-premise MS SQL Server technologies.
  • Mentor and guide junior developers, assisting them in technical design and ensuring best practices in database management and ETL processes.
  • Develop complex data models that drive strategic business decisions and high-level reporting.
  • Collaborate with business stakeholders to translate VBM methodologies into technical requirements.
  • Oversee the integration of data flows into visualization tools, ensuring data accuracy and top-tier performance.
Read More
Arrow Right