CrawlJobs Logo
Wissen Logo Wissen · IT - Software Development

Data Engineer (AWS & PySpark)

India, Bangalore South Employment contract · Job Posted June 28, 2026
Apply Position
Job Link Share

Job Description

We are looking for a hands-on Data Engineer with strong expertise in AWS data services and distributed data processing. The ideal candidate should have solid experience building and maintaining scalable data pipelines using PySpark, AWS EMR, Athena, and AWS Glue, along with strong SQL and Python skills.

Job Responsibility

  • Design, develop, and maintain scalable data pipelines using PySpark and AWS services
  • Build ETL workflows using AWS Glue and EMR
  • Develop data ingestion, transformation, and processing frameworks
  • Optimize large-scale data processing jobs and improve performance
  • Write efficient SQL queries for analytics and reporting requirements
  • Work with structured and semi-structured datasets in cloud environments
  • Collaborate with data analysts, architects, and business stakeholders
  • Ensure data quality, reliability, and operational excellence

Requirements

  • Strong Python programming skills
  • Hands-on PySpark development experience
  • AWS EMR
  • AWS Athena
  • AWS Glue
  • SQL
  • Data Warehousing Concepts
  • ETL Development
  • Data Pipeline Design

Nice to have

  • Airflow
  • Redshift
  • S3
  • Lambda
  • Spark Optimization
  • Data Modelling
  • CI/CD
Wissen - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Data Engineer (AWS & PySpark)

8 matching positions

Data Engineer - AWS, PySpark

You will be responsible for supporting the successful delivery of Location Strat...
Location
Location
India , Bengaluru; Pune
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hands on experience in pyspark and strong knowledge on Dataframes, RDD and SparkSQL
  • Hands on Experience in developing, testing and maintaining applications on AWS Cloud
  • Strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake formation, Athena)
  • Design and implement scalable and efficient data transformation/storage solutions using Snowflake
  • Experience in Data ingestion to Snowflake for different storage format such Parquet, Iceberg, JSON, CSV etc
  • Experience in using DBT (Data Build Tool) with snowflake for ELT pipeline development
  • Experience in Writing advanced SQL and PL SQL programs
  • Hands On Experience for building reusable components using Snowflake and AWS Tools/Technology
  • Should have worked at least on two major project implementations
  • Exposure to data governance or lineage tools such as Immuta and Alation is added advantage
Job Responsibility
Job Responsibility
  • Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data
  • Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures
  • Development of processing and analysis algorithms fit for the intended data complexity and volumes
  • Collaboration with data scientist to build and deploy machine learning models
What we offer
What we offer
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
  • Fulltime
Read More
Arrow Right

Data Engineer - AWS, Pyspark

You will be responsible for supporting the successful delivery of Location Strat...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hands on experience in pyspark and strong knowledge on Dataframes, RDD and SparkSQL
  • Hands on Experience in developing, testing and maintaining applications on AWS Cloud
  • Strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake formation, Athena)
  • Design and implement scalable and efficient data transformation/storage solutions using Snowflake
  • Experience in Data ingestion to Snowflake for different storage format such Parquet, Iceberg, JSON, CSV etc
  • Experience in using DBT (Data Build Tool) with snowflake for ELT pipeline development
  • Experience in Writing advanced SQL and PL SQL programs
  • Hands On Experience for building reusable components using Snowflake and AWS Tools/Technology
  • Should have worked at least on two major project implementations
  • Exposure to data governance or lineage tools such as Immuta and Alation is added advantage
Job Responsibility
Job Responsibility
  • Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data
  • Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures
  • Development of processing and analysis algorithms fit for the intended data complexity and volumes
  • Collaboration with data scientist to build and deploy machine learning models
  • To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure
What we offer
What we offer
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
  • Fulltime
Read More
Arrow Right

AWS Data Engineer

Join us as an AWS Data Engineer Barclays, responsible for supporting the success...
Location
Location
India , Pune
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Firsthand Experience in developing, testing and maintaining applications on AWS Cloud
  • Strong hands‑on experience with the AWS Data Analytics stack (Amazon S3, AWS Glue, Athena, Lambda, IAM, Lake Formation, KMS, STS, and Step Functions), with a proven ability to build, test, and support secure, scalable, and well‑governed data pipelines
  • Firsthand experience in Airflow and PySpark and strong knowledge of Python
  • Design and implement scalable and efficient data transformation/storage solutions using Snowflake
  • Experience in Data ingestion to Snowflake for different storage format such Parquet, Iceberg, JSON, CSV etc
  • Experience in using DBT (Data Build Tool) with snowflake for ELT pipeline development
  • Experience in Writing advanced SQL and PL SQL programs
  • Experience in AWS data pipeline development
  • HandsOn Experience for building reusable components using Snowflake and AWS Tools/Technology
  • Must have completed two major projects
Job Responsibility
Job Responsibility
  • Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data
  • Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures
  • Development of processing and analysis algorithms fit for the intended data complexity and volumes
  • Collaboration with data scientist to build and deploy machine learning models
What we offer
What we offer
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
  • Fulltime
Read More
Arrow Right

AWS Data Engineer

We are seeking a detail-oriented and capable Data Engineer – AWS to join our Dat...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience in data engineering or software engineering with data focus
  • Strong interest in cloud-based data platforms and distributed processing
  • Good analytical and problem-solving skills
  • Attention to detail and commitment to data quality and reliability
  • Effective communication and teamwork skills
  • Willingness to learn and develop in AWS and modern data engineering practices
  • Hands-on experience with AWS cloud services, especially AWS Glue, Python / PySpark, SQL querying and data manipulation
  • Exposure to YAML or configuration-driven pipelines (desirable)
  • Experience building or supporting data pipelines, ETL/ELT processes
  • Familiarity with data lakes and/or Lakehouse concepts, Distributed processing frameworks (e.g., Spark)
Job Responsibility
Job Responsibility
  • Support delivery across data engineering and platform development initiatives
  • Collaborate with architects, engineers, and stakeholders to implement data solutions on AWS
  • Assist in planning and executing engineering tasks, releases, and deliverables
  • Build and maintain data pipelines and workflows on AWS platforms
  • Develop ETL/ELT pipelines using AWS Glue, Python / PySpark, SQL, Configuration-driven frameworks (e.g., YAML)
  • Support ingestion, transformation, and processing of structured and semi-structured data
  • Contribute to the development of scalable, reusable data components and services
  • Test and validate data pipelines and processing jobs running on AWS services
  • Develop and execute data validation and reconciliation queries using SQL
  • Work with AWS services including AWS Glue, S3-based data lakes, Related data processing and orchestration services
Read More
Arrow Right

Senior AWS Data Engineer

We are seeking a highly skilled Senior Data Engineer with 5+ years of experience...
Location
Location
India , Pune
Salary
Salary:
Not provided
vidushiinfotech.com Logo
Vidushi Infotech SSP Pvt. Ltd.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in hands-on programming with Python and PySpark
  • Expertise in Boto3 and various Python frameworks and libraries, adhering strictly to Python best practices (PEP 8)
  • Strong experience in Spark SQL and PySpark optimization techniques (e.g., partitioning, caching, broadcast joins)
  • Deep architectural knowledge of AWS services, including: S3, EC2, Lambda, Redshift, CloudFormation
  • Advanced understanding of Git (branching strategies, PR reviews)
  • Experience with JFrog Artifactory for dependency management and artifact storage
  • Proficiency in CI/CD pipelines and automated testing frameworks
  • Analytical Mindset: Ability to debug complex, non-obvious issues in distributed systems
  • Clean Coder: Passion for writing 'clean code' and mentoring junior engineers on maintainability
  • Architectural Thinking: Ability to explain the 'why' behind choosing specific AWS components over others
Job Responsibility
Job Responsibility
  • Design, develop, and maintain complex ETL/ELT pipelines to build high-value data assets
  • Lead the code refactorization of legacy codebases, improving readability, maintainability, and performance
  • Perform deep code optimization using Spark SQL and PySpark to handle large-scale datasets efficiently
  • Implement a Test-Driven Development (TDD) approach, writing comprehensive unit tests to ensure functionality and catch bugs early in the lifecycle
  • Isolate and resolve difficult bugs, including those related to performance bottlenecks, concurrency issues, and complex logic flaws
  • Design and deploy solutions utilizing the full AWS stack, explaining the trade-offs and benefits of specific services for various use cases
  • Fulltime
Read More
Arrow Right

Senior Software Engineer (AWS & Data)

We are seeking a Senior Software Engineer – AWS & Data to join a remote developm...
Location
Location
Poland
Salary
Salary:
130.00 - 150.00 PLN / Hour
cyclad.pl Logo
Cyclad Sp. z o.o.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience in AWS cloud-native development, preferably within data-related projects
  • Strong hands-on experience with AWS Glue and PySpark
  • Experience with ECS and/or Kubernetes (development and basic administration)
  • Solid frontend development experience with React
  • Strong experience in building REST APIs using FastAPI
  • Familiarity with DevOps tools and practices (e.g., GitHub Actions, Terraform)
  • Experience with Terraform in infrastructure automation
  • Knowledge of CI/CD pipeline design and optimization
  • Exposure to scalable data architecture patterns
  • Strong problem-solving skills and attention to detail
Job Responsibility
Job Responsibility
  • Design, develop, and maintain cloud-native applications using AWS services (including AWS Glue and ECS)
  • Build and optimize data processing solutions using AWS Glue and PySpark
  • Develop and maintain RESTful APIs using FastAPI
  • Design and implement user interfaces using React, ensuring seamless frontend-backend integration
  • Work with containerized environments (ECS/Kubernetes) for application deployment and management
  • Collaborate with product owners and cross-functional teams to deliver high-quality solutions
  • Participate in code reviews and contribute to best development practices
  • Troubleshoot and resolve issues related to performance, scalability, and security
  • Continuously improve development processes by staying up to date with new technologies
What we offer
What we offer
  • Remote working model
  • Private medical care with dental care (covering 70% of costs)
  • Multisport card (also for an accompanying person)
  • Life insurance
  • Honey Pot
  • Fulltime
Read More
Arrow Right

Senior AWS Data Engineer

This role is to support and enhance enterprise business intelligence and analyti...
Location
Location
United States , Torrance
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or equivalent practical experience
  • 7+ years of experience building enterprise data platforms or data engineering solutions
  • 5+ years of hands‑on experience with AWS cloud services
  • Strong hands‑on experience with AWS Glue and PySpark for ETL processing
  • Experience developing serverless applications using AWS Lambda
  • Deep expertise with Amazon Redshift, including performance tuning and advanced SQL
  • Experience with workflow orchestration tools such as Apache Airflow
  • Experience implementing secure vendor integrations using AWS Transfer Family
  • Experience designing and supporting data migration and replication pipelines using AWS DMS
  • Advanced SQL skills and experience analyzing complex datasets
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ETL/ELT pipelines using AWS Glue with PySpark for large‑scale data processing
  • Develop and support serverless integrations using AWS Lambda for event‑driven workflows and system integrations
  • Design and optimize Amazon Redshift data warehouse solutions, including: Advanced SQL analytics, Stored procedures, Performance tuning
  • Lead implementation of secure vendor file transfer and ingestion solutions using AWS Transfer Family
  • Design and implement database migration and replication pipelines using AWS Database Migration Service (DMS)
  • Build and manage workflow orchestration using Apache Airflow or similar orchestration tools
  • Analyze data quality, transformation logic, and pipeline performance using SQL and data analysis techniques
  • Troubleshoot and resolve production data pipeline and integration issues across AWS services
  • Provide technical guidance to development team members on: AWS best practices, Cost optimization, Performance optimization
  • Partner with enterprise architecture, security, and compliance teams to ensure SOX and regulatory compliance
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right

Senior AWS Data Engineer

This role is to support and enhance enterprise business intelligence and analyti...
Location
Location
United States , Torrance
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or equivalent practical experience
  • 7+ years of experience building enterprise data platforms or data engineering solutions
  • 5+ years of hands‑on experience with AWS cloud services
  • Strong hands‑on experience with AWS Glue and PySpark for ETL processing
  • Experience developing serverless applications using AWS Lambda
  • Deep expertise with Amazon Redshift, including performance tuning and advanced SQL
  • Experience with workflow orchestration tools such as Apache Airflow
  • Experience implementing secure vendor integrations using AWS Transfer Family
  • Experience designing and supporting data migration and replication pipelines using AWS DMS
  • Advanced SQL skills and experience analyzing complex datasets
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ETL/ELT pipelines using AWS Glue with PySpark for large‑scale data processing
  • Develop and support serverless integrations using AWS Lambda for event‑driven workflows and system integrations
  • Design and optimize Amazon Redshift data warehouse solutions, including: Advanced SQL analytics, Stored procedures, Performance tuning
  • Lead implementation of secure vendor file transfer and ingestion solutions using AWS Transfer Family
  • Design and implement database migration and replication pipelines using AWS Database Migration Service (DMS)
  • Build and manage workflow orchestration using Apache Airflow or similar orchestration tools
  • Analyze data quality, transformation logic, and pipeline performance using SQL and data analysis techniques
  • Troubleshoot and resolve production data pipeline and integration issues across AWS services
  • Provide technical guidance to development team members on: AWS best practices, Cost optimization, Performance optimization
  • Partner with enterprise architecture, security, and compliance teams to ensure SOX and regulatory compliance
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right