CrawlJobs Logo

Engineer - PySpark/SQL

barclays.co.uk Logo

Barclays

Location Icon

Location:
India , Pune

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

Join us as an Engineer - PySpark/SQL responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards. You'll spearhead the evolution of our digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionise our digital offerings, ensuring unparalleled customer experiences.

Job Responsibility:

  • Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data
  • Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures
  • Development of processing and analysis algorithms fit for the intended data complexity and volumes
  • Collaboration with data scientist to build and deploy machine learning models

Requirements:

  • Hands on experience in pyspark and strong knowledge on Dataframes, RDD and SparkSQL
  • Hands on Experience in developing, testing and maintaining applications on AWS Cloud
  • Strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake formation, Athena)
  • Design and implement scalable and efficient data transformation/storage solutions using Snowflake
  • Experience in Data ingestion to Snowflake for different storage format such Parquet, Iceberg, JSON, CSV etc
  • Experience in using DBT (Data Build Tool) with snowflake for ELT pipeline development
  • Experience in Writing advanced SQL and PL SQL programs
  • Hands On Experience for building reusable components using Snowflake and AWS Tools/Technology
  • Should have worked at least on two major project implementations
  • Exposure to data governance or lineage tools such as Immuta and Alation is added advantage
  • Experience in using Orchestration tools such as Apache Airflow or Snowflake Tasks is added advantage
  • Knowledge on Abinitio ETL tool is a plus
  • Ability to engage with Stakeholders, elicit requirements/ user stories and translate requirements into ETL components
  • Ability to understand the infrastructure setup and be able to provide solutions either individually or working with teams
  • Good knowledge of Data Marts and Data Warehousing concepts
  • Resource should possess good analytical and Interpersonal skills
  • Implement Cloud based Enterprise data warehouse with multiple data platform along with Snowflake and NoSQL environment to build data movement strategy

Nice to have:

  • Exposure to data governance or lineage tools such as Immuta and Alation is added advantage
  • Experience in using Orchestration tools such as Apache Airflow or Snowflake Tasks is added advantage
  • Knowledge on Abinitio ETL tool is a plus
  • Ability to engage with Stakeholders, elicit requirements/ user stories and translate requirements into ETL components
  • Ability to understand the infrastructure setup and be able to provide solutions either individually or working with teams
  • Good knowledge of Data Marts and Data Warehousing concepts
  • Resource should possess good analytical and Interpersonal skills
  • Implement Cloud based Enterprise data warehouse with multiple data platform along with Snowflake and NoSQL environment to build data movement strategy
What we offer:
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution

Additional Information:

Job Posted:
May 05, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Icon
Barclays - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Engineer - PySpark/SQL

New

Engineer - PySpark/SQL

Join us as an Engineer - PySpark/SQL responsible for supporting the successful d...
Location
Location
India , Pune
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hands on experience in pyspark and strong knowledge on Dataframes, RDD and SparkSQL
  • Hands on Experience in developing, testing and maintaining applications on AWS Cloud
  • Strong hold on AWS Data Analytics Technology Stack (Glue, S3, Lambda, Lake formation, Athena)
  • Design and implement scalable and efficient data transformation/storage solutions using Snowflake
  • Experience in Data ingestion to Snowflake for different storage format such Parquet, Iceberg, JSON, CSV etc
  • Experience in using DBT (Data Build Tool) with snowflake for ELT pipeline development
  • Experience in Writing advanced SQL and PL SQL programs
  • Hands On Experience for building reusable components using Snowflake and AWS Tools/Technology
  • Should have worked at least on two major project implementations
Job Responsibility
Job Responsibility
  • Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data
  • Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures
  • Development of processing and analysis algorithms fit for the intended data complexity and volumes
  • Collaboration with data scientist to build and deploy machine learning models
What we offer
What we offer
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
  • Fulltime
Read More
Arrow Right

Data Engineer

Build and operate large-scale healthcare data pipelines across batch workflows, ...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Information Technology, or a related field with 5-9 years of experience
  • Advanced Python, PySpark, and SQL (window functions, complex joins, MERGE patterns, optimization)
  • Hands-on Databricks and Airflow experience in enterprise environments
  • Experience with cloud data platforms (AWS), object storage, and secure secret handling
  • Strong data quality engineering, monitoring, and troubleshooting in regulated data contexts
  • Solid understanding of ETL orchestration, dependency management, and SLA-driven delivery
Job Responsibility
Job Responsibility
  • Design and maintain PySpark/SQL pipelines in Databricks for landing, unified, unstitched, and published data layers
  • Build and support Airflow DAGs for scheduling, dependencies, retries, and production operations
  • Implement metadata/config-driven frameworks for ingestion, transformation, and rule-based processing
  • Develop robust data quality controls, DQ summaries, failure handling, and alerting workflows
  • Manage batch/process audit logs, run status tracking, release flags, and operational reporting
  • Integrate multi-source data (files, APIs, cloud storage, and relational systems) into governed Delta/Spark tables
  • Optimize pipeline performance using partitioning, parallelization, and query tuning
  • Collaborate on schema evolution, business-rule onboarding, and production support
Read More
Arrow Right

Data Engineering Lead

Role: Data Engineering Lead. Location: New York, NY. FTE ONLY. Must Have Technic...
Location
Location
United States , New York
Salary
Salary:
138000.00 USD / Year
realign-llc.com Logo
Realign
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • AWS Data Engineering Services (EMR/Glue, Redshift, Aurora, S3, Lambda)
  • Spark
  • Python
  • Collibra
  • Snowflake/Databricks
  • Tableau
  • 4–15 years of professional data engineering experience
  • Strong Python, SQL, and Spark (PySpark) skills, and/or Kafka
  • Snowflake (Snowpipe, Tasks, Streams) as a complementary warehouse
  • Databricks (Delta formats, workflows, cataloging) or equivalent Spark platforms
Job Responsibility
Job Responsibility
  • Ingest and model data from APIs, files/SFTP, and relational sources
  • implement layered architectures (raw/clean/serving) using PySpark/SQL and dbt, Python
  • Design and operate pipelines with Prefect (or Airflow), including scheduling, retries, parameterization, SLAs, and well documented runbooks
  • Build on cloud data platforms, leveraging S3/ADLS/GCS for storage and a Spark platform (e.g., Databricks or equivalent) for compute
  • manage jobs, secrets, and access
  • Publish governed data services and manage their lifecycle with Azure API Management (APIM) authentication/authorization, policies, versioning, quotas, and monitoring
  • Enforce data quality and governance through data contracts, validations/tests, lineage, observability, and proactive alerting
  • Optimize performance and cost via partitioning, clustering, query tuning, job sizing, and workload management
  • Uphold security and compliance (e.g., PII handling, encryption, masking) in line with firm standards
  • Collaborate with stakeholders (analytics, AI engineering, and business teams) to translate requirements into reliable, production ready datasets
  • Fulltime
Read More
Arrow Right
New

Master Tech

Nationally Branded Auto Repair Center is looking for the Most Important Part of ...
Location
Location
United States , South Elgin
Salary
Salary:
55000.00 - 95000.00 USD / Year
meineke.com Logo
Meineke
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous experience as an automotive Master Technician or B Tech
  • Expert Knowledge of diagnostic and repair equipment
  • Strong mechanical aptitude and troubleshooting skills
  • Deadline and detail-oriented
  • Exhaust & welding experience preferred
  • Must have own tools
Job Responsibility
Job Responsibility
  • Diagnose and repair automotive vehicles
  • Perform routine vehicle tune-ups and maintenance
  • Provide labor and time estimates for automotive repairs
  • Inspect and test vehicles for necessary adjustments
  • Drive Manual Transmission Vehicles
What we offer
What we offer
  • 401(k) matching
  • Competitive salary
  • Health insurance
  • Paid time off
  • Bonus Potential
  • Opportunity for Continued Career Growth
  • Friendly Team Environment
  • Certificate Reimbursement
  • Fulltime
Read More
Arrow Right
New

Housing Support Worker

We are seeking a dedicated and motivated Housing Support Worker to join our Keep...
Location
Location
United Kingdom , Rhyl
Salary
Salary:
Not provided
pavers.co.uk Logo
Pavers
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full UK driving licence and access to a vehicle
  • Right to work in the UK (sponsorship visas are not available)
Job Responsibility
Job Responsibility
  • Help people sustain their accommodation and prevent homelessness
  • Provide flexible and person-centred support to address challenges such as mental ill health, substance use, or other complex issues
  • Aim to improve overall health, wellbeing, and housing stability
What we offer
What we offer
  • Enhanced holiday entitlement
  • Employee assistance programme
  • Additional Stori Day
  • Occupational sick pay
  • Pension scheme
  • Refer a friend scheme
  • Staff recognition scheme
  • Flexible working hours
  • Comprehensive training
  • Parttime
Read More
Arrow Right
New

Staff Pharmacist PT

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , Fultondale
Salary
Salary:
60.00 - 76.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
August 30, 2026
Flip Icon
Requirements
Requirements
  • Active Pharmacy License in the state in which you are employed
  • Not on the DEA Excluded Parties List
  • Immunization Certification through an accredited organization (i.e. APhA)
  • Free of pending felony charges or convictions for criminal offenses involving controlled substances
Job Responsibility
Job Responsibility
  • Patient Safety
  • Pharmacy Professional Practice
  • Regulatory Requirements
  • Quality Assurance
  • Customer Service
  • Personnel Management
  • Inventory Management
  • Financial Profitability
  • Loss Prevention
  • Workflow Management
What we offer
What we offer
  • dental
  • vision
  • wellness resources
  • employee discounts
  • access to certain voluntary benefits
  • other programs
  • Parttime
Read More
Arrow Right
New

Automotive Technician

PARC Auto dba Meineke is hiring A/B Technicians, Master Techs, and Mechanics who...
Location
Location
United States , Molalla
Salary
Salary:
45000.00 - 100000.00 USD / Year
meineke.com Logo
Meineke
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience
  • Solid background in general vehicle maintenance and repair. All experience levels considered
  • Certifications
  • ASE certs are a plus — not required
  • Attitude
  • You show up, you do good work, and you don't leave your teammates hanging.
Job Responsibility
Job Responsibility
  • Brakes, alignments, tires, oil changes, flushes
  • A/C, cooling, exhaust, and suspension repair
  • General vehicle maintenance across a high volume of cars
  • Other duties as assigned.
What we offer
What we offer
  • 401(k)
  • 401(k) matching
  • Competitive salary
  • Dental insurance
  • Employee discounts
  • Health insurance
  • Paid time off
  • Training & development
  • Vision insurance
  • Wellness resources
  • Fulltime
Read More
Arrow Right
New

Staff Pharmacist FT

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , Lancaster
Salary
Salary:
65.00 - 81.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
July 20, 2026
Flip Icon
Requirements
Requirements
  • Active Pharmacy License in the state in which you are employed
  • Not on the DEA Excluded Parties List
  • Immunization Certification through an accredited organization (i.e. APhA)
  • Free of pending felony charges or convictions for criminal offenses involving controlled substances
Job Responsibility
Job Responsibility
  • Supporting the Pharmacy Manager in leading and directing your Pharmacy Technician Support Staff
  • Accountable for supporting the management, oversight and operation of all aspects within your pharmacy
  • Patient Safety
  • Pharmacy Professional Practice
  • Regulatory Requirements
  • Quality Assurance
  • Customer Service
  • Personnel Management
  • Inventory Management
  • Financial Profitability
What we offer
What we offer
  • medical, dental, and vision coverage
  • paid time off
  • retirement savings options
  • wellness programs
  • bonus, commission or short-term incentive program
  • Fulltime
Read More
Arrow Right