Spark Developer, Citi

Spark Developer

Citi

Location:
India, Chennai

Category:
IT - Software Development

Contract Type:
Employment contract

Salary:

Not provided

Save Job

Apply Position

Job Description:

ETL Developer will be responsible for designing, implementing, and optimizing distributed data processing jobs to handle large-scale data in Hadoop Distributed File System(HDFS) using Apache Spark and Python. This role requires deep understanding of data engineering principles, proficiency in Python and hands-on experience with Spark and Hadoop ecosystems. Developer will collaborate with data engineers, analysts, and business stakeholders to process, transform, and drive insights and data-driven decisions.

Job Responsibility:

design and implement of Spark applications to process and transform large datasets in HDFS
develop ETL pipelines in Spark using Python for data ingestion, cleaning, aggregation, and transformations
optimize Spark jobs for efficiency, reducing run time and resource usage
finetune memory management, caching, and partitioning strategies for optimal performance
load data from different sources into HDFS, ensuring data accuracy and integrity
integrate Spark applications with Hadoop frameworks like Hive, Sqoop etc.
troubleshoot and debug Spark job failures, monitor job logs, and Spark UI to identify issues

Requirements:

2-5 years of relevant experience
experience in programming/debugging used in business applications
working knowledge of industry practice and standards
comprehensive knowledge of specific business area for application development
working knowledge of program languages
consistently demonstrates clear and concise written and verbal communication
expertise in handling complex large-scale warehouse environments
hands-on experience writing complex SQL queries, exporting and importing large amounts of data using utilities
bachelor's degree in a quantitative field (such as engineering, computer science) or equivalent experience

Additional Information:

Job Posted:
November 21, 2025

Employment Type:

Fulltime

Work Type:

Hybrid work

View All Jobs In This Company

Job Link Share:

Welcome to CrawlJobs.com –
Your Global Job Discovery Platform

At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.