CrawlJobs Logo

Spark Developer

https://www.citi.com/ Logo

Citi

Location Icon

Location:
India , Chennai

Category Icon

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

Not provided

Job Description:

ETL Developer will be responsible for designing, implementing, and optimizing distributed data processing jobs to handle large-scale data in Hadoop Distributed File System(HDFS) using Apache Spark and Python. This role requires deep understanding of data engineering principles, proficiency in Python and hands-on experience with Spark and Hadoop ecosystems. Developer will collaborate with data engineers, analysts, and business stakeholders to process, transform, and drive insights and data-driven decisions.

Job Responsibility:

  • design and implement of Spark applications to process and transform large datasets in HDFS
  • develop ETL pipelines in Spark using Python for data ingestion, cleaning, aggregation, and transformations
  • optimize Spark jobs for efficiency, reducing run time and resource usage
  • finetune memory management, caching, and partitioning strategies for optimal performance
  • load data from different sources into HDFS, ensuring data accuracy and integrity
  • integrate Spark applications with Hadoop frameworks like Hive, Sqoop etc.
  • troubleshoot and debug Spark job failures, monitor job logs, and Spark UI to identify issues

Requirements:

  • 2-5 years of relevant experience
  • experience in programming/debugging used in business applications
  • working knowledge of industry practice and standards
  • comprehensive knowledge of specific business area for application development
  • working knowledge of program languages
  • consistently demonstrates clear and concise written and verbal communication
  • expertise in handling complex large-scale warehouse environments
  • hands-on experience writing complex SQL queries, exporting and importing large amounts of data using utilities
  • bachelor's degree in a quantitative field (such as engineering, computer science) or equivalent experience

Additional Information:

Job Posted:
November 21, 2025

Employment Type:
Fulltime
Work Type:
Hybrid work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Spark Developer

Java Spark Developer

Face to Face Interview is required - will not be able to consider non local reso...
Location
Location
United States , Seattle
Salary
Salary:
Not provided
enormousenterprise.com Logo
Enormous Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years overall software development experience
  • at least 3+ years of experience in large scale data platforms
  • excellent expertise in Spark and Java
  • good understanding of containerization using Docker and Kubernetes
  • understanding of version control like Git and CI/CD workflow
  • understanding of Parquet/Avro/Hive/Iceberg
  • well experienced in debugging and troubleshooting
  • good communication and excellent collaboration skills to interact with internal customers
Job Responsibility
Job Responsibility
  • Help support Data Replication Service (Spark jobs) and develop features/bug fixes
  • help support SparkCp (a library) in debugging and fixing issues encountered by DRS
  • help support the CDH Migration Tool in debugging and fixing issues encountered by customers
  • implement monitoring/alerting improvements to the 3 tools mentioned previously for better supportability
  • perform onboarding tasks to help move customer data
  • monitor and address user requests and issues as they arise
Read More
Arrow Right

Java - Spark Developer C11

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Engineering, or a related field
  • Experience in software development with a strong focus on Core Java and Spark
  • Hands-on experience with Spring Batch and Spring Boot
  • Hands-on experience with Apache Spark (with Java)
  • Experience with caching solutions like Pinot, Redis etc.
  • Strong knowledge of Oracle DB, including schema design and query optimization
  • Strong problem-solving skills and the ability to work independently
  • Excellent communication and teamwork skills
  • Strong computer science fundamentals in data structures, algorithms, databases, and operating systems
  • Highly experienced with Unix based operating systems and shell scripting
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
What we offer
What we offer
  • Global Benefits
  • Best-in-class benefits
  • Equal opportunity and affirmative action employer
  • Fulltime
Read More
Arrow Right

Java Spark Developer

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7 to 12 years of Application development experience through full lifecycle SME for UI architecture patterns - Micro Frontend, NX
  • Experience with Core Java/J2EE Application with complete command over OOPs and Design Patterns
  • Commendable in Data Structures and Algorithms
  • Worked on Core Application Development of complex size encompassing all areas of Java/J2EE
  • Thorough knowledge and hands on experience in following technologies Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig, Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
  • Cloudera certification (CCDH) is an added advantage
  • Implemented or part complex project execution in Big Data Spark eco system, where processing volumes of data thorough understanding of distributed processing and integrated applications
  • Exposure to ETL and BI tools will be good
  • Work in an agile environment following through the best practices of agile Scrum
  • Expertise in designing and optimizing the software solutions for performance and stability
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Senior Java Spark Developer

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7 to 12 years of Application development experience through full lifecycle
  • SME for UI architecture patterns - Micro Frontend, NX
  • Experience with Core Java/J2EE Application with complete command over OOPs and Design Patterns
  • Commendable in Data Structures and Algorithms
  • Thorough knowledge and hands on experience in Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig, Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
  • Cloudera certification (CCDH) is an added advantage
  • Implemented or part of complex project execution in Big Data Spark eco system
  • Exposure to ETL and BI tools
  • Work in an agile environment following through the best practices of agile Scrum
  • Expertise in designing and optimizing software solutions for performance and stability
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts
  • Consult with users, clients, and other technology groups on issues
  • Apply fundamental knowledge of programming languages for design specifications
  • Analyze applications to identify vulnerabilities and security issues
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience
  • Operate with a limited level of direct supervision
  • Exercise independence of judgement and autonomy
  • Act as SME to senior stakeholders and/or other team members
What we offer
What we offer
  • Best-in-class benefits
  • Global Benefits
  • Fulltime
Read More
Arrow Right

Big Data Spark Development Vice President

We are seeking a Software/Application Dev Lead Analyst for Global Data Platform ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or master's degree in computer science, Information Technology, or equivalent
  • Minimum 15 years of work experience
  • At least 8 years of experience of leading Enterprise Bigdata Development and Data Science (Analytical) teams
  • Experience developing Applications for High volume instanced payments and/or AML industry is a significant plus
  • Experience working in Banking technology is preferred
  • Very strong Engineer excellence knowledge and experience like Data patterns, Pipelines, DQs, Recons, Governance, Scalable application traits and cloud portable coding
  • Project management, interpersonal, and communication skills, with high attention to details
  • Must-Have: Experience in building enterprise-grade transaction management and AML system with focus on recovery, stability, reliability, scalability and performance
  • Must-Have: Strong hand-on seasoned engineer/lead in Bigdata data store application building using enterprise SDLC/Agile methodology, Data modelling/Architect, supporting Operational and production systems
  • Must-Have: Strong technical skills and hand-on in Spark/Scala, Kafka, Hadoop, Couchbase, HBASE and Snowflake
Job Responsibility
Job Responsibility
  • Manage one or more Applications Development teams to accomplish established goals as well as conduct personnel duties for team (e.g. performance evaluations, hiring and disciplinary actions)
  • Utilize in-depth knowledge and skills across multiple Applications Development areas to provide technical oversight across systems and applications
  • Review and analyze proposed technical solutions for projects
  • Contribute to formulation of strategies for applications development and other functional areas
  • Develop comprehensive knowledge of how areas of business integrate to accomplish business goals
  • Provide evaluative judgment based on analysis of factual data in complicated and unique situations
  • Impact the Applications Development area through monitoring delivery of end results, participate in budget management, and handling day-to-day staff management issues, including resource management and allocation of work within the team/project
  • Ensure essential procedures are followed and contribute to defining standards negotiating with external parties when necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency, as well as effectively supervise the activity of others and create accountability with those who fail to maintain these standards
  • Fulltime
Read More
Arrow Right

Senior Spark Java Developer

Seeking a senior Spark Java Developer with expertise in big data processing, Pyt...
Location
Location
Canada , Mississauga
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10 years + of experience in software development
  • at least 5 years of experience in Spark Java and Big data frameworks
  • strong proficiency in Python and Spark Java
  • knowledge of core spark concepts (RDDs, Dataframes, Spark Streaming, etc)
  • experience working in financial markets, risk management, and financial instruments
  • familiarity with market risk concepts including VaR, Greeks, scenario analysis, and stress testing
  • hands-on experience with Hadoop, Spark
  • proficiency in Git, Jenkins, and CI/CD pipelines
  • excellent problem-solving skills
  • strong mathematical and analytical mindset
Job Responsibility
Job Responsibility
  • Develop and optimize scalable Spark Java-based data pipelines for processing and analyzing large-scale financial data
  • design and implement distributed computing solutions for risk modeling, pricing, and regulatory compliance
  • ensure efficient data storage and retrieval using Big Data
  • implement best practices for Spark performance tuning including partition, caching, and memory management
  • maintain high code quality through testing, CI/CD pipelines, and version control (Git, Jenkins)
  • work on batch processing frameworks for Market risk analytics
  • Fulltime
Read More
Arrow Right

Java Spark Applications Development - Assistant Vice President

The Applications Development Lead is an intermediate level position responsible ...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8 to 12 years of Application development experience using Java and spark
  • Experience with Core Java/J2EE Application with complete command over OOPs and Design Patterns
  • Commendable in Data Structures and Algorithms
  • Thorough knowledge and hands on experience in Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig, Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
  • Cloudera certification (CCDH) is an added advantage
  • Implemented or part of complex project execution in Big Data Spark eco system
  • Exposure to ETL and BI tools
  • Work in an agile environment following through the best practices of agile Scrum
  • Expertise in designing and optimizing the software solutions for performance and stability
  • Expertise in troubleshooting and problem solving
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Application Developer - Java Spark Big Data

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Engineering, or a related field
  • Extensive development expertise in building the high scaled and performant software platforms for data computation and processing
  • Experience in software development with a strong focus on Core Java and Spark
  • Expert level knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns
  • Hands-on experience with Apache Spark (with Java)
  • Experience with caching solutions like Pinot, Redis etc.
  • Strong knowledge of Oracle DB, including schema design and query optimization
  • Strong problem-solving skills and the ability to work independently
  • Excellent communication and teamwork skills
  • Strong computer science fundamentals in data structures, algorithms, databases, and operating systems
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right