CrawlJobs Logo

Big Data Engineer

https://www.citi.com/ Logo

Citi

Location Icon

Location:
India , Chennai

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

We are seeking a talented and experienced Big Data Hadoop Developer to join our growing data engineering team. The ideal candidate will have 4-6 years of hands-on experience designing, developing, and optimizing big data solutions using the Hadoop ecosystem, with a strong focus on Apache Spark. You will be responsible for building and maintaining scalable data pipelines, processing large datasets, and collaborating with data scientists and analysts to deliver insights.

Job Responsibility:

  • Design, develop, and maintain robust and scalable ETL processes and data pipelines using Apache Hadoop and Apache Spark
  • Write efficient, clear, and well-documented code primarily in Scala, Python, or PySpark for big data processing
  • Implement data ingestion, transformation, and loading routines from various sources into Hadoop Distributed File System (HDFS) and other big data stores
  • Optimize existing Spark jobs and Hadoop ecosystem components for performance and scalability
  • Collaborate with data architects, data scientists, and other stakeholders to understand data requirements and translate them into technical solutions
  • Ensure data quality, integrity, and security across all big data platforms
  • Participate in code reviews, testing, and deployment of big data applications
  • Troubleshoot and resolve issues in big data environments
  • Stay up-to-date with the latest trends and technologies in the big data ecosystem

Requirements:

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
  • 3-4 years of professional experience in Big Data development
  • Proven experience with the Hadoop ecosystem, including HDFS, YARN, Hive, and other related technologies
  • Hands on experience in SQL and shell scripting
  • Strong expertise in Apache Spark for data processing and analysis
  • Proficiency in at least one of the following programming languages: Scala, Python, or PySpark
  • Experience with building and optimizing large-scale data pipelines
  • Familiarity with data warehousing concepts and ETL methodologies
  • Solid understanding of distributed computing principles
  • Excellent problem-solving skills and attention to detail
  • Ability to work independently and as part of a collaborative team

Nice to have:

  • Experience with cloud-based big data services (e.g., AWS EMR, Azure HDInsight, Google Cloud Dataproc)
  • Experience with Databricks platform
  • Knowledge of other big data tools like Kafka, HBase, Flink, or Presto
  • Experience with SQL and NoSQL databases
  • Familiarity with CI/CD practices and tools (e.g., Git, Jenkins)
  • Understanding of machine learning concepts and how they apply to big data

Additional Information:

Job Posted:
May 05, 2026

Employment Type:
Fulltime
Work Type:
Hybrid work
Icon
Citi - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Big Data Engineer

Big Data Engineer

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Understanding of the Big Data architecture and the ability to trouble shoot performance and/or development issues on Hadoop (Cloudera preferably)
  • Experience working with Hive, Impala, Kudu, HBase, Spark for data curation/conformance related work
  • proficiency in Spark for development work related to curation/conformance
  • Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event-based messaging
  • Strong data analysis skills and the ability to slice and dice the data as needed for business reporting
  • Experience working in an agile environment with a fast-paced changing requirement
  • Excellent planning and organizational skills and Strong Communication skills
  • Relational SQL and NoSQL database integration and data distribution principles experience
  • Experience with API development and use of JSON/XML/Hypermedia data formats
  • Align to Engineering Excellence Development principles and standards
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Senior Engineering Manager, Big Data

Checkr is looking for a Senior Engineering Manager to lead the Criminal Data tea...
Location
Location
United States , Denver
Salary
Salary:
201000.00 - 237000.00 USD / Year
https://checkr.com Logo
Checkr
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years as an engineering manager
  • 8+ years as an engineer
  • Exceptional verbal and written communication skills
  • Unparalleled bar for quality (data quality metrics, QC gates, data governance, automated regression test suites, data validations, etc)
  • Experience working on data products at scale and understanding the legal, human impact, and technical nuances of supporting a highly regulated product
  • Experience designing and maintaining: Real-time & batch processing data pipelines serving up billions of data points
  • Normalizing and cleansing data across a medallion lakehouse architecture
  • Systems that rely on high-volume, low-latency messaging infrastructure (e.g. Kafka or similar)
  • Highly tolerant production systems with streamlined operations (data lineage, logging, telemetry, alerting, etc.)
  • Familiarity with AWS Glue, OpenSearch, EMR, etc
Job Responsibility
Job Responsibility
  • Drive a motivating technical vision for the team
  • Partner closely with product management to solve business problems
  • Work with the team to build a world-class architecture that can scale into the next phase of Checkr’s growth
  • Hire the best talent and continue to raise the bar for the team
  • Represent the team in planning and product meetings
  • Optimize engineering processes and policies to drive velocity and quality
What we offer
What we offer
  • A fast-paced and collaborative environment
  • Learning and development allowance
  • Competitive compensation and opportunity for advancement
  • 100% medical, dental, and vision coverage
  • Up to 25K reimbursement for fertility, adoption, and parental planning services
  • Flexible PTO policy
  • Monthly wellness stipend, home office stipend
  • Fulltime
Read More
Arrow Right

Senior Engineering Manager, Big Data

Checkr is looking for a Senior Engineering Manager to lead the Criminal Data tea...
Location
Location
United States , San Francisco
Salary
Salary:
238000.00 - 280000.00 USD / Year
https://checkr.com Logo
Checkr
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years as an engineering manager
  • 8+ years as an engineer
  • Exceptional verbal and written communication skills
  • Unparalleled bar for quality (data quality metrics, QC gates, data governance, automated regression test suites, data validations, etc)
  • Experience working on data products at scale and understanding the legal, human impact, and technical nuances of supporting a highly regulated product
  • Experience designing and maintaining: Real-time & batch processing data pipelines serving up billions of data points
  • Normalizing and cleansing data across a medallion lakehouse architecture
  • Systems that rely on high-volume, low-latency messaging infrastructure (e.g. Kafka or similar)
  • Highly tolerant production systems with streamlined operations (data lineage, logging, telemetry, alerting, etc.)
  • Familiarity with AWS Glue, OpenSearch, EMR, etc
Job Responsibility
Job Responsibility
  • Drive a motivating technical vision for the team
  • Partner closely with product management to solve business problems
  • Work with the team to build a world-class architecture that can scale into the next phase of Checkr’s growth
  • Hire the best talent and continue to raise the bar for the team
  • Represent the team in planning and product meetings
  • Optimize engineering processes and policies to drive velocity and quality
What we offer
What we offer
  • A fast-paced and collaborative environment
  • Learning and development allowance
  • Competitive compensation and opportunity for advancement
  • 100% medical, dental, and vision coverage
  • Up to 25K reimbursement for fertility, adoption, and parental planning services
  • Flexible PTO policy
  • Monthly wellness stipend, home office stipend
  • Fulltime
Read More
Arrow Right

Big Data / Scala / Python Engineering Lead

The Applications Development Technology Lead Analyst is a senior level position ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least two years (Over all 10+ hands on Data Engineering experience) of experience building and leading highly complex, technical data engineering teams
  • Lead data engineering team, from sourcing to closing
  • Drive strategic vision for the team and product
  • Experience managing an data focused product, ML platform
  • Hands on experience relevant experience in design, develop, and optimize scalable distributed data processing pipelines using Apache Spark and Scala
  • Experience managing, hiring and coaching software engineering teams
  • Experience with large-scale distributed web services and the processes around testing, monitoring, and SLAs to ensure high product quality
  • 7 to 10+ years of hands-on experience in big data development, focusing on Apache Spark, Scala, and distributed systems
  • Proficiency in Functional Programming: High proficiency in Scala-based functional programming for developing robust and efficient data processing pipelines
  • Proficiency in Big Data Technologies: Strong experience with Apache Spark, Hadoop ecosystem tools such as Hive, HDFS, and YARN
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Fulltime
Read More
Arrow Right

Big Data Engineering Developer

The Applications Development Senior Programmer/Lead Analyst is an senior level p...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong solid understanding of the Big Data architecture and the ability to trouble shoot performance and/or development issues on Hadoop (Cloudera preferably)
  • 9+ years of experience working with Hive, Impala and Hbase, Kudu Spark for data curation/conformance related work
  • Strong proficiency in Spark for development work related to curation/conformance. Strong Scala developer (with previous Java background) preferred.
  • Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event based messaging
  • Strong data analysis skills and the ability to slice and dice the data as needed for business reporting
  • Experience working in an agile environment with a fast paced changing requirements.
  • Excellent planning and organizational skills
  • Strong Communication skills
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Big Data Engineering Lead

The Senior Big Data engineering lead will play a pivotal role in designing, impl...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master’s degree in Computer Science, Information Technology, or related field
  • Atleast 10 -12 years overall software development experience on majorly working with handling application with large scale data volumes from ingestion, persistence and retrieval
  • Deep understanding of big data technologies, including Hadoop, Spark, Kafka, Flink, NoSQL databases, etc.
  • Experience with Bigdata technologies Developer Hadoop, Apache Spark, Python, PySpark
  • Strong programming skills in languages such as Java, Scala, or Python
  • Excellent problem-solving skills with a knack for innovative solutions
  • Strong communication and leadership abilities
  • Proven ability to manage multiple projects simultaneously and deliver results
Job Responsibility
Job Responsibility
  • Lead the design and development of a robust and scalable big data architecture handling exponential data growth while maintaining high availability and resilience
  • Design complex data transformation processes using Spark and other big data technologies using Java, Pyspark or Scala
  • Design and implement data pipelines that ensure data quality, integrity, and availability
  • Collaborate with cross-functional teams to understand business needs and translate them into technical requirements
  • Evaluate and select technologies that improve data efficiency, scalability, and performance
  • Oversee the deployment and management of big data tools and frameworks such as Hadoop, Spark, Kafka, and others
  • Provide technical guidance and mentorship to the development team and junior architects
  • Continuously assess and integrate emerging technologies and methodologies to enhance data processing capabilities
  • Optimize big data frameworks, such as Hadoop, Spark, for performance improvements and reduced processing time across distributed systems
  • Implement data governance frameworks to ensure data accuracy, consistency, and privacy across the organization, leveraging metadata management and data lineage tracking
  • Fulltime
Read More
Arrow Right

Big Data Engineer

Inetum is seeking a seasoned Big Data Engineer to join our team and support the ...
Location
Location
Portugal , Lisbon
Salary
Salary:
Not provided
https://www.inetum.com Logo
Inetum
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science or equivalent
  • Minimum 5 years of experience in a similar role
  • Strong expertise in Hadoop Ecosystem & Data Lake Operations, Big Data Services & Query Engines, Security & Access Control in Distributed Environments, Monitoring, Automation & Platform Lifecycle Management
  • Proven ability to diagnose and resolve complex issues in Big Data environments
  • Strong communication skills to collaborate with cross-functional teams
  • English proficiency: B2-C1 level
Job Responsibility
Job Responsibility
  • Operate, monitor, and support Hadoop-based data lake platforms
  • Provide L3 incident response, deep troubleshooting, and performance tuning for big data components
  • Ensure data integrity, replication, and capacity management within HDFS clusters
  • Develop and automate monitoring and alerting for service health and node performance
  • Collaborate with platform and data teams to onboard new workloads with optimal resource allocation
  • Apply patches, upgrades, and configuration changes to maintain platform security and stability
  • Manage Kerberos authentication, Ranger/Sentry policies, and TLS encryption
  • Work with storage, network, and security teams to optimize throughput and access controls
  • Maintain cluster documentation and share knowledge across teams
What we offer
What we offer
  • Opportunity to grow your expertise
  • Certified Top Employer Europe 2025
  • Fulltime
Read More
Arrow Right

Big Data Engineer

At Citi we’re not just building technology, we’re building the future of banking...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2 - 5 years of relevant experience in Big data platforms such as Hadoop, Hive
  • Proficient in one or more programming languages commonly used in data engineering such as Python, PySpark
  • Database Management: SQL
  • Working knowledge of program languages
  • Consistently demonstrates clear and concise written and verbal communication
  • Bachelor’s degree/University degree or equivalent experience
What we offer
What we offer
  • Programs and services for your physical and mental well-being including access to telehealth options, health advocates, confidential counseling
  • Fulltime
Read More
Arrow Right