CrawlJobs Logo

Technical Lead, Spark (Java)

cloudera.com Logo

Cloudera

Location Icon

Location:

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises. The Data Platform Pillar is the bedrock of Cloudera’s technology, where we design and build the core components that let our customers store, manage, and process data with unmatched scalability, security, and performance. Cloudera is seeking a Technical Lead, Spark (Java) with strong distributed systems expertise to work on the Cloudera distribution of Apache Spark and Livy. The role involves building enterprise-grade systems for customers running Spark on thousands of nodes and processing petabytes of data. We are looking for a passionate engineer eager to enhance a product already supporting major production systems and to drive the next-generation Data Engineering experience. You will collaborate with a distributed team across the United States and Hungary, including multiple Apache Spark committers.

Job Responsibility:

  • Design new features for Cloudera’s data engineering experience, and take them from prototypes to leading a team to deliver the feature in production at scale
  • Contribute to Apache Spark, Livy
  • Develop new features in Scala/Java/Python on modern platforms
  • Gain expertise in distributed data processing, from SQL planners and optimizers, to data layout and table formats like Apache Parquet and Iceberg, to fault tolerance in distributed systems
  • Gain a solid understanding and deep technical knowledge of components across the Cloudera Data Engineering Experience stack, but focusing on Iceberg and Spark
  • Get to work on large-scale distributed systems, from 100s to 1000s of nodes, in production clusters
  • Debug system-level deployment issues, root cause analysis, perform system test analysis, and resolve failures
  • Work on improving internal infrastructure
  • Collaborate with other team members and stakeholders

Requirements:

  • 8-10+ years of professional software development
  • Experience leading and delivering complex product enhancements
  • Strong understanding of at least one of the following languages: Java, Scala, Python
  • Experience with systems design, development
  • Passionate about programming, clean coding habits, attention to detail, and focus on quality
  • Strong oral and written communication skills
  • Strong ability to research and solve problems independently without constant supervision
  • Open-minded, desire to learn new things and build great products
  • Experience with distributed systems

Nice to have:

  • Experience with SQL planners
  • Experience with using/developing Apache Spark, Livy or other related technologies
  • Experience with large-scale, distributed systems design and development with an understanding of scaling, performance, and scheduling
  • Solid experience with at least one cloud
  • Contributors to open-source projects
What we offer:
  • Generous PTO Policy
  • Support work life balance with Unplugged Days
  • Flexible WFH Policy
  • Mental & Physical Wellness programs
  • Phone and Internet Reimbursement program
  • Access to Continued Career Development
  • Comprehensive Benefits and Competitive Packages
  • Paid Volunteer Time
  • Employee Resource Groups

Additional Information:

Job Posted:
January 26, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Technical Lead, Spark (Java)

Java / Spark Senior Lead Developer

The Applications Development Technology Lead Analyst is a senior level position ...
Location
Location
India , Pune/Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of relevant experience in Apps Development or systems analysis role
  • Extensive experience system analysis and in programming of software applications
  • Experience in managing and implementing successful projects
  • Subject Matter Expert (SME) in at least one area of Applications Development
  • Ability to adjust priorities quickly as circumstances dictate
  • Demonstrated leadership and project management skills
  • Consistently demonstrates clear and concise written and verbal communication
  • Highly experienced and skilled Java technical lead with 10+years of experience with software building and platform engineering
  • Extensive development expertise in building the high scaled and performant software platforms for data computation and processing
  • Expert level knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
What we offer
What we offer
  • Global Benefits
  • We bring the best to our people. We put our employees first and provide the best-in-class benefits they need to be well, live well and save well.
  • Fulltime
Read More
Arrow Right

Java Spark Senior Lead Developer

The Applications Development Technology Lead Analyst is a senior level position ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of relevant experience in Apps Development or systems analysis role
  • Extensive experience system analysis and in programming of software applications
  • Experience in managing and implementing successful projects
  • Subject Matter Expert (SME) in at least one area of Applications Development
  • Ability to adjust priorities quickly as circumstances dictate
  • Demonstrated leadership and project management skills
  • Consistently demonstrates clear and concise written and verbal communication
  • Highly experienced and skilled Java technical lead with 10+ years of experience with software building and platform engineering
  • Hands-on development expertise in Java with Spark
  • Extensive knowledge of HDFS, Hive (4-5 years experience)
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals
  • Identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming
  • Ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made
  • Fulltime
Read More
Arrow Right

Java-Python Lead Developer

We're seeking a passionate and highly skilled Lead Java Developer to join our ta...
Location
Location
Canada , Mississauga
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of hands-on experience developing high-performance Java applications (Java 11+ preferred) with a strong foundation in core Java concepts, OOP, and OOAD
  • Proven experience building and maintaining data pipelines using technologies like Kafka, Apache Spark, or Apache Flink
  • Familiarity with event-driven architectures and experience in developing real-time, low-latency applications
  • Deep understanding of distributed systems concepts and experience with MPP platforms such as Trino (Presto), Pinot or Snowflake
  • Experience deploying and managing applications on container orchestration platforms like Kubernetes, OpenShift, or ECS
  • Demonstrated ability to lead and mentor engineering teams, communicate complex technical concepts effectively, and collaborate across diverse teams
  • Excellent problem-solving skills and a data-driven approach to decision-making
  • Python knowledge with exposure to Django or Flask
  • Bachelor’s degree/University degree or equivalent experience
  • Master’s degree preferred
Job Responsibility
Job Responsibility
  • Provide technical leadership and mentorship to a team of data engineers, fostering a culture of collaboration, innovation, and continuous learning
  • Lead the design and development of highly scalable, low-latency, fault-tolerant data pipelines and platform components that meet Citi's evolving business needs
  • Stay abreast of emerging open-source data technologies and evaluate their suitability for integration into Citi's platform
  • Continuously identify and implement performance optimizations across the data platform to ensure optimal efficiency and responsiveness
  • Partner closely with stakeholders across engineering, data science, and business teams to understand requirements and translate them into robust technical solutions
  • Drive the timely and high-quality delivery of data platform projects, adhering to agile methodologies and best practices
  • Fulltime
Read More
Arrow Right

Capital Markets Technical Lead

The Capital Markets Technical Lead is a senior-level position responsible for es...
Location
Location
Canada , Mississauga
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of relevant experience in Apps Development or systems analysis role
  • Experience with Bigdata/Hadoop, Spark, SQL and scripting technology is required
  • Experience with Java and Microservices is a plus
  • Experience with data quality rule engine, data analytics, issue resolution lifecycle related to Capital Markets flow is a plus
  • Knowledge of Data Quality and Controls is a plus
  • Extensive experience system analysis and in programming of software applications
  • Experience in managing and implementing successful projects
  • Subject Matter Expert (SME) in at least one area of Applications Development
  • Ability to adjust priorities quickly as circumstances dictate
  • Demonstrated leadership and project management skills
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
What we offer
What we offer
  • Accessibility support for persons with disabilities
  • Equal opportunity employment policies
  • Best-in-class global benefits
  • Fulltime
Read More
Arrow Right

Java Tech Lead

The Stress testing Team is responsible for delivering Stress testing related sol...
Location
Location
Canada , Toronto
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8-10 years of industry experience
  • 7+ years of engineering management
  • 4+ years of managing individual contributors
  • 2+ years of managing managers
  • BS or MS in Computer Science or related technical field
  • Graduate in STEM or Finance disciplines
  • Broad experience in relational and document databases, data structures, caching, and reporting
  • Develop enterprise-grade applications using Java 8/JEE (and higher), No SQL, Spring
  • Experience managing and mentoring junior and senior engineers across multiple time zones
  • Experience attracting and hiring top talents
Job Responsibility
Job Responsibility
  • Lead team(s) of passionate engineers supporting multiple systems
  • Set up and maintain processes that empower the team to move quickly
  • Be hands-on in technical architecture and reviews
  • Independently own and drive multiple critical work streams
  • Establish and drive adoption of design and coding best practices within your team
  • Liaise with senior stakeholders across Risk, Finance, and Front Office business organizations, and external Market Regulators
  • Mentor and guide professional development of analysts & engineers on your team
  • Collaborate with and across teams to design, build, and deliver high-quality software
What we offer
What we offer
  • Global Benefits
  • Best-in-class benefits
  • Equal opportunity and affirmative action employer
  • Reasonable accommodation for persons with disabilities
  • Fulltime
Read More
Arrow Right

Tech Lead - Ab Initio / Spark ETL

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS or MS Degree in Computer Science, Information Technology, or equivalent
  • Minimum 10 years of Ab Initio application design and development experience
  • Minimum 6 years of experience in leading globally distributed teams successfully
  • Demonstrated people and technical management skills
  • Strong understanding of ETL analysis and design, frameworks on Ab Initio
  • Experience in developing and managing big data solutions using Apache Spark
  • Strong programming skills in Scala, Java, or Python
  • Hands on experience on Technologies like Apache Hive, Apache Kafka, HBase, Couchbase, Sqoop, Flume etc.
  • Demonstrated excellent software development skills
  • Experience in building enterprise ETL based system with focus on recovery, stability, reliability, scalability and performance
Job Responsibility
Job Responsibility
  • Responsible for providing technical leadership of multiple large scale/complex global software solutions
  • Experience of developing teams of permanent employees and vendors from 5 – 20 developers in size
  • Build and sustain strong relationships with the senior business leaders associated with the platform
  • Design, code, test, document and implement application release projects as part of development team
  • Work with onsite development partners to ensure design and coding best practices
  • Work closely with Program Management and Quality Control teams to deliver quality software to agreed project schedules
  • Proactively notify Development Project Manager of risks, bottlenecks, problems, issues, and concerns
  • Compliance with Citi's System Development Lifecycle and Information Security requirements
  • Oversee development scope, budgets, time line documents
  • Monitor, update and communicate project timelines and milestones
  • Fulltime
Read More
Arrow Right

Senior Technology Development Lead

The Senior Technology Development Lead is a senior level position responsible fo...
Location
Location
Canada , Mississauga
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of relevant experience in technology and development
  • Technical skillset required: (1) Language: Java or Python (2) Experience with BigData/Hadoop/Spark/Kafka (3) Experience with APIs and microservices architecture (4) UI Development and integration experience would be a plus
  • Domain experience required: Candidate must have prior experience working in a capital market domain
  • Experience in Regulatory reporting or reconciliations (Tech) would be a big plus
  • Proven track record of delivering large scale projects with globally distributed teams
  • Strong experience in applications development, design paradigms
  • Previous experience with data domain would be a plus
  • Experience in stakeholder management and managing global technology team is a must
  • Consistently demonstrates clear and concise written and verbal communication
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Manage multiple teams of professionals spread across various regions
  • Represent various functions and global teams in the meetings with stakeholders
  • Work with various leads within the group and outside to liaise the work efforts
  • Participate in technical design discussion, production support issues and development activities
  • Influence and negotiate with senior leaders across functions, as well as communicate with external parties as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
What we offer
What we offer
  • Best-in-class benefits
  • Global Benefits
  • Fulltime
Read More
Arrow Right

Tech Lead – Scala/Spark

We are seeking a Spark, Big Data - ETL Tech Lead for Commercial Card’s Global Da...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or master’s degree in computer science, Information Technology, or equivalent
  • Minimum 10 years of Proven experience in developing and managing big data solutions using Apache Spark. Having strong hold on Spark-core, Spark-SQL & Spark Streaming
  • Minimum 6 years of experience in leading globally distributed teams successfully
  • Strong programming skills in Scala, Java, or Python
  • Hands on experience on Technologies like Apache Hive, Apache Kafka, HBase, Couchbase, Sqoop, Flume etc.
  • Proficiency in SQL and experience with relational (Oracle/PL-SQL) and NoSQL databases like mongoDB
  • Demonstrated people and technical management skills
  • Demonstrated excellent software development skills. Strong experiences in implementation of complex file transformations like positional, xmls
  • Experience in building enterprise system with focus on recovery, stability, reliability, scalability and performance
  • Experience in working on Kafka, JMS / MQ applications
Job Responsibility
Job Responsibility
  • Lead the design and implementation of large-scale data processing pipelines using Apache Spark on BigData Hadoop Platform
  • Develop and optimize Spark applications for performance and scalability
  • Responsible for providing technical leadership of multiple large scale/complex global software solutions
  • Integrate data from various sources, including Couchbase, Snowflake, and HBase, ensuring data quality and consistency
  • Experience of developing teams of permanent employees and vendors from 5 – 15 developers in size
  • Build and sustain strong relationships with the senior business leaders associated with the platform
  • Design, code, test, document and implement application release projects as part of development team
  • Work with onsite development partners to ensure design and coding best practices
  • Work closely with Program Management and Quality Control teams to deliver quality software to agreed project schedules
  • Proactively notify Development Project Manager of risks, bottlenecks, problems, issues, and concerns
  • Fulltime
Read More
Arrow Right