CrawlJobs Logo
Citi Logo Citi · IT - Software Development

Lead Big Data Spark Engineer

Canada, Mississauga 120800.00 - 170800.00 USD / Year · Job Posted February 16, 2026
Apply Position
Job Link Share

Job Description

We are seeking an experienced and highly skilled Big Data Engineer to lead the design, development, and implementation of our next-generation big data platforms and applications. The ideal candidate will be a technical leader with a strong background in Scala, Spark, and other big data technologies, capable of driving innovation, mentoring engineers, and collaborating with cross-functional teams to deliver robust and scalable solutions. This role requires a strategic thinker who can translate business requirements into technical architecture and lead their execution.

Job Responsibility

  • Lead the architecture, design, and development of high-performance, scalable, and reliable big data processing systems using Scala and Apache Spark
  • Drive technical vision and strategy for big data initiatives
  • Evaluate and recommend new technologies and tools
  • Design, develop, and optimize data pipelines for ingestion, transformation, and storage of massive datasets
  • Implement robust and efficient data processing jobs using Scala and Spark (batch and streaming)
  • Ensure data quality, integrity, and security
  • Promote and enforce best practices in coding, testing, and deployment
  • Mentor and guide a team of talented big data engineers
  • Conduct code reviews, provide constructive feedback
  • Participate in the recruitment and hiring
  • Partner with product managers, data scientists, and other engineering teams to understand business requirements
  • Communicate complex technical concepts effectively
  • Work closely with DevOps and SRE teams
  • Contribute to the strategic roadmap for big data engineering
  • Stay abreast of emerging trends and technologies

Requirements

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field
  • 8+ years of progressive experience in software development
  • 5+ years focusing on big data technologies
  • 3+ years of experience in a leadership or senior architectural role
  • Extensive hands-on experience with Scala for big data processing
  • Demonstrated expertise with Apache Spark (Spark Core, Spark SQL, Spark Streaming)
  • Strong experience with distributed systems and big data ecosystems (e.g., Hadoop, Kafka, Cassandra, HBase, Delta Lake, Snowflake, Databricks)
  • Proficiency with cloud platforms (AWS, Azure, GCP) and their big data services
  • Experience with containerization technologies (Docker, Kubernetes) and CI/CD pipelines
  • Solid understanding of data warehousing concepts, ETL/ELT processes, and data modeling
  • Familiarity with functional programming paradigms in Scala
  • Exceptional problem-solving and analytical skills
  • Strong leadership, communication, and interpersonal skills
  • Ability to work independently and collaboratively in a fast-paced, dynamic environment
  • Proactive and results-oriented with a strong sense of ownership

Nice to have

  • Experience with real-time data processing and stream analytics
  • Knowledge of machine learning frameworks and their application in big data
  • Contributions to open-source big data projects
  • Experience in the financial services industry
Citi - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Lead Big Data Spark Engineer

8 matching positions

Lead Java Big Data Engineer Vice President

At Citi, we are at the forefront of financial technology, driven by a belief in ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of progressive experience in professional software engineering, with at least 3 years in a technical leadership or architect role
  • Proven track record of designing and building complex, high-performance, scalable server-side applications using Java
  • Deep, hands-on experience with the Big Data ecosystem, including mastery of Apache Spark, Hadoop (HDFS), and real-time data streaming with Kafka
  • Extensive experience with relational databases, data modeling, and data warehousing concepts
  • Demonstrated experience leading and mentoring technical teams and successfully delivering complex, large-scale data projects from concept to production
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
Job Responsibility
Job Responsibility
  • Define the end-to-end architectural vision and technical roadmap for migrating from Sybase IQ to a modern Big Data platform, ensuring solutions are scalable, resilient, and secure
  • Lead the design, development, and deployment of robust, large-scale data processing pipelines using technologies like Apache Spark, Kafka, and distributed data stores
  • Develop and execute a comprehensive, phased strategy for migrating petabytes of historical and transactional data from Sybase IQ, ensuring data integrity, minimal downtime, and zero business disruption
  • Oversee the design and development of Java-based microservices that interact with the new data platform, ensuring seamless integration with the broader Oasys application ecosystem
  • Lead, inspire, and mentor a high-performing team of Java and Big Data engineers. Foster a culture of engineering excellence, innovation, and accountability
  • Partner with global business leaders, product owners, and other senior technology managers to define requirements, manage expectations, and deliver solutions that drive significant business value
  • Remain deeply technical and contribute to coding, design, and architectural decisions, leading by example
  • Fulltime
Read More
Arrow Right

Data Engineer (Big Data, Cloud - AWS, Databricks) - Assistant Vice President

Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Scala, Spark/Pyspark is must, Hadoop ( BIG Data ), + AWS,Databricks
  • 8 to 11 years’ experience implementing data-intensive solutions using agile methodologies
  • Experience of relational databases and using SQL for data querying, transformation and manipulation
  • Experience of modelling data for analytical consumers
  • Ability to automate and streamline the build, test and deployment of data pipelines
  • Experience in cloud native technologies and patterns
  • A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
  • Excellent communication and problem-solving skills
  • An inclination to mentor
  • an ability to lead and deliver medium sized components independently
Job Responsibility
Job Responsibility
  • Developing and supporting scalable, extensible, and highly available data solutions
  • Deliver on critical business priorities while ensuring alignment with the wider architectural vision
  • Identify and help address potential risks in the data supply chain
  • Follow and contribute to technical standards
  • Design and develop analytical data models
  • Fulltime
Read More
Arrow Right
New

Lead Data Engineer - Data Transformation (Modeling and Architecture)

Lead Data Engineer - Data Transformation (Modeling and Architecture). Do you lov...
Location
Location
United States , Richmond, Virginia; McLean, Virginia; Cambridge, Massachusetts
Salary
Salary:
179400.00 - 225100.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree
  • At least 4 years of experience in application development
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Job Responsibility
Job Responsibility
  • Build and maintain comprehensive data models—spanning conceptual, logical, and physical layers—to ensure scalable architecture and high data integrity across enterprise systems
  • Lead design of the org data landscape by applying Consumer Driven design principles, ensuring that data structures reflect business realities and evolving organizational needs
  • Architect and implement robust data ecosystem solutions, including Data Lake and Data Warehouse patterns, to support diverse analytical and operational requirements
  • Support high-performance data pipelines and complex transformations that utilize SQL, Spark, and Python to process large-scale datasets efficiently
  • Define and Enforce rigorous data governance standards while managing metadata frameworks to ensure data compliance and discoverability
  • Translate complex technical concepts into actionable business insights, working independently to lead initiatives and collaborate with stakeholders to meet organizational goals
  • Contribute to the evolution of the data ecosystem by designing AI-ready architectures
  • Collaborate with and across Agile teams to design, develop, implement, and support technical solutions
  • Work with a team of developers with deep experience in machine learning, AI, distributed microservices, and full stack systems
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
What we offer
What we offer
  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • Comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right
New

Lead Data Engineer - HR Tech

Do you love building and pioneering in the technology space? Do you enjoy solvin...
Location
Location
United States , McLean, Virginia; Richmond, Virginia
Salary
Salary:
197300.00 - 225100.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree
  • At least 4 years of experience in application development (Internship experience does not apply)
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
  • 7+ years of experience in application development including Python, SQL, Scala, or Java
  • 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)
  • 4+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
  • 4+ year experience working on real-time data and streaming applications
  • 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
  • 4+ years of data warehousing experience using Snowflake
Job Responsibility
Job Responsibility
  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
  • Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems
  • Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services using Snowflake
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance
  • Build and deploy GenAI and Agentic AI solutions
What we offer
What we offer
  • performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right

Lead Data Engineer

Lead Data Engineer Do you love building and pioneering in the technology space?...
Location
Location
United States , San Francisco; New York
Salary
Salary:
215200.00 - 245600.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree
  • At least 4 years of experience in application development (Internship experience does not apply)
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
  • 7+ years of experience in application development including Python, SQL, Scala, or Java
  • 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)
  • 4+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
  • 4+ year experience working on real-time data and streaming applications
  • 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
  • 4+ years of data warehousing experience (Redshift or Snowflake)
Job Responsibility
Job Responsibility
  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
  • Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems
  • Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance
What we offer
What we offer
  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right

Senior Lead Data Engineer (Bank Tech)

Senior Lead Data Engineer (Bank Tech) - Do you love building and pioneering in t...
Location
Location
United States , Wilmington
Salary
Salary:
Not provided
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s Degree
  • At least 6 years of experience in application development
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Job Responsibility
Job Responsibility
  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
  • Lead a team of developers with deep experience in machine learning, distributed microservices, and full stack systems
  • Utilize programming languages like Python and Java and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance
What we offer
What we offer
  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right

Senior Software Development Engineer - Big Data

Zillow’s Comprehensive Customer Data team is part of the Data Engineering organi...
Location
Location
Mexico , Mexico City
Salary
Salary:
Not provided
Zillow
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A degree (BS+) in Computer Science or a related discipline
  • 5+ years of experience with building and maintaining data-intensive applications
  • Experience developing complex software systems scaling to millions of users with production quality deployment, monitoring and reliability
  • Experience with modern data technologies (Spark, Flink, Kafka, Airflow)
  • Strong background in Java or Python
  • Excellent interpersonal skills and passionate about collaborating across organizational boundaries
Job Responsibility
Job Responsibility
  • Design, implement, and take ownership of mission-critical data pipelines and data products
  • Collaborate with various internal teams (platform, product, other stakeholders) to scope and deliver tools ensuring ease of system use
  • Carefully review specifications, designs, pull requests and provide constructive/helpful feedback
  • Write high-quality code, refactor and optimize code for better scalability, performance, and readability
  • Integrate new frameworks, libraries, technologies, and components into codebase
  • Proactively log/collect and review metrics, monitor, alert and address issues quickly
  • Ensure the consistency and maintainability of existing applications adhering to standards/procedures for implementing technical solutions
  • Execute production/support tasks according to a documented schedule that meets or exceeds customer expectations
  • Support and mentor teams through design, implementation and deployment of solutions using relevant technologies
  • Mentor junior engineers
What we offer
What we offer
  • competitive base salary and benefits
  • equity awards based on factors such as experience, performance and location
  • Fulltime
Read More
Arrow Right

Senior Data Engineer Lead / Architect - Senior Vice President

At Citi Services - Global Trade Technology Organization, we are on a mission to ...
Location
Location
India , Pune, Maharashtra, India, Chennai, Tamil Nadu, India
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of professional experience in data engineering, with a proven track record of designing and building large-scale data systems
  • 3+ years in a technical leadership or architect role, with experience mentoring junior and senior engineers
  • Expert-level proficiency in at least one programming language (Python or Scala preferred) and exceptional SQL skills
  • Proven hands-on experience with Python or Scala for data manipulation, scripting, machine learning, and backend development
  • Deep, hands-on experience with a major cloud platform (AWS, GCP, or Azure) and its data ecosystem (e.g., S3/GCS, Redshift/BigQuery, EMR/Dataproc, Kinesis/Dataflow)
  • Extensive hands-on experience with modern big data technologies and Data streaming (like Hadoop, Hive, Impala, Apache Spark, Kafka, or Flink)
  • Proficiency with workflow orchestration tools such as Airflow, Dagster, or Prefect
  • Proficiency in designing and implementing microservices architectures, RESTful APIs, and event-driven systems with 'Data as a Product' Principle
  • Solid understanding of data modeling concepts and database design for both analytical (OLAP) and transactional (OLTP) workloads
  • Deep understanding and hands-on experience with relational databases (e.g., PostgreSQL, Oracle), NoSQL databases (e.g., MongoDB, Cassandra), data warehousing, and big data technologies (e.g., Spark, Kafka)
Job Responsibility
Job Responsibility
  • Architect & Design: Design, architect, and oversee the development of robust, scalable, and reliable data infrastructure, including data lakes, data warehouses, and real-time streaming platforms on the cloud
  • Build & Code: Act as a senior individual contributor and hands-on technical leader. Write clean, maintainable, and high-performance code for data ingestion, transformation, and serving layers (e.g., using Python, Scala, SQL, and Spark)
  • Lead & Mentor: Lead a team of data engineers, providing technical guidance, mentorship, and career development support. Foster a collaborative and inclusive team environment
  • Champion Culture: Define, document, and champion data engineering best practices across the organization, including CI/CD, data quality, testing frameworks, observability, and code review standards
  • Drive Strategy: Partner with leadership, product managers, data scientists, and analysts to understand data needs and develop a long-term data strategy and roadmap
  • Innovate & Evaluate: Stay at the forefront of data engineering technologies. Evaluate, prototype, and recommend new tools and frameworks to continuously improve our data platform
  • Ensure Governance: Implement and enforce robust data governance, security, and privacy policies in partnership with our security and compliance teams
  • Fulltime
Read More
Arrow Right