Lead Data Engineer Spark and SQL – Vice President Job at Citi (Mississauga)

Senior Data Engineer Lead / Architect - Senior Vice President

At Citi Services - Global Trade Technology Organization, we are on a mission to ...

Location

India , Pune, Maharashtra, India, Chennai, Tamil Nadu, India

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

10+ years of professional experience in data engineering, with a proven track record of designing and building large-scale data systems
3+ years in a technical leadership or architect role, with experience mentoring junior and senior engineers
Expert-level proficiency in at least one programming language (Python or Scala preferred) and exceptional SQL skills
Proven hands-on experience with Python or Scala for data manipulation, scripting, machine learning, and backend development
Deep, hands-on experience with a major cloud platform (AWS, GCP, or Azure) and its data ecosystem (e.g., S3/GCS, Redshift/BigQuery, EMR/Dataproc, Kinesis/Dataflow)
Extensive hands-on experience with modern big data technologies and Data streaming (like Hadoop, Hive, Impala, Apache Spark, Kafka, or Flink)
Proficiency with workflow orchestration tools such as Airflow, Dagster, or Prefect
Proficiency in designing and implementing microservices architectures, RESTful APIs, and event-driven systems with 'Data as a Product' Principle
Solid understanding of data modeling concepts and database design for both analytical (OLAP) and transactional (OLTP) workloads
Deep understanding and hands-on experience with relational databases (e.g., PostgreSQL, Oracle), NoSQL databases (e.g., MongoDB, Cassandra), data warehousing, and big data technologies (e.g., Spark, Kafka)

Job Responsibility

Architect & Design: Design, architect, and oversee the development of robust, scalable, and reliable data infrastructure, including data lakes, data warehouses, and real-time streaming platforms on the cloud
Build & Code: Act as a senior individual contributor and hands-on technical leader. Write clean, maintainable, and high-performance code for data ingestion, transformation, and serving layers (e.g., using Python, Scala, SQL, and Spark)
Lead & Mentor: Lead a team of data engineers, providing technical guidance, mentorship, and career development support. Foster a collaborative and inclusive team environment
Champion Culture: Define, document, and champion data engineering best practices across the organization, including CI/CD, data quality, testing frameworks, observability, and code review standards
Drive Strategy: Partner with leadership, product managers, data scientists, and analysts to understand data needs and develop a long-term data strategy and roadmap
Innovate & Evaluate: Stay at the forefront of data engineering technologies. Evaluate, prototype, and recommend new tools and frameworks to continuously improve our data platform
Ensure Governance: Implement and enforce robust data governance, security, and privacy policies in partnership with our security and compliance teams

Fulltime

Data Engineer (Big Data, Cloud - AWS, Databricks) - Assistant Vice President

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Scala, Spark/Pyspark is must, Hadoop ( BIG Data ), + AWS,Databricks
8 to 11 years’ experience implementing data-intensive solutions using agile methodologies
Experience of relational databases and using SQL for data querying, transformation and manipulation
Experience of modelling data for analytical consumers
Ability to automate and streamline the build, test and deployment of data pipelines
Experience in cloud native technologies and patterns
A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
Excellent communication and problem-solving skills
An inclination to mentor
an ability to lead and deliver medium sized components independently

Job Responsibility

Developing and supporting scalable, extensible, and highly available data solutions
Deliver on critical business priorities while ensuring alignment with the wider architectural vision
Identify and help address potential risks in the data supply chain
Follow and contribute to technical standards
Design and develop analytical data models

Fulltime

Senior Java Angular Engineer - Tech Lead - Vice President

We are seeking a highly skilled and experienced Senior Java Spark Engineer to jo...

Location

United States , Jersey City

Salary:

142320.00 - 213480.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

8+ years of extensive professional experience in Java development, with significant expertise in Angular, Java/J2EE & Spring integration
Proven experience with Apache Spark for large-scale data processing and distributed computing
Thorough knowledge and hands-on experience with Hive, Hadoop, YARN, Spark SQL, Unix, Impala, and Oracle DB
Strong understanding of data architecture principles and data infrastructure
Experience with developing and maintaining coding standards, testing, and debugging for complex applications
Demonstrated ability to provide technical leadership and mentor junior engineers
Excellent analytical, problem-solving, and critical thinking skills
Strong communication and collaboration abilities, with experience partnering with various stakeholders
Knowledge of risk assessment, data security, performance optimization, and scalability considerations
Familiarity with regulatory compliance and ethical considerations in data processing

Job Responsibility

Lead the design, development, and implementation of complex Java Spring integration and Java Spark applications for processing large datasets, leveraging expertise in distributed computing and data manipulation
Partner with management teams to seamlessly integrate Java Integration Framework & Spark-based solutions into existing data infrastructure, ensuring optimal performance and efficient data flow
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency

What we offer

medical
dental & vision coverage
401(k)
life, accident, and disability insurance
wellness programs
paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays
discretionary and formulaic incentive and retention awards

Fulltime

Python Full Stack Data Engineer - Assistant Vice President

We are assembling an A-team of highly skilled, autonomous, and AI-first engineer...

Location

Canada , Mississauga

Salary:

94300.00 - 141500.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

Experience: 4+ years of progressive, hands-on experience as a Data Engineer, with a proven track record of delivering complex, large-scale data solutions
Expert-level proficiency in Python, with deep expertise in developing highly optimized, scalable, and production-grade PySpark applications for mission-critical data processing
Deep understanding and extensive hands-on experience with the entire Apache Spark ecosystem (Spark Core, Spark SQL, Spark Streaming)
Advanced proficiency with Hive for enterprise data warehousing, including optimization techniques for large and complex queries
Expert knowledge of distributed computing fundamentals, HDFS, and other components of the Hadoop ecosystem
Proficiency in SQL, complex query optimization, and advanced data warehousing concepts (e.g., dimensional modeling, data vault, data lakes)
Extensive experience with various data storage formats (e.g., Parquet, ORC, Avro) and leading data lake solutions (e.g., Delta Lake, Iceberg)
Proven experience with enterprise-grade NoSQL databases (e.g., Cassandra, MongoDB, HBase) and understanding of their architectural trade-offs
Expert-level experience with Apache Kafka, including design and implementation of high-throughput, low-latency real-time data pipelines and event-driven architectures
Extensive experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift/Kinesis, Azure Databricks/Data Factory/Synapse/Event Hubs, GCP Dataflow/Dataproc/BigQuery/Pub/Sub), including cloud-native architectural patterns

Job Responsibility

Operate end-to-end in the design, development, and implementation of full-stack data solutions, ensuring optimal performance, scalability, data quality, security, and compliance across the data lifecycle
Collaborate closely within small, co-located squads (4-7 person teams), fostering an environment of high communication and minimal coordination overhead, to deliver impactful data products
Develop, maintain, and optimize highly efficient and resilient data ingestion, processing, and transformation pipelines using advanced Python and PySpark techniques for large-scale datasets
Implement sophisticated data storage solutions leveraging a diverse set of big data technologies including Hive, distributed file systems (e.g., HDFS, S3), and enterprise-grade NoSQL databases (e.g., Cassandra, MongoDB)
Design and implement scalable data models and schemas that support advanced analytics, machine learning, and critical reporting needs, ensuring data integrity, accessibility, and discoverability
Engage effectively with data consumers, data scientists, and business stakeholders to deeply understand their requirements, translating them into robust data solutions and providing expert guidance on data utilization and interpretation
Implement real-time data streaming and complex event-driven architectures using technologies like Apache Kafka, ensuring low-latency data availability for critical business functions
Adhere to and contribute to best practices in data engineering and software development, participating in rigorous code reviews, implementing comprehensive automated testing strategies, and supporting robust CI/CD pipelines within a DevOps culture
Exhibit High Autonomy and Agency, taking ownership of technical challenges, making well-reasoned architectural decisions, and proactively identifying and implementing continuous improvements across the data landscape
Innovate with AI-Powered Development, actively leveraging, integrating, and contributing to AI coding tools (e.g., internal Citi AI tools, Copilot, Claude Code, Codex, Antigravity) to significantly enhance productivity, code quality, and development velocity, and inspiring others to do the same

Fulltime

Big Data / PySpark Engineering Lead - Vice President

The Applications Development Technology Lead Analyst is a senior level position ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Highly experienced and skilled technical lead with 12+years of experience with software building and platform engineering
Experience in Data Engineering, focused on Big Data ecosystems
Knowledge in Hadoop, YARN, Hive, Impala, Spark, and Spark SQL with extensive high volume of data processing pipeline development
Programming Expert level and hand on experience in Python
Familiarity with data formats like Avro, Parquet, CSV, JSON
Hands-on experience in writing SQL queries
Highly experienced with Unix based operating systems and shell scripting
Experience with source code management tools such as Bitbucket, Git etc
Big Data Tech Proficiency and hands-on in Hadoop, Spark, Hive, Kafka, and NoSQL databases (MongoDB, HBase)
Experience working with query engines like Trino, Presto, Starburst

Job Responsibility

Design and implement scalable, fault-tolerant batch and real-time data processing pipelines
Develop robust data models and schema designs optimized for both performance and storage efficiency
Evaluate and integrate emerging tools and frameworks (e.g., Spark, Flink, Kafka) into the existing stack
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Legacy Systems Decommissioning: Lead the strategic migration of data and logic from legacy platforms (e.g. on-premises SQL Servers) to a modern Data Lakehouse environment
ETL/ELT Transformation: Re-engineer existing stored procedures and complex legacy ETL jobs into scalable, distributed processing frameworks using Spark (Python) and Starburst/Trino
Validation & Parity Testing: Design and implement automated frameworks for Data Parity Testing to ensure 100% accuracy and consistency between legacy outputs and new big data results
Schema Evolution: Map and transform rigid, legacy relational schemas into flexible, high-performance formats optimized for the cloud (e.g., Parquet, Avro, or Iceberg)
Phased Cutover Management: Orchestrate a phased migration strategy (Parallel Run, Shadow Execution) to ensure zero downtime for downstream business applications and reporting tools

Fulltime

Vice President, Big Data Scala Engineer

We are seeking an experienced and highly skilled Vice President, Big Data Scala ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Bachelor's or Master's degree in Computer Science, Engineering, or a related field
12+ years of progressive experience in software development, with at least 5+ years focusing on big data technologies
3+ years of experience in a leadership or senior architectural role
Extensive hands-on experience with Scala for big data processing
Demonstrated expertise with Apache Spark (Spark Core, Spark SQL, Spark Streaming)
Strong experience with distributed systems and big data ecosystems (e.g., Hadoop, Kafka, Cassandra, HBase, Delta Lake, Snowflake, Databricks)
Proficiency with cloud platforms (AWS, Azure, GCP) and their big data services (e.g., EMR, Redshift, Glue, DataProc, BigQuery)
Experience with containerization technologies (Docker, Kubernetes) and CI/CD pipelines
Solid understanding of data warehousing concepts, ETL/ELT processes, and data modeling
Familiarity with functional programming paradigms in Scala

Job Responsibility

Lead the architecture, design, and development of high-performance, scalable, and reliable big data processing systems using Scala and Apache Spark
Drive technical vision and strategy for big data initiatives
Evaluate and recommend new technologies and tools
Design, develop, and optimize data pipelines for ingestion, transformation, and storage of massive datasets
Implement robust and efficient data processing jobs using Scala and Spark (batch and streaming)
Ensure data quality, integrity, and security
Promote and enforce best practices in coding, testing, and deployment
Mentor and guide a team of talented big data engineers
Conduct code reviews, provide constructive feedback
Participate in the recruitment and hiring

What we offer

Opportunity to work on cutting-edge big data technologies and impactful projects
A collaborative and innovative work environment
Competitive compensation and benefits package
Opportunities for professional growth and career advancement

Fulltime

Senior Java -Spark-Bigdata Engineer-Assistant Vice President

The Applications Development Senior Programmer Analyst is a senior-level positio...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

7-10 years of relevant experience in Data Engineering or a similar role, preferably within the Financial Services industry
Senior-level experience in an Applications Development or Data Engineering role
Consistently demonstrates clear and concise written and verbal communication
Demonstrated problem-solving and decision-making skills
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Bachelor's degree/University degree or equivalent experience
Hands-on expertise in Java (8+), Spring Boot, Python, and PySpark for building high-performance data applications
Extensive experience with the BigData ecosystem, including Apache Spark for large-scale data processing
Solid understanding of Data Warehouse concepts, design principles, and best practices
Strong proficiency with both relational SQL databases and NoSQL databases (e.g., MongoDB, Couchbase)

Job Responsibility

Utilize expert knowledge of data engineering principles, big data technologies, and software development best practices to design and implement robust data solutions
Collaborate with business stakeholders, data scientists, and other technology teams to understand data requirements and deliver effective solutions
Apply deep expertise in programming languages like Python and Java for building high-performance data processing applications
Ensure data solutions are secure, scalable, and adhere to the firm's security and architectural standards
Mentor and guide junior engineers, fostering a culture of technical excellence and continuous learning
Lead the analysis of complex data-related issues, identify root causes, and implement sustainable solutions
Operate with a high degree of autonomy and independence, exercising sound judgment and decision-making
Act as a Subject Matter Expert (SME) in big data technologies for senior stakeholders and other team members
Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency

Fulltime

Senior Lead Developer (Java, Spark, HDFS, Hive) - Vice President

The Applications Development Technology Lead Analyst is a senior level position ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

10+ years of relevant experience in Apps Development or systems analysis role
Extensive experience system analysis and in programming of software applications
Experience in managing and implementing successful projects
Subject Matter Expert (SME) in at least one area of Applications Development
Ability to adjust priorities quickly as circumstances dictate
Demonstrated leadership and project management skills
Consistently demonstrates clear and concise written and verbal communication
Highly experienced and skilled Java technical lead with 10+years of experience with software building and platform engineering
Extensive development expertise in building the high scaled and performant software platforms for data computation and processing
Expert level knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns

Job Responsibility

Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency

Fulltime

Select Country

Lead Data Engineer Spark and SQL – Vice President

Job Description

Job Responsibility

Requirements

Nice to have

Looking for more opportunities?