CrawlJobs Logo

Big Data / PySpark Engineering Lead - Vice President

https://www.citi.com/ Logo

Citi

Location Icon

Location:
India , Pune

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

The Applications Development Technology Lead Analyst is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications systems analysis and programming activities.

Job Responsibility:

  • Design and implement scalable, fault-tolerant batch and real-time data processing pipelines
  • Develop robust data models and schema designs optimized for both performance and storage efficiency
  • Evaluate and integrate emerging tools and frameworks (e.g., Spark, Flink, Kafka) into the existing stack
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Legacy Systems Decommissioning: Lead the strategic migration of data and logic from legacy platforms (e.g. on-premises SQL Servers) to a modern Data Lakehouse environment
  • ETL/ELT Transformation: Re-engineer existing stored procedures and complex legacy ETL jobs into scalable, distributed processing frameworks using Spark (Python) and Starburst/Trino
  • Validation & Parity Testing: Design and implement automated frameworks for Data Parity Testing to ensure 100% accuracy and consistency between legacy outputs and new big data results
  • Schema Evolution: Map and transform rigid, legacy relational schemas into flexible, high-performance formats optimized for the cloud (e.g., Parquet, Avro, or Iceberg)
  • Phased Cutover Management: Orchestrate a phased migration strategy (Parallel Run, Shadow Execution) to ensure zero downtime for downstream business applications and reporting tools
  • Performance Benchmarking: Establish performance baselines on legacy systems and ensure the new Big Data architecture meets or exceeds those benchmarks at scale
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Write clean, high-performance code in Python
  • Optimize complex SQL queries and fine-tune distributed computing clusters to reduce latency and costs
  • Ensure data integrity and security by implementing rigorous validation and encryption standards
  • Build and maintain CI/CD pipelines for automated testing and deployment of data jobs
  • Monitor system health and troubleshoot performance bottlenecks across the data lifecycle
  • Provide technical mentorship and conduct code reviews for junior and mid-level engineers
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Translate complex business requirements into technical specifications
  • Collaborate with Product Managers to ensure data availability for downstream analytics, business models and users
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements

Requirements:

  • Highly experienced and skilled technical lead with 12+years of experience with software building and platform engineering
  • Experience in Data Engineering, focused on Big Data ecosystems
  • Knowledge in Hadoop, YARN, Hive, Impala, Spark, and Spark SQL with extensive high volume of data processing pipeline development
  • Programming Expert level and hand on experience in Python
  • Familiarity with data formats like Avro, Parquet, CSV, JSON
  • Hands-on experience in writing SQL queries
  • Highly experienced with Unix based operating systems and shell scripting
  • Experience with source code management tools such as Bitbucket, Git etc
  • Big Data Tech Proficiency and hands-on in Hadoop, Spark, Hive, Kafka, and NoSQL databases (MongoDB, HBase)
  • Experience working with query engines like Trino, Presto, Starburst
  • Strong computer science fundamentals in data structures, algorithms, databases, and operating systems
  • Reverse Engineering, ability to read "spaghetti" SQL or old scripts and document the business logic before moving it
  • Data Lineage, Experience using tools (like Collibra or Informatica) to track where data comes from and where it’s going
  • Change Management, Experience managing the technical "shock" to the business when switching from legacy BI tools to modern query engines like Starburst

Nice to have:

  • Problem Solver: You don't just fix bugs
  • you identify the root cause to prevent recurrence
  • Communicator: You can explain the "why" behind a technical decision to non-technical stakeholders
  • Automation and AI Mindset: You believe that if a task has to be done twice, it should be automated. Familiarity with AI tools to expedite deliveries

Additional Information:

Job Posted:
March 01, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Icon
Citi - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Big Data / PySpark Engineering Lead - Vice President

Data Analytics Senior Analyst - Assistant Vice President

The Data Analytics Senior Analyst is a seasoned professional role. Applies in-de...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years experience using tools for statistical modeling of large data sets
  • Database Development & Architecture:Design, develop, and maintain complex database solutions across MongoDB, Oracle, and other SQL databases. Create optimal data models, schemas, and stored procedures to support high-throughput applications
  • Data Pipeline Construction:Build and manage resilient, scalable ETL/ELT pipelines using Python to process and integrate large volumes of data from diverse source systems into our core data platforms
  • Big Data Engineering:Engineer and implement solutions within our Big Data ecosystem (e.g., Hadoop, Spark, Hive, Kafka) to handle large-scale data processing, batch analytics, and real-time data streams
  • Python Development:Write high-quality, production-ready Python code for data manipulation, API development, and automation. Utilize a range of libraries and frameworks relevant to data engineering (e.g., Pandas, PySpark, SQLAlchemy, PyMongo)
  • Performance Optimization:Proactively monitor, troubleshoot, and optimize the performance of our databases and data pipelines. Focus on query tuning, indexing strategies, and resource management to ensure low-latency data access
  • Data Quality and Integrity:Implement data quality checks, validation rules, and monitoring frameworks within the data pipelines to ensure the accuracy, consistency, and reliability of our KYC data
  • Collaboration:Work closely with application developers, data scientists, and data analysts to understand their data requirements and provide robust, well-documented data solutions and services
  • Technical Leadership:Provide subject matter expertise on database and data engineering best practices. Mentor junior engineers and contribute to a culture of technical excellence
Job Responsibility
Job Responsibility
  • Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flows
  • Coordinates and contribute to the objectives of data science initiatives and overall business through leveraging in-depth understanding of how areas collectively integrate within the sub-function
  • Assumes informal/formal leadership role through coaching and training of new recruits
  • Significantly influences decisions, work, and performance of all teams through advice, counsel and/or facilitating services to others in the business
  • Conducts strategic data analysis, identifies insights and implications and make strategic recommendations, develops data displays that clearly communicate complex analysis
  • Mines and analyzes data from various banking platforms to drive optimization and improve data quality
  • Delivers analytics initiatives to address business problems with the ability to identify data required, assess time & effort required and establish a project plan
  • Consults with business clients to identify system functional specifications. Applies comprehensive understanding of how multiple areas collectively integrate to contribute towards achieving business goals
  • Consults with users and clients to solve complex system issues/problems through in-depth evaluation of business processes, systems and industry standards
  • recommends solutions
  • Fulltime
Read More
Arrow Right

Data Analytics Senior Analyst - Assistant Vice President

The Data Analytics Senior Analyst is a seasoned professional role. Applies in-de...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years experience using tools for statistical modeling of large data sets
  • Database Development & Architecture:Design, develop, and maintain complex database solutions across MongoDB, Oracle, and other SQL databases. Create optimal data models, schemas, and stored procedures to support high-throughput applications
  • Data Pipeline Construction:Build and manage resilient, scalable ETL/ELT pipelines using Python to process and integrate large volumes of data from diverse source systems into our core data platforms
  • Big Data Engineering:Engineer and implement solutions within our Big Data ecosystem (e.g., Hadoop, Spark, Hive, Kafka) to handle large-scale data processing, batch analytics, and real-time data streams
  • Python Development:Write high-quality, production-ready Python code for data manipulation, API development, and automation. Utilize a range of libraries and frameworks relevant to data engineering (e.g., Pandas, PySpark, SQLAlchemy, PyMongo)
  • Performance Optimization:Proactively monitor, troubleshoot, and optimize the performance of our databases and data pipelines. Focus on query tuning, indexing strategies, and resource management to ensure low-latency data access
  • Data Quality and Integrity:Implement data quality checks, validation rules, and monitoring frameworks within the data pipelines to ensure the accuracy, consistency, and reliability of our KYC data
  • Collaboration:Work closely with application developers, data scientists, and data analysts to understand their data requirements and provide robust, well-documented data solutions and services
  • Technical Leadership:Provide subject matter expertise on database and data engineering best practices. Mentor junior engineers and contribute to a culture of technical excellence
  • Bachelor’s/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flows
  • Coordinates and contribute to the objectives of data science initiatives and overall business through leveraging in-depth understanding of how areas collectively integrate within the sub-function
  • Assumes informal/formal leadership role through coaching and training of new recruits
  • Significantly influences decisions, work, and performance of all teams through advice, counsel and/or facilitating services to others in the business
  • Conducts strategic data analysis, identifies insights and implications and make strategic recommendations, develops data displays that clearly communicate complex analysis
  • Mines and analyzes data from various banking platforms to drive optimization and improve data quality
  • Delivers analytics initiatives to address business problems with the ability to identify data required, assess time & effort required and establish a project plan
  • Consults with business clients to identify system functional specifications. Applies comprehensive understanding of how multiple areas collectively integrate to contribute towards achieving business goals
  • Consults with users and clients to solve complex system issues/problems through in-depth evaluation of business processes, systems and industry standards
  • recommends solutions
  • Fulltime
Read More
Arrow Right

Fullstack Big Data Developer Application Development Technical Lead Analyst Vice President

Discover your future at Citi. Working at Citi is far more than just a job. A car...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of Application development experience
  • 6+ years of experience in full stack development, with a focus on Bigdata and Python/Scala
  • 6+ years experience with big data technologies such as Python, Pyspark, Hadoop, Kafka, etc.
  • Experience with Core Java/J2EE Application with complete command over OOPs and Design Patterns
  • Commendable in Data Structures and Algorithms
  • Worked on Core Application Development of complex size encompassing all areas of Java/J2EE
  • Thorough knowledge and hands on experience in following technologies Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig , Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
  • Should have implemented or part complex project execution in Big Data Spark eco system, where processing volumes of data thorough understanding of distributed processing and integrated applications
  • Exposure to ETL and BI tools
  • Work in an agile environment following through the best practices of agile Scrum
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Design, develop, and maintain scalable and robust architecture for the project using Java/Python/Scala and other full stack technologies
  • Manage big data technologies such as python, pyspark to ensure seamless data integration, storage, and analysis
  • Fulltime
Read More
Arrow Right
New

Data Analytics Lead - Data Scientist - Vice President

The Data Analytics Lead / Data Scientist is a strategic professional who stays a...
Location
Location
India , Chennai; Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10-15 years of relevant experience in Data Analytics, Data Science, or Advanced Analytics roles
  • Advanced proficiency in SQL and relational database concepts
  • Strong programming experience in Python (required)
  • PySpark preferred
  • Hands-on experience building and deploying machine learning models (supervised and unsupervised)
  • Experience with ML libraries such as scikit-learn, XGBoost, TensorFlow, or PyTorch
  • Strong knowledge of statistical modeling, feature engineering, and model validation techniques
  • Experience with BI tools such as Tableau or Power BI
  • Familiarity with MLOps practices (model deployment, monitoring, versioning) is strongly preferred
  • Experience working with large-scale enterprise or financial datasets
Job Responsibility
Job Responsibility
  • Integrates subject matter and industry expertise within a defined area
  • Contributes to data analytics standards around which others will operate
  • Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinate and contribute to the objectives of the entire function
  • Employs developed communication and diplomacy skills are required in order to guide, influence and convince others, in particular colleagues in other areas and occasional external customers
  • Resolves occasionally complex and highly variable issues
  • Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/ taken
  • Responsible for volume, quality, timeliness and delivery of data science projects along with short-term planning resource planning
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Lead the design and execution of complex data analysis and AI/ML initiatives across large, structured, and unstructured datasets
  • Develop and deploy predictive, classification, clustering, and forecasting models to support business strategy and risk management
  • Fulltime
Read More
Arrow Right
New

Residential Children’s Support Worker

Join Alcedo Care as a Residential Children’s Support Worker working within our W...
Location
Location
United Kingdom , Newton-le-Willows
Salary
Salary:
36500.00 GBP / Year ▼
alcedocare.co.uk Logo
Alcedo Care
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous experience of working within a childrens residential ofsted setting required
  • Minimum of NVQ Level 3 in Residential Child Care or prepared to work towards
  • Experience of working with children / young people – ideally those with learning Difficulties and EBD needs
  • We are looking for day staff, to work 24 hour shifts inclusive of a sleep in
  • Driving licence is essential
  • The position is subject to satisfactory references and enhanced DBS checks
Job Responsibility
Job Responsibility
  • Providing emotional and social support: Building meaningful relationships with the children and encouraging their interaction with peers and adults within the community
  • Promoting positive life choices and independence: Supporting the children in making positive decisions and achieving levels of independence appropriate to their age and abilities
  • Daily living support: Assisting the children with their daily living needs in accordance with their care plans such as attending activities/appointments/ education etc
  • Domestic duties: Conducting household tasks and involving the children when appropriate, including planning, shopping, and preparing meals to ensure a varied and balanced diet
What we offer
What we offer
  • DBS cost covered by Alcedo Care
  • Bank holiday work paid at hourly rate x1.5
  • Ongoing support from senior staff
  • Blue Light Card offering hundreds of discounts after 6 months of service
  • Free mental wellbeing support available 24/7
  • Fantastic training program including online/practical training
  • Fulltime
Read More
Arrow Right
New

Benefits Analyst

Robert Half HR Solutions is looking for a detail-oriented Benefits Analyst to jo...
Location
Location
United States , Atlanta
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 3 years of experience in benefits analysis or coordination
  • Proficiency in Excel, including expertise in pivot tables and VLOOKUP functions
  • Strong written and verbal communication skills for employee engagement and documentation
  • Demonstrated ability to manage multi-state employee benefits programs
  • Familiarity with retirement plans such as 401K, 457B, and 457F
  • Experience with benefits platforms like TotalSource is highly preferred
  • Exceptional attention to detail and organizational skills
  • Ability to work independently while maintaining a customer-focused approach
Job Responsibility
Job Responsibility
  • Manage and oversee employee benefits programs, including medical, dental, vision, and retirement plans
  • Utilize ticketing systems to address employee inquiries and resolve benefits-related issues efficiently
  • Collaborate with third-party administrators to ensure seamless handling of multi-state employee benefits
  • Prepare and maintain accurate documentation related to benefits processes and policies
  • Support HR teams by providing expert advice and assistance on compensation and benefits matters
  • Analyze and interpret data using advanced Excel functions such as pivot tables and VLOOKUP to optimize benefits operations
  • Communicate benefits updates and information clearly and professionally to employees across the organization
  • Ensure compliance with relevant regulations, including FMLA, and monitor benefits administration for accuracy
  • Provide exceptional customer service with a calm and patient demeanor to address employee concerns effectively
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right
New

Assistant Restaurant Manager

Areas of responsibility include Restaurants/Bars and Room Service, if applicable...
Location
Location
Spain , Barcelona
Salary
Salary:
Not provided
https://www.marriott.com Logo
Marriott Bonvoy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or GED
  • 4 years experience in the food and beverage, culinary, or related professional area
  • OR 2-year degree from an accredited university in Food Service Management, Hotel and Restaurant Management, Hospitality, Business Administration, or related major
  • 2 years experience in the food and beverage, culinary, or related professional area
Job Responsibility
Job Responsibility
  • Assists in the daily supervision restaurant operations
  • Assists with menu planning
  • Maintains sanitation standards
  • Assists servers and hosts on the floor during peak meal periods
  • Strives to continually improve guest and employee satisfaction
  • Determines training needed to accomplish goals, then implements plan
  • Handles employee questions and concerns
  • Monitors employees to ensure performance expectations are met
  • Provides feedback to employees based on observation of service behaviors
  • Assists in supervising daily shift operations
  • Fulltime
Read More
Arrow Right
New

Financial Analyst

Entry‑Level Financial Analyst role. Contract‑to‑hire opportunity offering long‑t...
Location
Location
United States , Hiawatha
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Finance, Accounting, Business, Economics, or a related field
  • Strong analytical and problem‑solving skills
  • Proficiency in Excel (pivot tables, VLOOKUPs, formulas)
  • Excellent attention to detail and accuracy
  • Strong communication skills, both written and verbal
  • Ability to work well in a team‑oriented environment
  • Must currently reside in the state of Iowa
Job Responsibility
Job Responsibility
  • Support financial reporting, forecasting, and analysis activities
  • Assist in compiling monthly, quarterly, and annual financial packages
  • Analyze trends, variances, and KPIs to support decision‑making
  • Prepare and maintain spreadsheets, models, and dashboards
  • Help reconcile financial data and ensure accuracy across systems
  • Work cross‑functionally to gather information and support ongoing projects
  • Participate in ad‑hoc analysis and special initiatives as assigned
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
  • Fulltime
Read More
Arrow Right
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Canada Jobs Australia Jobs Germany Jobs Ireland Jobs Poland Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
CrawlJobs Group
ITFlashcards AllDevBlogs
© 2026 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy