CrawlJobs Logo

Big Data / PySpark Engineering Lead - Vice President

https://www.citi.com/ Logo

Citi

Location Icon

Location:
India , Pune

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

The Applications Development Technology Lead Analyst is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications systems analysis and programming activities.

Job Responsibility:

  • Design and implement scalable, fault-tolerant batch and real-time data processing pipelines
  • Develop robust data models and schema designs optimized for both performance and storage efficiency
  • Evaluate and integrate emerging tools and frameworks (e.g., Spark, Flink, Kafka) into the existing stack
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Legacy Systems Decommissioning: Lead the strategic migration of data and logic from legacy platforms (e.g. on-premises SQL Servers) to a modern Data Lakehouse environment
  • ETL/ELT Transformation: Re-engineer existing stored procedures and complex legacy ETL jobs into scalable, distributed processing frameworks using Spark (Python) and Starburst/Trino
  • Validation & Parity Testing: Design and implement automated frameworks for Data Parity Testing to ensure 100% accuracy and consistency between legacy outputs and new big data results
  • Schema Evolution: Map and transform rigid, legacy relational schemas into flexible, high-performance formats optimized for the cloud (e.g., Parquet, Avro, or Iceberg)
  • Phased Cutover Management: Orchestrate a phased migration strategy (Parallel Run, Shadow Execution) to ensure zero downtime for downstream business applications and reporting tools
  • Performance Benchmarking: Establish performance baselines on legacy systems and ensure the new Big Data architecture meets or exceeds those benchmarks at scale
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Write clean, high-performance code in Python
  • Optimize complex SQL queries and fine-tune distributed computing clusters to reduce latency and costs
  • Ensure data integrity and security by implementing rigorous validation and encryption standards
  • Build and maintain CI/CD pipelines for automated testing and deployment of data jobs
  • Monitor system health and troubleshoot performance bottlenecks across the data lifecycle
  • Provide technical mentorship and conduct code reviews for junior and mid-level engineers
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Translate complex business requirements into technical specifications
  • Collaborate with Product Managers to ensure data availability for downstream analytics, business models and users
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements

Requirements:

  • Highly experienced and skilled technical lead with 12+years of experience with software building and platform engineering
  • Experience in Data Engineering, focused on Big Data ecosystems
  • Knowledge in Hadoop, YARN, Hive, Impala, Spark, and Spark SQL with extensive high volume of data processing pipeline development
  • Programming Expert level and hand on experience in Python
  • Familiarity with data formats like Avro, Parquet, CSV, JSON
  • Hands-on experience in writing SQL queries
  • Highly experienced with Unix based operating systems and shell scripting
  • Experience with source code management tools such as Bitbucket, Git etc
  • Big Data Tech Proficiency and hands-on in Hadoop, Spark, Hive, Kafka, and NoSQL databases (MongoDB, HBase)
  • Experience working with query engines like Trino, Presto, Starburst
  • Strong computer science fundamentals in data structures, algorithms, databases, and operating systems
  • Reverse Engineering, ability to read "spaghetti" SQL or old scripts and document the business logic before moving it
  • Data Lineage, Experience using tools (like Collibra or Informatica) to track where data comes from and where it’s going
  • Change Management, Experience managing the technical "shock" to the business when switching from legacy BI tools to modern query engines like Starburst

Nice to have:

  • Problem Solver: You don't just fix bugs
  • you identify the root cause to prevent recurrence
  • Communicator: You can explain the "why" behind a technical decision to non-technical stakeholders
  • Automation and AI Mindset: You believe that if a task has to be done twice, it should be automated. Familiarity with AI tools to expedite deliveries

Additional Information:

Job Posted:
March 01, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Icon
Citi - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Big Data / PySpark Engineering Lead - Vice President

Fullstack Big Data Developer Application Development Technical Lead Analyst Vice President

Discover your future at Citi. Working at Citi is far more than just a job. A car...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of Application development experience
  • 6+ years of experience in full stack development, with a focus on Bigdata and Python/Scala
  • 6+ years experience with big data technologies such as Python, Pyspark, Hadoop, Kafka, etc.
  • Experience with Core Java/J2EE Application with complete command over OOPs and Design Patterns
  • Commendable in Data Structures and Algorithms
  • Worked on Core Application Development of complex size encompassing all areas of Java/J2EE
  • Thorough knowledge and hands on experience in following technologies Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig , Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
  • Should have implemented or part complex project execution in Big Data Spark eco system, where processing volumes of data thorough understanding of distributed processing and integrated applications
  • Exposure to ETL and BI tools
  • Work in an agile environment following through the best practices of agile Scrum
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Design, develop, and maintain scalable and robust architecture for the project using Java/Python/Scala and other full stack technologies
  • Manage big data technologies such as python, pyspark to ensure seamless data integration, storage, and analysis
  • Fulltime
Read More
Arrow Right

Data Analytics Lead - Data Scientist - Vice President

The Data Analytics Lead / Data Scientist is a strategic professional who stays a...
Location
Location
India , Chennai; Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10-15 years of relevant experience in Data Analytics, Data Science, or Advanced Analytics roles
  • Advanced proficiency in SQL and relational database concepts
  • Strong programming experience in Python (required)
  • PySpark preferred
  • Hands-on experience building and deploying machine learning models (supervised and unsupervised)
  • Experience with ML libraries such as scikit-learn, XGBoost, TensorFlow, or PyTorch
  • Strong knowledge of statistical modeling, feature engineering, and model validation techniques
  • Experience with BI tools such as Tableau or Power BI
  • Familiarity with MLOps practices (model deployment, monitoring, versioning) is strongly preferred
  • Experience working with large-scale enterprise or financial datasets
Job Responsibility
Job Responsibility
  • Integrates subject matter and industry expertise within a defined area
  • Contributes to data analytics standards around which others will operate
  • Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinate and contribute to the objectives of the entire function
  • Employs developed communication and diplomacy skills are required in order to guide, influence and convince others, in particular colleagues in other areas and occasional external customers
  • Resolves occasionally complex and highly variable issues
  • Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/ taken
  • Responsible for volume, quality, timeliness and delivery of data science projects along with short-term planning resource planning
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Lead the design and execution of complex data analysis and AI/ML initiatives across large, structured, and unstructured datasets
  • Develop and deploy predictive, classification, clustering, and forecasting models to support business strategy and risk management
  • Fulltime
Read More
Arrow Right
New

Engineer I, EHS

You, as Engineer I, EHS, will support supervisors by providing tools and advice ...
Location
Location
Costa Rica , Cartago
Salary
Salary:
Not provided
https://www.baxter.com/ Logo
Baxter
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in occupational health and safety, or equivalent experience, or higher education in related areas
  • Experience in related fields of 1 years minimum
  • Sophisticated English, bilingual desirable
  • Certification in ISO 14001:2014, ISO 45001:2018 and ISO 50001 standards (desirable)
  • Internal Auditor Certificate (desirable)
Job Responsibility
Job Responsibility
  • Promote compliance with quality requirements, Good Manufacturing Practices (GMPs) and Good Documentation Practices (GDP)
  • Promote and participate in ICare's initiatives: if you see something, do something
  • Respond to and resolve in a timely manner the problems that may affect the quality, safety or efficiency of the product or the regulatory consistency of the process
  • Continuously seek regulatory completion in your process
  • Stay aligned with EHS rules and procedures
  • Ensure the people in your charge follow them
  • Provide crucial training spaces for EHS issues
  • Promote the culture of safety, and ensure compliance with all the guidelines that the organization settles vital to align with the applicable legal and/or corporate requirements of its area passionate about protecting safety, its own health and that of its personnel in charge, and the environment
  • Build, coordinate and implement the company's Ergonomics program
  • Build, coordinate and implement the company's Environmental program (Management of waste, wastewater, air emissions, drinking water, identification and evaluation of environmental aspects, assessment of relevant regulations, training, among others)
Read More
Arrow Right
New

Store Operator

GrainCorp Feeds is a national animal feed business, based in Hamilton NZ, dealin...
Location
Location
New Zealand , Waharoa
Salary
Salary:
Not provided
graincorp.com.au Logo
GrainCorp
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Current forklift licence with F and W endorsement
  • Experience working in a warehousing / logistics role
  • Good communication skills
  • Ability to follow direction
  • Ability to work collaboratively and autonomously
  • Capable of lifting up to 20kg in weight
  • Solid understanding of safety standards
Job Responsibility
Job Responsibility
  • Bagging and pallet stacking
  • Moving / loading palletised stock
What we offer
What we offer
  • Ongoing training and safety programs
  • Paid Parental leave and birthday leave
  • Employee referral bonus scheme
  • Standard Southern Cross Health Insurance that covers employee, spouse and kids up to the age of 18
  • Family Inclusive Workplace accredited employer, committed to supporting you both on and off the job
  • Fulltime
Read More
Arrow Right
New

Marketing Manager

We are looking for a dynamic Marketing Manager to spearhead social media and dig...
Location
Location
United States , Los Angeles
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-7 years of marketing experience with a strong emphasis on social media and digital strategies
  • Proven experience in building and growing brand presence, preferably within the fashion or footwear industries
  • Expertise in content strategy, platform trends, and performance analytics
  • Strategic thinker with the ability to manage day-to-day marketing activities effectively
  • Excellent collaboration skills with a history of working cross-functionally, especially with eCommerce teams
  • Proficiency in email campaigns, digital marketing, and enhancing brand awareness
  • Strong understanding of social media platforms and their role in driving business growth
Job Responsibility
Job Responsibility
  • Develop and implement comprehensive social media strategies to establish a strong and consistent brand presence across multiple platforms
  • Collaborate with the eCommerce team to ensure marketing efforts align seamlessly with sales objectives and customer journey goals
  • Plan and oversee content calendars, digital campaigns, and product launches to drive engagement and conversions
  • Lead wholesale marketing strategies, including organizing events at brick-and-mortar locations to boost brand visibility
  • Monitor and analyze campaign performance metrics, optimizing strategies to increase traffic, engagement, and revenue
  • Identify and pursue opportunities for expanding the brand's digital footprint, including partnerships, new platforms, and innovative content formats
  • Refine and maintain brand guidelines to ensure consistent messaging across all marketing channels
  • Work closely with creative teams, influencers, and external partners to produce high-quality content that aligns with the brand identity
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
Read More
Arrow Right
New

Behavior Analyst in Training

Behavior Analysis Practicum (Master's Level). Premier ABA Provider for Children ...
Location
Location
United States , Minneapolis
Salary
Salary:
24.00 - 30.00 USD / Hour
actionbehavior.com Logo
Action Behavior Centers
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Supervisee has accrued 85% of fieldwork hours (1700/2000hours)
  • All Restricted hours completed
  • Proof of enrollment in an accredited applicable masters program
Job Responsibility
Job Responsibility
  • Providing early intervention therapy in a center-based setting
  • Shaping the minds of kiddos in early childhood (ages 18 months to 8 years old)
  • Collecting data and implementing individualized treatment plans for each child
  • Collaborating with BCBA’s and our team of like-minded individuals dedicated to living ABC’s core values
  • Allocate 8 hours per week off direct care schedule to work on BDS modules
  • Sit for the BACB exam within 6 months of joining our team
What we offer
What we offer
  • No-cost supervision hours
  • A $5,000 BCBA Pass Bonus
  • Generous time off: 10 holidays, 10 paid days PLUS 2 flex days
  • College Tuition Employee Discount
  • Health Benefits: Paid up to 90% by the company with 4 plans to choose from
  • 401K + Match
  • Free lunch EVERY Friday
  • Complimentary DoorDash DashPass
  • Calm subscription
  • Student loan Repayment Employer Contributions
  • Fulltime
Read More
Arrow Right
New

Assistant Chief Engineer

Assists department head in managing budget, capital expenditures, preventative m...
Location
Location
Vietnam , Danang Hoi An
Salary
Salary:
Not provided
https://www.marriott.com Logo
Marriott Bonvoy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or GED
  • 4 years experience in the engineering and maintenance or related professional area
  • technical training in HVAC-R/electrical/plumbing
  • OR 2-year degree from an accredited university in Building and Construction, Engineering, Mechanics, or related major
  • 2 years experience in the engineering and maintenance or related professional area
  • technical training in HVAC-R/electrical/plumbing
Job Responsibility
Job Responsibility
  • Assists department head in managing budget, capital expenditures, preventative maintenance and energy conservation
  • Accountable for maintaining the building and grounds with particular attention towards safety, security and asset protection
  • Assigns, tracks and follows up on status of work assigned to staff and contractors
  • Helps to increase guest and employee satisfaction through effective communications and training
  • Supervises distribution of repair work orders
  • Ensures property policies are administered fairly and consistently
  • Assists in effectively planning, scheduling and evaluating preventative maintenance programs
  • Establishes and maintains open, collaborative relationships with employees
  • Monitors timeliness and quality of completion of repair work orders
  • Supervises Engineering and related areas in absence of the Director of Engineering
  • Fulltime
Read More
Arrow Right
New

Video/Multimedia Developer

We are seeking a creative and technically skilled Video / Multimedia Developer t...
Location
Location
United States , Rochester
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Media Arts, Communications, or a related field (or equivalent experience)
  • Minimum of 3 years of video production experience (5+ years preferred)
  • Hands-on experience with: Adobe Premiere Pro
  • Adobe After Effects
  • Adobe Photoshop
  • Strong technical expertise in: Video editing
  • Lighting setup
  • Audio recording and processing
  • Proven ability to manage multiple projects and meet deadlines
Job Responsibility
Job Responsibility
  • Plan, produce, and deliver high-quality video and multimedia content for internal communications
  • Manage end-to-end production, including concept development, scripting, filming, editing, and post-production
  • Execute both studio and field-based video and audio projects
  • Collaborate with internal clients to translate business needs into engaging visual narratives
  • Handle lighting, camera operation, and audio recording to ensure professional production quality
  • Edit and enhance video content using industry-standard tools
  • Manage project timelines, production logistics, and deliverables across multiple assignments
  • Adapt quickly to evolving creative direction and business priorities
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right
Remote jobs Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Canada Jobs Australia Jobs Germany Jobs Ireland Jobs Poland Jobs
Company
Blog Salaries About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
CrawlJobs Group
ITFlashcards AllDevBlogs
© 2026 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy