CrawlJobs Logo
Citi Logo Citi · IT - Software Development

Senior Databricks & Apache Spark Developer - Vice President

India, Pune · Job Posted May 27, 2026
Apply Position
Job Link Share

Job Description

We are looking for a highly skilled Senior Databricks Engineer to contribute to the engineering, modernization, and continuous evolution of data processing platform on Databricks on AWS. While supporting the transition from the legacy Cloudera Hadoop platform to Databricks on AWS, this role will continue to play a key part in enhancing performance, simplifying pipelines, and delivering new capabilities on the Databricks platform over the long term. The ideal candidate is a strong hands‑on Spark engineer with solid design experience, capable of contributing to architectural decisions while leading complex implementation and optimization efforts.

Job Responsibility

  • Platform Engineering & Modernization: Refactor and modernize existing Spark pipelines to Databricks native architectures
  • Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
  • Enhance and extend existing processing logic using optimized Spark (JavaSpark / PySpark) on Databricks
  • Databricks Native Development: Build and optimize solutions using Databricks features, including Delta Lake, Databricks Workflows for orchestration and Auto scaling and job clusters
  • Design & Solution Engineering: Contribute to low and mid level architecture and design
  • Translate high level architecture into detailed technical designs
  • Define data models, pipeline patterns, and reusable components
  • Ensure solutions are scalable, maintainable, and production ready
  • Performance Optimization & Simplification: Analyze, improve Spark job performance and simplify complex or over engineered pipelines into standardized, efficient patterns
  • Engineering Standards & Best Practices: Follow and contribute to Databricks and Spark engineering standards
  • Write clean, modular, and testable code
  • Contribute to shared frameworks, reusable libraries, and quality standards
  • Collaboration & Stakeholder Engagement: Work closely with senior architects, platform teams, and DevOps engineers
  • Provide technical inputs, troubleshooting support, and implementation guidance
  • Participate in design discussions and technical decision making
  • Testing & Quality Assurance: Develop unit, integration, and data validation tests
  • Support production releases and post deployment validation

Requirements

  • 10+ years in data engineering or distributed systems
  • Strong expertise in Apache Spark (JavaSpark / PySpark), Databricks on AWS, and Delta Lake
  • Experience with AWS services and large‑scale distributed data processing
  • Experience modernizing or refactoring legacy data platforms into cloud‑based architectures
  • Strong background in Spark performance tuning and large‑scale batch optimization
  • Ability to translate architecture into implementable designs
  • Understanding of data modeling and pipeline orchestration patterns
  • Strong problem‑solving mindset for complex distributed systems
  • Comfortable working in time‑bound, high‑impact environments
  • Proactive, accountable, and collaborative
  • Clear communication skills across global teams
  • Bachelor’s degree/University degree or equivalent experience
Citi - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Databricks & Apache Spark Developer - Vice President

8 matching positions

Pyspark Big Data Senior Developer - Vice President

We are building an A-team of highly skilled and autonomous engineers, and we are...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of extensive, hands-on experience as a Senior Big Data Developer, with a strong emphasis on PySpark and the Apache Spark ecosystem, operating as a player/coach
  • Expert proficiency in Python, with a proven track record of developing robust, scalable, and high-performance PySpark applications for large-scale data processing
  • Deep understanding and extensive hands-on experience with Apache Spark (Spark Core, Spark SQL, Spark Streaming) and its ecosystem
  • Experience with distributed computing frameworks such as Hadoop (HDFS, YARN)
  • Expert proficiency in SQL and extensive experience with data warehousing concepts and technologies (e.g., Hive, Snowflake, Redshift, Databricks SQL)
  • Proven experience with various data storage formats (e.g., Parquet, ORC, Avro) and data lake solutions (e.g., Delta Lake, Iceberg)
  • Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
  • Strong experience with Apache Kafka for building real-time data pipelines and event-driven architectures
  • Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
  • Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is a mandatory requirement
Job Responsibility
Job Responsibility
  • Operate end-to-end in the design, development, and implementation of robust big data solutions, ensuring optimal performance, scalability, data quality, and security
  • Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for big data processing and analytical solutions
  • Act as a player/coach within the team, mentoring junior members and leading by example in the development of efficient and innovative big data architectures
  • Design, develop, and optimize large-scale data pipelines using PySpark for data ingestion, transformation, and aggregation, always with an eye towards efficiency and domain relevance
  • Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka
  • Design and implement sophisticated data warehousing solutions and dimensional models for efficient data storage and retrieval, ensuring alignment with business needs
  • Work with various distributed data storage technologies, including distributed file systems (e.g., HDFS, S3) and NoSQL databases (e.g., MongoDB, Cassandra), selecting the right tool for the right problem
  • Implement efficient data processing and storage strategies to optimize the performance and scalability of big data applications, with a strong focus on the 'why' behind the technology choices
  • Champion best practices in software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines
  • Demonstrate high autonomy and agency in driving projects forward, making informed decisions, and proactively identifying areas for improvement
  • Fulltime
Read More
Arrow Right

Senior PySpark Developer - Vice President

We are seeking a highly skilled and experienced Senior PySpark Developer to join...
Location
Location
United States , Tampa
Salary
Salary:
113840.00 - 170760.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
June 05, 2026
Flip Icon
Requirements
Requirements
  • 10+ years of experience in Applications Development, Systems Analysis, or equivalent senior engineering roles
  • Extensive hands‑on experience delivering enterprise‑scale, database‑driven platforms in a regulated environment
  • Expert-level proficiency in Python programming, including object-oriented design, data structures, algorithms, and extensive experience with various Python libraries
  • Deep expertise in developing, optimizing, and deploying PySpark applications for large-scale data processing, ETL, and real-time analytics on distributed systems (e.g., Spark SQL, Spark Streaming, DataFrames)
  • Strong understanding of Apache Spark architecture, Hadoop ecosystem, and experience with distributed computing concepts. Familiarity with big data storage formats (e.g., Parquet, ORC)
  • Solid experience with both relational databases (e.g., Oracle) and NoSQL databases (e.g., MongoDB). Strong SQL writing and optimization skills
  • Experience in designing, developing, and consuming RESTful APIs using Python frameworks (e.g., Flask, FastAPI, Django REST Framework)
  • Strong understanding and practical experience with CI/CD tools (e.g., Jenkins) and containerization technologies (Docker, Kubernetes)
  • Expert-level proficiency with Git
  • Experience with unit testing (e.g., Pytest), integration testing, and performance testing frameworks for Python and PySpark applications
Job Responsibility
Job Responsibility
  • Design, develop, and implement robust, scalable, and high-performance data pipelines and applications using Python, PySpark, and Big Data technologies
  • Work autonomously to analyze requirements, propose technical solutions, and deliver high-quality code and data products, ensuring alignment with architectural standards and business objectives
  • Utilize expertise in various Big Data platforms (e.g., Hadoop, Hive, Kafka, Spark) to process, transform, and manage large datasets efficiently
  • Write complex SQL queries, stored procedures, and optimize database performance for large-scale data warehousing and analytics solutions
  • Develop and enhance ETL (Extract, Transform, Load) processes, ensuring data quality, integrity, and timely delivery. Experience with various ETL tools and methodologies is a plus
  • Proactively research, evaluate, and integrate new and emerging technologies, frameworks, and tools to improve development processes and solution capabilities
  • Ensure adherence to coding standards, conduct thorough code reviews, and implement best practices for software development, data governance, and security
  • Diagnose and resolve complex technical issues related to data pipelines, performance bottlenecks, and system integrations in a fast-paced environment
  • Collaborate effectively with cross-functional teams including architects, data scientists, business analysts, and QA engineers. Provide technical guidance and mentorship to junior team members
  • Identify opportunities to use AI tools to speed up development, code reviews, unit testing and deployment.
What we offer
What we offer
  • medical, dental & vision coverage
  • 401(k)
  • life, accident, and disability insurance
  • wellness programs
  • paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays
  • discretionary and formulaic incentive and retention awards
  • Fulltime
!
Read More
Arrow Right

Vice President, Big Data Scala Engineer

We are seeking an experienced and highly skilled Vice President, Big Data Scala ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field
  • 12+ years of progressive experience in software development, with at least 5+ years focusing on big data technologies
  • 3+ years of experience in a leadership or senior architectural role
  • Extensive hands-on experience with Scala for big data processing
  • Demonstrated expertise with Apache Spark (Spark Core, Spark SQL, Spark Streaming)
  • Strong experience with distributed systems and big data ecosystems (e.g., Hadoop, Kafka, Cassandra, HBase, Delta Lake, Snowflake, Databricks)
  • Proficiency with cloud platforms (AWS, Azure, GCP) and their big data services (e.g., EMR, Redshift, Glue, DataProc, BigQuery)
  • Experience with containerization technologies (Docker, Kubernetes) and CI/CD pipelines
  • Solid understanding of data warehousing concepts, ETL/ELT processes, and data modeling
  • Familiarity with functional programming paradigms in Scala
Job Responsibility
Job Responsibility
  • Lead the architecture, design, and development of high-performance, scalable, and reliable big data processing systems using Scala and Apache Spark
  • Drive technical vision and strategy for big data initiatives
  • Evaluate and recommend new technologies and tools
  • Design, develop, and optimize data pipelines for ingestion, transformation, and storage of massive datasets
  • Implement robust and efficient data processing jobs using Scala and Spark (batch and streaming)
  • Ensure data quality, integrity, and security
  • Promote and enforce best practices in coding, testing, and deployment
  • Mentor and guide a team of talented big data engineers
  • Conduct code reviews, provide constructive feedback
  • Participate in the recruitment and hiring
What we offer
What we offer
  • Opportunity to work on cutting-edge big data technologies and impactful projects
  • A collaborative and innovative work environment
  • Competitive compensation and benefits package
  • Opportunities for professional growth and career advancement
  • Fulltime
Read More
Arrow Right

Senior Data Software Engineer (Python & PySpark) - Vice President

The Senior Data Software Engineer is a senior level position responsible for est...
Location
Location
Singapore , Singapore
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
  • 7+ years of experience in data engineering, with a strong focus on Python and big data technologies
  • Proven expertise in designing and implementing large-scale data processing solutions using PySpark
  • Extensive experience with distributed computing frameworks like Apache Spark
  • Strong understanding of data warehousing concepts, dimensional modeling, and ETL/ELT principles
  • Proficiency in SQL and experience with various relational and NoSQL databases
  • Experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, ADLS, Google Cloud Storage, Redshift, Snowflake, BigQuery, Databricks)
  • Familiarity with workflow orchestration tools (e.g., Apache Airflow, Azure Data Factory, AWS Step Functions)
  • Experience with version control systems (e.g., Git)
  • Excellent problem-solving, analytical, and communication skills.
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
  • Fulltime
Read More
Arrow Right
New

Graphic designer & video producer

As a graphic designer & video producer at LumApps, you'll work closely with our ...
Location
Location
France , Lyon; Tassin-la-Demi-Lune; Paris
Salary
Salary:
Not provided
lumapps.com Logo
LumApps
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4-6 years of experience in digital design and/or video production in a B2B tech or SaaS environment
  • A strong portfolio that shows both design and video work across digital and print formats
  • Highly fluent in both French and English — written and verbal — with the ability to move seamlessly between the two
  • Solid experience with video editing and production tools: Adobe Premiere Pro, After Effects, and/or Final Cut Pro
  • Comfortable with motion graphics production, whether in After Effects or an equivalent tool
  • Proficiency in Figma and Adobe Creative Suite for digital design, layout and print production
  • Strong skills in Google Slides with a sharp eye for layout, typography, and visual hierarchy
  • A track record of working across time zones with a globally distributed team
  • Strong project management skills — you handle multiple briefs, hit deadlines, and communicate clearly about timelines and dependencies
  • An understanding of how design and video influence marketing metrics and conversion
Job Responsibility
Job Responsibility
  • Design and produce engaging visual content across digital channels: Website, email, paid media, social, and downloadable assets like eBooks, guides, and one-pagers
  • Own end-to-end video production — from concept and scripting through to filming, editing, and delivery — for social media, product explainers, webinars, customer stories, and event content
  • Create motion graphics and animated content for social media, presentations, and web campaigns
  • Partner with field marketing on print and event design: Booth backdrops, signage, banners, invitations, direct mail, and conference materials
  • Build and format Google Slides and presentation templates that are clear, consistent, and on-brand
  • Expand and evolve our global brand asset library: Icons, illustrations, templates, and motion assets
  • Maintain organized, accessible design files and video assets for the wider team
  • Collaborate with external agencies, freelancers, and production partners as needed
What we offer
What we offer
  • Hybrid work model – 2 days at the office, 3 days remote
  • RTT days – ~10 extra days off per year
  • Meal vouchers (SWILE) + free snacks & coffee
  • Yoga classes
  • Supportive parental leave and family moments
  • Health insurance (ALAN) – 60% covered + full life & disability cover
  • Afterworks, team celebrations & seasonal parties
  • Equipment of your choice
  • French & English lessons, professional development & access to Leeto CSE
  • Fulltime
Read More
Arrow Right
New

Warehouse Coordinator

The Warehouse Coordinator (WHC) is responsible for the maintenance and managemen...
Location
Location
United States , Hobbs
Salary
Salary:
Not provided
cactuswhd.com Logo
Cactus Wellhead LLC
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Associate's degree (AA) or equivalent from two-year college or technical school
  • or six months to one-year related experience and/or training
  • or equivalent combination of education and experience
  • Minimum of 1 year warehouse/manufacturing/construction experience with average experience of 3+ years
  • Licensure and/or experience in the operation of forklifts and other general construction equipment as necessary will be required
  • A high school graduate or GED preferred
  • minimum 3 years of experience will be accepted in lieu of education. In addition, 3 plus years of experience in a related field, with general knowledge of warehouse procedures preferred
  • Ability to speak fluent English in writing and speaking
  • Results oriented self-driven individual with strong organizational and people skills
  • Ability to read and interpret documents such as safety rules, operating and maintenance instructions, and procedure manuals
Job Responsibility
Job Responsibility
  • Uphold safety values and culture of FlexSteel including personal commitment to safety, ability to identify and mitigate safety risks, follow procedures, and perform JSAs, safety audits and safety meetings
  • Perform planned and unplanned repairs and maintenance to equipment on site and in a warehouse setting
  • Provide maintenance and housekeeping at the facility to ensure a clean, organized, and safe work environment. Ensure facility passes safety and warehouse audits as required
  • Support the receipt and inspection of materials on arrival at site in terms of integrity, conformity, and quantities, as per Mechanical Integrity Plan and manufacturing recommendations, Purchase Orders and Packing Lists
  • Support the adequate handling, storage, and preservation of materials, in accordance with the relevant procedures defined by HSEQ, manufacturers, engineering, policy, clients and projects
  • Ensure all assets, tools and load outs are available to meet A+ customer service and expectation
  • Ensure the correct receipt, issue, and restock of all parts in Visual
  • Work with staff to identify and solve material and logistic problems within the organization
  • Maintain a full inventory of all stores with special attention to fast moving and long lead items
  • Responsible for timely and accurate reporting including end of year and end of month counts, auditing, investigating, and reconciling all inventories with the ability to take consequence action
What we offer
What we offer
  • Cactus Companies maintains a drug-free workplace and participates in E-Verify
  • Fulltime
Read More
Arrow Right
New

Data Engineering Intern

At Boeing, we innovate and collaborate to make the world a better place. We’re c...
Location
Location
Canada , Richmond
Salary
Salary:
23.50 - 27.00 CAD / Hour
boeing.com Logo
Boeing
Expiration Date
June 05, 2026
Flip Icon
Requirements
Requirements
  • Currently enrolled as a full-time student with an accredited university or college with studies in Computer Science or similar technical discipline
  • Must be at least a 2nd year or higher student with studies in a relevant program
  • Experience with data engineering and data processing tools
  • Experience with Python
  • Experience with relational and non-relational database technologies
  • Willing and able to work 40 hours per week during the internship
  • Provide consent to Canadian Government Controlled Goods Program (CGP) assessment and willing and eligible to work on government and defense-related programs
  • Must be legally able to work in Canada
  • Individuals must not pose a risk for safeguarding of controlled goods
  • Must be eligible to handle US export-controlled data
Job Responsibility
Job Responsibility
  • Work directly with an assigned mentor or a buddy
  • Design and implement robust, highly performant, and scalable streaming data pipelines using Python and Spark
  • Define, enforce, and measure data quality as part of the data pipelines
  • Work with cross-functional teams to solve their complex data problems
  • Foster the creation of a data-driven culture, data stewardship, related competencies, and data literacy across the organization
  • Generate ad-hoc data analysis
  • Report status and progress to Manager and Team Lead
  • Stay up to date on all new Spark/Databricks features and functionalities and find ways to introduce them to the organization
What we offer
What we offer
  • Opportunity to take part in impactful business projects
  • Network with experienced professionals
  • Engage in a strong student network
  • Gain valuable exposure in the Aerospace industry
  • Competitive base pay and incentive programs
  • Industry-leading tuition assistance program pays your institution directly
  • Resources and opportunities to grow your career
  • Up to $10,000 match when you support your favorite nonprofit organizations
  • Fulltime
!
Read More
Arrow Right
New

Pastry Sous Chef

We are seeking a talented and passionate Pastry Sous Chef to support the creatio...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.marriott.com Logo
Marriott Bonvoy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience in pastry or bakery operations within a hotel or premium restaurant environment
  • Strong technical knowledge of baking, pastry techniques and dessert presentation
  • Creative flair with the ability to design and execute innovative pastry concepts
  • Leadership skills with the ability to motivate, train and inspire a team
  • Strong organisational and time-management abilities
  • High attention to detail and commitment to quality
  • Knowledge of food safety, hygiene and HACCP standards
  • Ability to work under pressure while maintaining consistency and excellence
  • Excellent communication and collaboration skills
Job Responsibility
Job Responsibility
  • Support the daily operation of the pastry kitchen, ensuring smooth and efficient service
  • Lead by example through hands-on preparation, baking and presentation of high-quality pastries and desserts
  • Supervise and coordinate the activities of pastry chefs, cooks and kitchen associates
  • Drive creativity by developing new recipes, seasonal menus and artistic presentations
  • Ensure consistency in taste, quality and visual presentation across all pastry offerings
  • Maintain strict adherence to food safety, sanitation and hygiene standards
  • Monitor food preparation, storage and handling processes in line with company and legal requirements
  • Support inventory management, purchasing and cost control to meet budget targets
  • Train, coach and develop team members, fostering a culture of continuous learning and improvement
  • Assist with recruitment, onboarding and performance management of pastry team members
What we offer
What we offer
  • Competitive salary designed to recognise excellence
  • Workplace pension
  • Company sick pay
  • Additional holiday allowance
  • Access to BenefitHub's exclusive retail, wellness and travel privileges
  • Friends & Family preferred rates at Marriott hotels worldwide
  • Clear pathway for internal promotions and transfers
  • Cross-department training to refine your craft and broaden your expertise
  • Expert-led development programmes
  • Continuous learning through structured programmes
  • Fulltime
Read More
Arrow Right