CrawlJobs Logo
Citi Logo Citi · IT - Software Development

Pyspark Big Data Senior Developer - Vice President

Canada, Mississauga 120800.00 - 170800.00 USD / Year · Job Posted May 03, 2026
Apply Position
Job Link Share

Job Description

We are building an A-team of highly skilled and autonomous engineers, and we are seeking an exceptional PySpark Big Data Senior Developer to join our dynamic and focused squads. This role is for a hands-on player/coach who thrives in a high-autonomy environment, is deeply committed to leveraging AI for maximum productivity, and possesses a profound understanding of the functional domains our work impacts. The ideal candidate will be instrumental in designing, developing, and optimizing large-scale data processing solutions using PySpark and cutting-edge big data technologies. We are looking for an AI-first thinker who can raise the bar, coach others, and strategically contribute to our evolving technology landscape.

Job Responsibility

  • Operate end-to-end in the design, development, and implementation of robust big data solutions, ensuring optimal performance, scalability, data quality, and security
  • Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for big data processing and analytical solutions
  • Act as a player/coach within the team, mentoring junior members and leading by example in the development of efficient and innovative big data architectures
  • Design, develop, and optimize large-scale data pipelines using PySpark for data ingestion, transformation, and aggregation, always with an eye towards efficiency and domain relevance
  • Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka
  • Design and implement sophisticated data warehousing solutions and dimensional models for efficient data storage and retrieval, ensuring alignment with business needs
  • Work with various distributed data storage technologies, including distributed file systems (e.g., HDFS, S3) and NoSQL databases (e.g., MongoDB, Cassandra), selecting the right tool for the right problem
  • Implement efficient data processing and storage strategies to optimize the performance and scalability of big data applications, with a strong focus on the 'why' behind the technology choices
  • Champion best practices in software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines
  • Demonstrate high autonomy and agency in driving projects forward, making informed decisions, and proactively identifying areas for improvement
  • Proactively leverage and contribute to the development of AI-powered development tools, including internal Citi AI tools like Copilot, Claude Code, Codex, and Antigravity, to significantly enhance productivity, code quality, and accelerate development cycles
  • Lead technical discussions and contribute strategically to the evolution of our big data technology stack, always seeking innovative approaches
  • Troubleshoot and resolve complex technical issues within big data environments, demonstrating strong analytical and problem-solving skills

Requirements

  • 6+ years of extensive, hands-on experience as a Senior Big Data Developer, with a strong emphasis on PySpark and the Apache Spark ecosystem, operating as a player/coach
  • Expert proficiency in Python, with a proven track record of developing robust, scalable, and high-performance PySpark applications for large-scale data processing
  • Deep understanding and extensive hands-on experience with Apache Spark (Spark Core, Spark SQL, Spark Streaming) and its ecosystem
  • Experience with distributed computing frameworks such as Hadoop (HDFS, YARN)
  • Expert proficiency in SQL and extensive experience with data warehousing concepts and technologies (e.g., Hive, Snowflake, Redshift, Databricks SQL)
  • Proven experience with various data storage formats (e.g., Parquet, ORC, Avro) and data lake solutions (e.g., Delta Lake, Iceberg)
  • Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
  • Strong experience with Apache Kafka for building real-time data pipelines and event-driven architectures
  • Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
  • Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is a mandatory requirement
  • A strong 'AI-first thinker' mindset, demonstrating how to leverage and integrate AI tools into the development workflow for continuous improvement
  • Experience with or a strong willingness to actively explore and implement other AI-powered tools to optimize big data development processes
  • Strong ability to articulate the functional domain being worked in, understanding the business context, and explaining 'why' the technical solutions matter
  • Advanced understanding of data structures, algorithms, and performance optimization techniques for large-scale distributed data processing
  • Experience with RESTful API design and development for data ingestion or exposure points
  • Familiarity with containerization technologies (e.g., Docker, Kubernetes) for deploying and managing big data applications
  • Expert proficiency with version control systems, especially Git, and advanced branching strategies
  • Exceptional problem-solving, analytical, and debugging skills in highly complex, distributed big data environments
  • Superior communication and interpersonal skills, with a proven ability to work effectively and autonomously within small, high-performing teams, and to mentor others
  • Demonstrated high autonomy and agency in tackling complex challenges and delivering impactful solutions
  • Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related quantitative field is required

Nice to have

  • Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
  • Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
Citi - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Pyspark Big Data Senior Developer - Vice President

8 matching positions

Senior Data Software Engineer (Python & PySpark) - Vice President

The Senior Data Software Engineer is a senior level position responsible for est...
Location
Location
Singapore , Singapore
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
  • 7+ years of experience in data engineering, with a strong focus on Python and big data technologies
  • Proven expertise in designing and implementing large-scale data processing solutions using PySpark
  • Extensive experience with distributed computing frameworks like Apache Spark
  • Strong understanding of data warehousing concepts, dimensional modeling, and ETL/ELT principles
  • Proficiency in SQL and experience with various relational and NoSQL databases
  • Experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, ADLS, Google Cloud Storage, Redshift, Snowflake, BigQuery, Databricks)
  • Familiarity with workflow orchestration tools (e.g., Apache Airflow, Azure Data Factory, AWS Step Functions)
  • Experience with version control systems (e.g., Git)
  • Excellent problem-solving, analytical, and communication skills.
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
  • Fulltime
Read More
Arrow Right

Data Engineer (Big Data, Python, Databricks) - Assistant Vice President

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Chennai, Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of relevant handson experience in Big Data technologies like Cloudera, Python, HQL, Java/PySpark
  • Knowledge on Machine Learning, AI would be added advantage
  • Experience in systems analysis, data analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Data Engineering Python and Pyspark - Assistant Vice President

Location
Location
India , Chennai; Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8 to 14 years of relevant experience in Data engineering role
  • Advanced SQL/ RDBMS skills and experience with relational databases and database design
  • Strong proficiency in object-oriented languages: Python, PySpark is must
  • Experience working with Bigdata - Hive/Impala/S3/HDFS
  • Experience working with data ingestion tools such as Talend or Ab Initio
  • Strong experience in data analysis, data exploration, and trend identification
  • Solid understanding of machine learning fundamentals like Regression, classification, clustering, Feature engineering
  • Strong proficiency in scripting languages like Bash, UNIX Shell scripting
  • Strong proficiency in data pipeline and workflow management tools
  • Strong project management and organizational skills
Job Responsibility
Job Responsibility
  • Build and maintain batch or real-time data pipelines in data platform
  • Maintain and optimize the data infrastructure required for accurate extraction, transformation, and loading of data from a wide variety of data sources
  • Develop ETL (extract, transform, load) processes to help extract and manipulate data from multiple sources
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Automate data workflows such as data ingestion, aggregation, and ETL processing
  • Analyze large datasets to identify trends, anomalies, and behavioral patterns
  • Apply machine learning and AI concepts (supervised / unsupervised learning) to support predictive and exploratory analysis
  • Perform feature engineering and data transformations to enable ML models
  • Support trend analysis, segmentation, clustering, and forecasting use cases
  • Interpret analytical results and translate them into business-friendly insights
  • Fulltime
Read More
Arrow Right

Senior Data Engineer (ETL/Big Data/Python) - Assistant Vice President

CITI Bank Enterprise Analytical Services organization is seeking a highly skille...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Information Technology, or a related field
  • 8+ years of experience in data engineering, software development, or a similar role, with at least 2 years in a lead capacity
  • Proven experience with ETL pipeline design and development
  • Expert proficiency in Python and Spark
  • Strong experience with advanced scripting
  • Deep knowledge of relational databases (Oracle) and NoSQL databases (MongoDB), Snowflake
  • Solid understanding of data modeling, data warehousing, and performance tuning
  • Experience with CI/CD pipelines and tools (e.g., Jenkins, GitLab CI, GitHub Actions)
  • Experience with monitoring, logging, and alerting tools (e.g., Prometheus, Grafana, ELK Stack, Splunk, Datadog)
  • Excellent problem-solving and analytical skills, with a keen eye for detail
Job Responsibility
Job Responsibility
  • Design, develop, and implement ETL based reporting framework, focusing on data collection, transformation, and presentation
  • Utilize a robust technology stack including Databricks, Spark, Hive, Ozone, and Hadoop to process large volumes of data efficiently
  • Write and optimize complex ETL jobs using PySpark and advanced Python scripting
  • Design and manage data storage in various databases, including Oracle, MongoDB and Snowflake for flexible data models
  • Develop and maintain automated job schedules using Autosys, Apache Airflow for seamless data pipeline execution
  • Ability to develop visualization reporting dashboards
  • Able to leverage enterprise approved productivity tools like Copilot, etc., in daily analysis & development tasks
  • Act as the technical subject matter expert (SME) for the reporting framework, providing guidance and mentorship to the development team
  • Lead requirements gathering discussions with business stakeholders and product owners to understand reporting needs and translate them into technical solutions
  • Coordinate closely with the QA team to ensure thorough testing and data validation, maintaining high standards of data accuracy and integrity
  • Fulltime
Read More
Arrow Right

Data Analytics Lead - Data Scientist - Vice President

The Data Analytics Lead / Data Scientist is a strategic professional who stays a...
Location
Location
India , Chennai; Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10-15 years of relevant experience in Data Analytics, Data Science, or Advanced Analytics roles
  • Advanced proficiency in SQL and relational database concepts
  • Strong programming experience in Python (required)
  • PySpark preferred
  • Hands-on experience building and deploying machine learning models (supervised and unsupervised)
  • Experience with ML libraries such as scikit-learn, XGBoost, TensorFlow, or PyTorch
  • Strong knowledge of statistical modeling, feature engineering, and model validation techniques
  • Experience with BI tools such as Tableau or Power BI
  • Familiarity with MLOps practices (model deployment, monitoring, versioning) is strongly preferred
  • Experience working with large-scale enterprise or financial datasets
Job Responsibility
Job Responsibility
  • Integrates subject matter and industry expertise within a defined area
  • Contributes to data analytics standards around which others will operate
  • Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinate and contribute to the objectives of the entire function
  • Employs developed communication and diplomacy skills are required in order to guide, influence and convince others, in particular colleagues in other areas and occasional external customers
  • Resolves occasionally complex and highly variable issues
  • Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/ taken
  • Responsible for volume, quality, timeliness and delivery of data science projects along with short-term planning resource planning
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Lead the design and execution of complex data analysis and AI/ML initiatives across large, structured, and unstructured datasets
  • Develop and deploy predictive, classification, clustering, and forecasting models to support business strategy and risk management
  • Fulltime
Read More
Arrow Right

Application Development Lead - Senior Vice President

The Applications Development Technology Senior Lead Analyst is a senior level po...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of relevant experience
  • Experience in implementing projects
  • Experience in systems analysis and programming of software applications
  • Demonstrated Subject Matter Expert (SME) in area(s) of Applications Development
  • Demonstrated knowledge of client core business functions
  • Demonstrated leadership, project management, and development skills
  • Relationship and consensus building skills
  • 10-12 years of experience in developing and managing Enterprise Applications
  • Core Java, Spring Hibernate and related skills
  • Solid foundation in OOP concepts
Job Responsibility
Job Responsibility
  • Lead integration of functions to meet goals, deploy new products, and enhance processes
  • Analyze complex business processes, system processes, and industry standards to define and develop solutions to high level problems
  • Provide expertise in area of advanced knowledge of applications programming and plan assignments involving large budgets, cross functional project, or multiple projects
  • Develop application methodologies and standards for program analysis, design, coding, testing, debugging, and implementation
  • Utilize advanced knowledge of supported main system flows and comprehensive knowledge of multiple areas to achieve technology goals
  • Consult with end users to identify system function specifications and incorporate into overall system design
  • Allocate work, and act as an advisor/coach developers, analysts, and new team members
  • Influence and negotiate with senior leaders and communicate with external parties
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Design & Development: Create and optimize scalable ETL (Extraction, Transformation, Loading) pipelines using PySpark for massive datasets
  • Fulltime
Read More
Arrow Right

Application Development Lead - Senior Vice President

The Applications Development Technology Senior Lead Analyst is a senior level po...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of relevant experience
  • Experience in implementing projects
  • Experience in systems analysis and programming of software applications
  • Demonstrated Subject Matter Expert (SME) in area(s) of Applications Development
  • Demonstrated knowledge of client core business functions
  • Demonstrated leadership, project management, and development skills
  • Relationship and consensus building skills
  • 10-12 years in developing and managing enterprise-grade applications
  • Solid foundation in OOP concepts
  • Expertise in PySpark, HDFS, Hive, Sqoop, and Hadoop for Big Data environments
Job Responsibility
Job Responsibility
  • Lead integration of functions to meet goals, deploy new products, and enhance processes
  • Analyze complex business processes, system processes, and industry standards to define and develop solutions to high level problems
  • Provide expertise in area of advanced knowledge of applications programming and plan assignments involving large budgets, cross functional project, or multiple projects
  • Develop application methodologies and standards for program analysis, design, coding, testing, debugging, and implementation
  • Utilize advanced knowledge of supported main system flows and comprehensive knowledge of multiple areas to achieve technology goals
  • Consult with end users to identify system function specifications and incorporate into overall system design
  • Allocate work, and act as an advisor/coach developers, analysts, and new team members
  • Influence and negotiate with senior leaders and communicate with external parties
  • Appropriately assess risk when business decisions are made
  • Design & Development: Create and optimize scalable ETL pipelines using PySpark for massive datasets
  • Fulltime
Read More
Arrow Right

Senior Java -Spark-Bigdata Engineer-Assistant Vice President

The Applications Development Senior Programmer Analyst is a senior-level positio...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7-10 years of relevant experience in Data Engineering or a similar role, preferably within the Financial Services industry
  • Senior-level experience in an Applications Development or Data Engineering role
  • Consistently demonstrates clear and concise written and verbal communication
  • Demonstrated problem-solving and decision-making skills
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor's degree/University degree or equivalent experience
  • Hands-on expertise in Java (8+), Spring Boot, Python, and PySpark for building high-performance data applications
  • Extensive experience with the BigData ecosystem, including Apache Spark for large-scale data processing
  • Solid understanding of Data Warehouse concepts, design principles, and best practices
  • Strong proficiency with both relational SQL databases and NoSQL databases (e.g., MongoDB, Couchbase)
Job Responsibility
Job Responsibility
  • Utilize expert knowledge of data engineering principles, big data technologies, and software development best practices to design and implement robust data solutions
  • Collaborate with business stakeholders, data scientists, and other technology teams to understand data requirements and deliver effective solutions
  • Apply deep expertise in programming languages like Python and Java for building high-performance data processing applications
  • Ensure data solutions are secure, scalable, and adhere to the firm's security and architectural standards
  • Mentor and guide junior engineers, fostering a culture of technical excellence and continuous learning
  • Lead the analysis of complex data-related issues, identify root causes, and implement sustainable solutions
  • Operate with a high degree of autonomy and independence, exercising sound judgment and decision-making
  • Act as a Subject Matter Expert (SME) in big data technologies for senior stakeholders and other team members
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Fulltime
Read More
Arrow Right