Senior Big Data Pyspark Developer Job at Citi (Mississauga)

Pyspark Big Data Senior Developer - Vice President

We are building an A-team of highly skilled and autonomous engineers, and we are...

Location

Canada , Mississauga

Salary:

120800.00 - 170800.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

6+ years of extensive, hands-on experience as a Senior Big Data Developer, with a strong emphasis on PySpark and the Apache Spark ecosystem, operating as a player/coach
Expert proficiency in Python, with a proven track record of developing robust, scalable, and high-performance PySpark applications for large-scale data processing
Deep understanding and extensive hands-on experience with Apache Spark (Spark Core, Spark SQL, Spark Streaming) and its ecosystem
Experience with distributed computing frameworks such as Hadoop (HDFS, YARN)
Expert proficiency in SQL and extensive experience with data warehousing concepts and technologies (e.g., Hive, Snowflake, Redshift, Databricks SQL)
Proven experience with various data storage formats (e.g., Parquet, ORC, Avro) and data lake solutions (e.g., Delta Lake, Iceberg)
Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
Strong experience with Apache Kafka for building real-time data pipelines and event-driven architectures
Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is a mandatory requirement

Job Responsibility

Operate end-to-end in the design, development, and implementation of robust big data solutions, ensuring optimal performance, scalability, data quality, and security
Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for big data processing and analytical solutions
Act as a player/coach within the team, mentoring junior members and leading by example in the development of efficient and innovative big data architectures
Design, develop, and optimize large-scale data pipelines using PySpark for data ingestion, transformation, and aggregation, always with an eye towards efficiency and domain relevance
Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka
Design and implement sophisticated data warehousing solutions and dimensional models for efficient data storage and retrieval, ensuring alignment with business needs
Work with various distributed data storage technologies, including distributed file systems (e.g., HDFS, S3) and NoSQL databases (e.g., MongoDB, Cassandra), selecting the right tool for the right problem
Implement efficient data processing and storage strategies to optimize the performance and scalability of big data applications, with a strong focus on the 'why' behind the technology choices
Champion best practices in software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines
Demonstrate high autonomy and agency in driving projects forward, making informed decisions, and proactively identifying areas for improvement

Fulltime

PySpark Big Data Developer

The Applications Development Intermediate Programmer Analyst is an intermediate ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

2-5 years of relevant experience in the Financial Service industry
Intermediate level experience in Applications Development role
Consistently demonstrates clear and concise written and verbal communication
Demonstrated problem-solving and decision-making skills
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Bachelor's degree/University degree or equivalent experience
4-8 years of experience in developing and managing Enterprise Applications
Expertise in PySpark, HDFS, Hive, Sqoop, and Hadoop
Good exposure to SQL Server and ORACLE databases
Proficient in Shell Scripting

Job Responsibility

Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
Apply fundamental knowledge of programming languages for design specifications
Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
Serve as advisor or coach to new or lower level analysts
Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

PySpark Big Data Developer

The Applications Development Intermediate Programmer Analyst is an intermediate ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

2-5 years of relevant experience in the Financial Service industry
Intermediate level experience in Applications Development role
Consistently demonstrates clear and concise written and verbal communication
Demonstrated problem-solving and decision-making skills
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Bachelor’s degree/University degree or equivalent experience
Enterprise Application Development: 6-8 years in developing and managing enterprise-grade applications
Object-Oriented Programming (OOP): Solid foundation in OOP concepts
Big Data Development: Expertise in PySpark, HDFS, Hive, Sqoop, and Hadoop for Big Data environments
Database Technologies: Good exposure to SQL Server and ORACLE databases

Job Responsibility

Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
Apply fundamental knowledge of programming languages for design specifications
Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
Serve as advisor or coach to new or lower level analysts
Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Senior Big Data Developer

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

8-14 years of relevant experience
Experience in systems analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Bachelor’s degree/University degree or equivalent experience
Strong Object-Oriented Programming (OOP) concepts
proficient in Python (specifically for PySpark)
Extensive experience with Apache Spark (PySpark), Hadoop, and related components like Hive and Sqoop
skilled in writing shell scripts

Job Responsibility

Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
Ensure essential procedures are followed and help define operating standards and processes
Serve as advisor or coach to new or lower level analysts
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Senior Pyspark Data Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

Canada , Mississauga

Salary:

94300.00 - 141500.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

5-8 years of relevant experience
Experience in systems analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
Proficiency in Python programming
Strong expertise in data processing frameworks such as Apache Spark, Hadoop
Expertise in Data Lakehouse technologies (Apache Iceberg, Trino, Deltalake)
Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)

Job Responsibility

Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
Ensure essential procedures are followed and help define operating standards and processes
Serve as advisor or coach to new or lower level analysts
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Senior Pyspark Data Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

Canada , Mississauga

Salary:

94300.00 - 141500.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

5-8 years of relevant experience
Experience in systems analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
Proficiency in Python programming
Strong expertise in data processing frameworks such as Apache Spark, Hadoop
Expertise in Data Lakehouse technologies (Apache Iceberg, Trino, Deltalake)
Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)

Job Responsibility

Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
Ensure essential procedures are followed and help define operating standards and processes
Serve as advisor or coach to new or lower level analysts
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Senior Big Data Engineer

Citi is looking for a Senior Big Data Engineer to design, build, and optimize la...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Hands-on expertise in PySpark and Big Data processing
Practical knowledge of the Hadoop ecosystem, including Hive, HDFS, Sqoop, Spark, Impala, and Scala
Proficiency in complex SQL query development
Solid understanding of distributed systems architecture
Demonstrated knowledge of data modelling and data design
Competence in shell scripting and job scheduling using Autosys
Strong analytical and problem-solving ability
Clear and effective communication skills

Job Responsibility

Build and maintain scalable data pipelines using PySpark
Design and develop solutions across the Hadoop ecosystem
Develop and manage real-time and batch data workflows using streaming data platforms
Write complex SQL queries
Design and implement data models and data architecture patterns
Automate pipeline scheduling and orchestration using shell scripting and Autosys
Independently identify, assess, and resolve technical risks and data issues

What we offer

Hybrid working model
Access to continuous learning and development programmes
Exposure to large-scale, complex data systems
A collaborative and inclusive team environment
Competitive compensation and a comprehensive benefits package
Wellbeing and work-life balance support
Global connectivity

Fulltime

Senior Big Data Engineer

Start.io is a mobile marketing and audience platform. Start.io empowers the mobi...

Location

Poland , Warsaw

Salary:

Not provided

Start.io

Expiration Date

Until further notice

Requirements

B.Sc. or M.Sc. in Computer Science, Software Engineering, or other equivalent fields.
5+ years of hands-on experience in backend or ML engineering
Strong Python skills and experience working with distributed systems and parallel data processing frameworks such as Spark (using PySpark or Scala), Dask, or similar technologies. Familiarity with Scala is a strong advantage, especially in performance-critical environments.
Proven track record in designing and scaling ML infrastructure
Deep understanding of ML workflows and lifecycle management
Experience in cloud environments (AWS, GCP, OCI) and containerized deployment (Kubernetes)
Understanding databases and SQL for data retrieval.
Strong communication skills and ability to drive initiatives independently
A passion for clean code, elegant architecture, and measurable impact
Monitoring and alerting tools (e.g. Grafana, Kibana)

Job Responsibility

Design and implement large-scale, distributed ML training pipelines
Build scalable infrastructure for data preprocessing, feature engineering, and model evaluation
Lead the technical design and development of new ML systems: from architecture to production
Collaborate cross-functionally with DS, infra teams, Product, BA and Engineering teams to define and deliver impactful solutions
Own the full lifecycle of ML infra: tooling, versioning, monitoring, automation, measuring results and quickly responding to critical issues.
Continuously research and adopt best-in-class practices in MLOps, performance tuning, and distributed systems

Select Country

Senior Big Data Pyspark Developer

Job Description

Job Responsibility

Requirements

Nice to have

Looking for more opportunities?