Senior Python Pyspark Engineer Job at Citi (Pune)

Senior Data Software Engineer (Python & PySpark) - Vice President

The Senior Data Software Engineer is a senior level position responsible for est...

Location

Singapore , Singapore

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
7+ years of experience in data engineering, with a strong focus on Python and big data technologies
Proven expertise in designing and implementing large-scale data processing solutions using PySpark
Extensive experience with distributed computing frameworks like Apache Spark
Strong understanding of data warehousing concepts, dimensional modeling, and ETL/ELT principles
Proficiency in SQL and experience with various relational and NoSQL databases
Experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, ADLS, Google Cloud Storage, Redshift, Snowflake, BigQuery, Databricks)
Familiarity with workflow orchestration tools (e.g., Apache Airflow, Azure Data Factory, AWS Step Functions)
Experience with version control systems (e.g., Git)
Excellent problem-solving, analytical, and communication skills.

Job Responsibility

Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Fulltime

Senior Python Engineer

Location

United States , Rutherford

Salary:

121200.00 - 181800.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

8+ years of overall experience in large-scale application development with recent mandatory platform for the secure and scalable deployment of AI agents into application contexts
Minimum of 5+ years of proven experience in a Python and pyspark Engineering lead role focused on building enterprise-grade, high-volume ELT/ETL processes using the PySpark and Databricks ecosystem
Hands-on experience with agentic AI development using YAML, JSON, FAST API or Spring boot, Google ADK, LLM itegrations, including Devin.AI or Github Copilot, and integrating models via platforms like MCP using advanced prompt engineering
Proven experience developing and automating microservice integrations to support data-intensive applications
Proficiency in at least one programming language commonly used for data analytics, engineering, such as Python or Scala
Strong SQL skills and experience with various relational databases
Deep understanding of data modeling, data warehousing concepts, Data Mesh architecture, and data federation
Excellent communication, collaboration, and problem-solving skills

Job Responsibility

Design, develop, and maintain scalable, enterprise-grade AI agents, supporting ELT/ETL processes to handle large data volumes using the Python, FAST API, Microservices, PySpark, Kafka and Databricks ecosystem
Build and Deploy GEN AI Agents using Googles ADK and Google Flash 2.5+ LLMs to support application automation supports and its deep insights, workflow support with HIL - Human in loop architecture
Build and maintain data federation layers for lambda and Data Mesh architectures using tools like Starburst, with a strategy for adopting AI-based use cases (e.g., machine learning, deep learning, NLP) to drive efficiency
Develop, deploy, and automate microservice integrations to support data-intensive applications, ensuring scalability, resilience, and maintainability using cloud native infrastructure and openshift or Kubernates architecture including CI/CD pipelines
Integrate and leverage agentic AI tools (e.g., Devin.AI, Github Copilot) and platforms (e.g., MCP) through advanced prompt engineering to enhance development and operational efficiency
Ensure data quality, integrity, and security throughout the entire data lifecycle
Contribute to the continuous improvement of data engineering processes, standards, and best practices within the team
Appropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citi, its clients, and assets by driving compliance with applicable laws, rules, and regulations
Adhere to Policy, apply sound ethical judgment, and escalate, manage, and report control issues with transparency

What we offer

Medical, dental & vision coverage
401(k)
Life, accident, and disability insurance
Wellness programs
Paid time off packages including planned time off (vacation), unplanned time off (sick leave), and paid holidays

Fulltime

Senior Python Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

8+ Years of Experience in Python Development
Experience is working with High Volume Data Pipeline development
Experience working with PyIceberg, PySpark and Polars data frames
Fully Hands on Development
Needs to be able to participatein Design and Architectire discussions
Should be able to work with distributed geographical teams
Bachelor's degree/University degree or equivalent experience

Job Responsibility

Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
Ensure essential procedures are followed and help define operating standards and processes
Serve as advisor or coach to new or lower level analysts
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Senior Pyspark Data Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

Canada , Mississauga

Salary:

94300.00 - 141500.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

5-8 years of relevant experience
Experience in systems analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
Proficiency in Python programming
Strong expertise in data processing frameworks such as Apache Spark, Hadoop
Expertise in Data Lakehouse technologies (Apache Iceberg, Trino, Deltalake)
Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)

Job Responsibility

Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
Ensure essential procedures are followed and help define operating standards and processes
Serve as advisor or coach to new or lower level analysts
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Senior Pyspark Data Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

Canada , Mississauga

Salary:

94300.00 - 141500.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

5-8 years of relevant experience
Experience in systems analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
Proficiency in Python programming
Strong expertise in data processing frameworks such as Apache Spark, Hadoop
Expertise in Data Lakehouse technologies (Apache Iceberg, Trino, Deltalake)
Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)

Job Responsibility

Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
Ensure essential procedures are followed and help define operating standards and processes
Serve as advisor or coach to new or lower level analysts
Has the ability to operate with a limited level of direct supervision
Can exercise independence of judgement and autonomy
Acts as SME to senior stakeholders and /or other team members

Fulltime

Senior PySpark Data Engineer

We are seeking a highly skilled and experienced Senior PySpark Data Engineer to ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

6+ years of professional relevant experience in a data engineering role
Extensive hands-on experience with PySpark and advanced Python programming skills
Proven experience with Big Data ecosystems, including Cloudera and/or DataBricks
Hands-on experience with distributed query engines like Starburst (Trino/Presto)
Proficient in designing and managing complex workflows using scheduling tools, particularly Apache Airflow
Strong expertise in SQL and experience with relational and non-relational databases
Solid understanding of data warehousing concepts, ETL/ELT processes, and data modeling techniques
Experience working in a Linux/Unix environment
GIT HUB, CI/CD Pipeline
Bachelor’s degree/University degree or equivalent experience

Job Responsibility

Design, develop, and maintain robust, scalable, and high-performance data pipelines using PySpark
Develop, schedule, and monitor complex data workflows using orchestration tools like Apache Airflow
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality data solutions
Optimize and tune Spark jobs for performance and efficiency
Implement data quality checks and ensure data integrity across all data pipelines
Design and implement data models for optimal storage and retrieval
Mentor junior data engineers and promote best practices in data engineering
Ensure compliance with data governance and security policies
Troubleshoot and resolve data-related issues in a timely manner

Fulltime

Senior Data Engineer with Python

Intellectsoft is a software development company delivering innovative solutions ...

Location

India , Bengaluru; Gurugram

Salary:

Not provided

Intellectsoft

Expiration Date

Until further notice

Requirements

4+ years of professional experience, including 2+ years of data engineering with Apache Spark and SQL
Proficiency in Python for data processing and automation
Knowledge of PySpark, distributed computing, analytical databases and other big data technologies
Expertise in designing and managing ETL pipelines and distributed data processing frameworks
Strong knowledge of database systems, data modeling, and analytical databases
Hands-on experience with workflow orchestration tools such as Apache Airflow
Familiarity with cloud platforms like AWS, GCP, or Azure
Solid understanding of software development lifecycles, including coding standards, version control, and testing

Job Responsibility

Design and build highly reliable and scalable data pipelines using PySpark and big data technologies
Collaborate with the data science team to develop new features that enhance model accuracy and performance
Create standardized data models to improve consistency across various deployments
Troubleshoot and resolve issues in existing ETL pipelines and optimize workflows
Conduct POCs to evaluate new technologies and integrate additional data sources
Follow and promote best practices for software development, ensuring high-quality solutions that meet requirements and deadlines
Document development updates and maintain clear technical documentation

What we offer

Awesome projects with an impact
Udemy courses of your choice
Team-buildings, events, marathons & charity activities to connect and recharge
Workshops, trainings, expert knowledge-sharing that keep you growing
Clear career path
Absence days for work-life balance
Flexible hours & work setup - work from anywhere and organize your day your way

Fulltime

Senior Applications Developer : Ab Initio, Bigdata - Python, Pyspark

We are seeking a highly skilled and experienced Senior Developer (C12) to join o...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

8 + years of experience in Applications Development, Systems Analysis, or equivalent senior engineering roles
Extensive hands‑on experience delivering enterprise‑scale, database‑driven platforms in a regulated environment
Expert-level proficiency in Python programming, including object-oriented design, data structures, algorithms, and extensive experience with various Python libraries (e.g., Pandas, NumPy, Flask/Django for web development, asyncio for async programming)
Deep expertise in developing, optimizing, and deploying PySpark applications for large-scale data processing, ETL, and real-time analytics on distributed systems (e.g., Spark SQL, Spark Streaming, DataFrames)
Strong understanding of Apache Spark architecture, Hadoop ecosystem, and experience with distributed computing concepts. Familiarity with big data storage formats (e.g., Parquet, ORC)
Hands-on experience with at least one major cloud provider (AWS, Azure, or GCP), specifically with their compute, storage, and data services (e.g., S3, ADLS, EMR, Databricks, Azure Synapse)
Solid experience with both relational databases (e.g., PostgreSQL, Oracle, SQL Server, MySQL) and NoSQL databases (e.g., MongoDB). Strong SQL writing and optimization skills
Experience in designing, developing, and consuming RESTful APIs using Python frameworks (e.g., Flask, FastAPI, Django REST Framework)
Strong understanding and practical experience with CI/CD tools (e.g., Jenkins, GitLab CI, Azure DevOps) and containerization technologies (Docker, Kubernetes)
Expert-level proficiency with Git

Job Responsibility

Design, develop, and implement robust, scalable, and high-performance data pipelines and applications using Abinitio, Python, PySpark, and Big Data technologies
Work autonomously to analyze requirements, propose technical solutions, and deliver high-quality code and data products, ensuring alignment with architectural standards and business objectives
Utilize expertise in various Big Data platforms (e.g., Hadoop, Hive, Kafka, Spark) to process, transform, and manage large datasets efficiently
Write complex SQL queries, stored procedures, and optimize database performance for large-scale data warehousing and analytics solutions
Develop and enhance ETL (Extract, Transform, Load) processes, ensuring data quality, integrity, and timely delivery
Proactively research, evaluate, and integrate new and emerging technologies, frameworks, and tools to improve development processes and solution capabilities
Ensure adherence to coding standards, conduct thorough code reviews, and implement best practices for software development, data governance, and security
Diagnose and resolve complex technical issues related to data pipelines, performance bottlenecks, and system integrations in a fast-paced environment
Collaborate effectively with cross-functional teams including architects, data scientists, business analysts, and QA engineers. Provide technical guidance and mentorship to junior team members
Leverage KYC domain knowledge to develop data solutions that support client due diligence (CDD), and other financial crime compliance initiatives, understanding the nuances of regulatory data requirements

Fulltime

Select Country

Senior Python Pyspark Engineer

Job Description

Job Responsibility

Requirements

Looking for more opportunities?