CrawlJobs Logo

Senior Big Data Engineer

India, Pune · Job Posted April 23, 2026
Apply Position
Job Link Share

Job Description

Senior Big Data Engineer - Assistant Vice President is accountable for developing high quality data products to support the Bank’s regulatory requirements and data driven decision making. A Data Engineer will serve as an example to other team members, work closely with customers, and remove or escalate roadblocks. By applying their knowledge of data architecture standards, data warehousing, data structures, and business intelligence they will contribute to business outcomes on an agile team.

Job Responsibility

  • Developing and supporting scalable, extensible, and highly available data solutions
  • Deliver on critical business priorities while ensuring alignment with the wider architectural vision
  • Identify and help address potential risks in the data supply chain
  • Follow and contribute to technical standards
  • Design and develop analytical data models

Requirements

  • First Class Degree in Engineering/Technology (4-year graduate course)
  • 8 to 11 years’ experience implementing data-intensive solutions using agile methodologies
  • Experience of relational databases and using SQL for data querying, transformation and manipulation
  • Experience of modelling data for analytical consumers
  • Ability to automate and streamline the build, test and deployment of data pipelines
  • Experience in cloud native technologies and patterns
  • A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
  • Excellent communication and problem-solving skills
  • An inclination to mentor
  • an ability to lead and deliver medium sized components independently
  • ETL: Hands on experience of building data pipelines. Proficiency in two or more data integration platforms such as Apache Spark, Talend and Informatica
  • Cloud: Hands on experience on Cloud preferably AWS
  • Big Data: Experience of ‘big data’ platforms such as Hadoop, Hive or Snowflake for data storage and processing
  • Data Warehousing & Database Management: Expertise around Data Warehousing concepts, Relational (Oracle, MSSQL, MySQL) and NoSQL (MongoDB, DynamoDB) database design
  • Data Modeling & Design: Good exposure to data modeling techniques
  • design, optimization and maintenance of data models and data structures
  • Languages: Proficient in one or more programming languages commonly used in data engineering such as Python, Java or Scala
  • DevOps: Exposure to concepts and enablers - CI/CD platforms, version control, automated quality control management
  • Data Governance: A strong grasp of principles and practice including data quality, security, privacy and compliance

Nice to have

  • Ab Initio: Experience developing Co>Op graphs
  • ability to tune for performance. Demonstrable knowledge across full suite of Ab Initio toolsets e.g., GDE, Express>IT, Data Profiler and Conduct>IT, Control>Center, Continuous>Flows
  • Cloud: Good exposure to public cloud data platforms such as AWS, S3, Snowflake, Redshift, Databricks, BigQuery, etc. Demonstratable understanding of underlying architectures and trade-offs
  • Data Quality & Controls: Exposure to data validation, cleansing, enrichment and data controls
  • Containerization: Fair understanding of containerization platforms like Docker, Kubernetes
  • File Formats: Exposure in working on Event/File/Table Formats such as Avro, Parquet, Protobuf, Iceberg, Delta
  • Others: Experience of using a Job scheduler e.g., Autosys. Exposure to Business Intelligence tools e.g., Tableau, Power BI
  • Certification on any one or more of the above topics would be an advantage

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Big Data Engineer

8 matching positions

Senior Big Data Engineer

Start.io is a mobile marketing and audience platform. Start.io empowers the mobi...
Location
Location
Poland , Warsaw
Salary
Salary:
Not provided
start.io Logo
Start.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • B.Sc. or M.Sc. in Computer Science, Software Engineering, or other equivalent fields.
  • 5+ years of hands-on experience in backend or ML engineering
  • Strong Python skills and experience working with distributed systems and parallel data processing frameworks such as Spark (using PySpark or Scala), Dask, or similar technologies. Familiarity with Scala is a strong advantage, especially in performance-critical environments.
  • Proven track record in designing and scaling ML infrastructure
  • Deep understanding of ML workflows and lifecycle management
  • Experience in cloud environments (AWS, GCP, OCI) and containerized deployment (Kubernetes)
  • Understanding databases and SQL for data retrieval.
  • Strong communication skills and ability to drive initiatives independently
  • A passion for clean code, elegant architecture, and measurable impact
  • Monitoring and alerting tools (e.g. Grafana, Kibana)
Job Responsibility
Job Responsibility
  • Design and implement large-scale, distributed ML training pipelines
  • Build scalable infrastructure for data preprocessing, feature engineering, and model evaluation
  • Lead the technical design and development of new ML systems: from architecture to production
  • Collaborate cross-functionally with DS, infra teams, Product, BA and Engineering teams to define and deliver impactful solutions
  • Own the full lifecycle of ML infra: tooling, versioning, monitoring, automation, measuring results and quickly responding to critical issues.
  • Continuously research and adopt best-in-class practices in MLOps, performance tuning, and distributed systems
Read More
Arrow Right

Senior Big Data Engineer

The Big Data Engineer is a senior level position responsible for establishing an...
Location
Location
Canada , Mississauga
Salary
Salary:
94300.00 - 141500.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ Years of Experience in Big Data Engineering (PySpark)
  • Data Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from multiple sources
  • Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
  • Proficiency in programming languages like Python, or Scala
  • Strong expertise in data processing frameworks such as Apache Spark, Hadoop
  • Expertise in Data Lakehouse technologies (Apache Iceberg, Apache Hudi, Trino)
  • Experience with cloud data platforms like AWS (Glue, EMR, Redshift), Azure (Synapse), or GCP (BigQuery)
  • Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)
  • Experience with data orchestration tools like Apache Airflow or Prefect
  • Familiarity with containerization (Docker, Kubernetes) is a plus
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients and assets
  • Fulltime
Read More
Arrow Right

Senior Big Data Engineer

The Big Data Engineer is a senior level position responsible for establishing an...
Location
Location
Canada , Mississauga
Salary
Salary:
94300.00 - 141500.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ Years of Experience in Big Data Engineering (PySpark)
  • Data Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from multiple sources
  • Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
  • Proficiency in programming languages like Python, or Scala
  • Strong expertise in data processing frameworks such as Apache Spark, Hadoop
  • Expertise in Data Lakehouse technologies (Apache Iceberg, Apache Hudi, Trino)
  • Experience with cloud data platforms like AWS (Glue, EMR, Redshift), Azure (Synapse), or GCP (BigQuery)
  • Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)
  • Experience with data orchestration tools like Apache Airflow or Prefect
  • Familiarity with containerization (Docker, Kubernetes) is a plus
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
What we offer
What we offer
  • Well-being support
  • Growth opportunities
  • Work-life balance support
  • Fulltime
Read More
Arrow Right

Senior Big Data Engineer

Location
Location
United States , Flowood
Salary
Salary:
Not provided
phasorsoft.com Logo
PhasorSoft Group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Python programming for data manipulation and analysis
  • Experience with PySpark for processing large-scale data
  • Strong understanding and practical experience with big data technologies such as Hadoop, Spark, Kafka, etc.
  • Knowledge of designing and implementing ETL processes for data integration
  • Ability to work with large datasets, perform data cleansing, transformations, and aggregations
  • Familiarity with machine learning concepts and experience implementing ML models
  • Understanding of data governance principles and experience implementing data security measures
  • Ability to create clear and concise documentation for data pipelines and processes
  • Strong teamwork and collaboration skills to work with cross-functional teams
  • Analytical and problem-solving skills to optimize data workflows and processes
Job Responsibility
Job Responsibility
  • Design and develop scalable data pipelines and solutions using Python and PySpark
  • Utilize big data technologies such as Hadoop, Spark, Kafka, or similar tools for processing and analyzing large datasets
  • Develop and maintain ETL processes to extract, transform, and load data into data lakes or warehouses
  • Collaborate with data engineers and scientists to implement machine learning models and algorithms
  • Optimize and tune data processing workflows for performance and efficiency
  • Implement data governance and security measures to ensure data integrity and privacy
  • Create and maintain documentation for data pipelines, workflows, and processes
  • Provide technical leadership and mentorship to junior team members
  • Fulltime
Read More
Arrow Right

Senior Data Engineer, Big Data

This role is essential for designing and developing data architectures across on...
Location
Location
United States , New York; Philadelphia
Salary
Salary:
105100.00 - 189600.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree plus 5 years of related work experience OR Advanced degree with 3 years of related experience
  • 4-7 years Developing cloud solutions using data series
  • experience with cloud platforms (Amazon Web Services, Azure, or Google Cloud)
  • 4-7 years Hands-on development using and migrating data to cloud platforms
  • 4-7 years Experience in SQL, NoSQL, and/or relational database design and development
  • 4-7 years Advanced knowledge and experience in building complex data pipelines with Python, Experience in languages such as SQL, DAX Python, Java, Scala, and/or Go
  • At least 18 years of age
  • Legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Develop data engineering solutions that enable data pipelines, visualization, and analytical tools to support business requirements
  • Design and develop data architectures across on-premise, cloud, and hybrid platforms to ensure scalable data infrastructure
  • Perform data wrangling, exploration, and discovery of heterogeneous data to generate new business insights
  • Contribute to team knowledge sharing and drive the advancement of new data engineering capabilities
  • Mentor team members to build and enhance their data engineering skillsets and professional growth
  • Assist management in project definition, including estimating, planning, and scoping work to meet objectives
  • Also responsible for other duties/projects as assigned by business management as needed
What we offer
What we offer
  • Medical, dental and vision insurance
  • Flexible spending account
  • 401(k)
  • Employee stock grants
  • Employee stock purchase plan
  • Paid time off
  • Up to 12 paid holidays
  • Paid parental and family leave
  • Family building benefits
  • Back-up care
  • Fulltime
Read More
Arrow Right

Senior Data Engineer, Big Data

This role is essential for designing and developing data architectures across on...
Location
Location
United States , New York; Philadelphia
Salary
Salary:
105100.00 - 189600.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree plus 5 years of related work experience OR Advanced degree with 3 years of related experience
  • 4-7 years Developing cloud solutions using data series
  • experience with cloud platforms (Amazon Web Services, Azure, or Google Cloud)
  • 4-7 years Hands-on development using and migrating data to cloud platforms
  • 4-7 years Experience in SQL, NoSQL, and/or relational database design and development
  • 4-7 years Advanced knowledge and experience in building complex data pipelines with Python, Experience in languages such as SQL, DAX Python, Java, Scala, and/or Go
  • At least 18 years of age
  • Legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Develop data engineering solutions that enable data pipelines, visualization, and analytical tools to support business requirements
  • Design and develop data architectures across on-premise, cloud, and hybrid platforms to ensure scalable data infrastructure
  • Perform data wrangling, exploration, and discovery of heterogeneous data to generate new business insights
  • Contribute to team knowledge sharing and drive the advancement of new data engineering capabilities
  • Mentor team members to build and enhance their data engineering skillsets and professional growth
  • Assist management in project definition, including estimating, planning, and scoping work to meet objectives
  • Also responsible for other duties/projects as assigned by business management as needed
What we offer
What we offer
  • annual stock grant
  • employee stock purchase plan
  • 401(k)
  • free, year-round money coaches
  • medical insurance
  • dental insurance
  • vision insurance
  • flexible spending account
  • paid time off
  • up to 12 paid holidays
  • Fulltime
Read More
Arrow Right

Senior Solutions Engineer – Big Data & Data Infrastructure

This is a great opportunity to be part of one of the fastest-growing infrastruct...
Location
Location
Israel , Tel Aviv
Salary
Salary:
Not provided
vastdata.com Logo
VAST Data
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2–4 years in software / solution or infrastructure engineering
  • 2–4 years focused on building / maintaining large-scale data pipelines / storage & database solutions
  • Proficiency in Trino, Spark (Structured Streaming & batch) and solid working knowledge of Apache Kafka
  • Coding background in Python (must-have)
  • Deep understanding of data storage architectures including SQL, NoSQL, and HDFS
  • Solid grasp of DevOps practices, including containerization (Docker), orchestration (Kubernetes), and infrastructure provisioning (Terraform)
  • Experience with distributed systems, stream processing, and event-driven architecture
  • Hands-on familiarity with benchmarking and performance profiling for storage systems, databases, and analytics engines
  • Excellent communication skills
Job Responsibility
Job Responsibility
  • Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes
  • Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools
  • Implement event-driven and serverless workflows
  • Create technical guides, architecture docs, and demo pipelines
  • Integrate data validation, observability tools, and governance directly into the pipeline lifecycle
  • Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark)
  • Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets
  • Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads
  • Operate and debug object store–backed data lake infrastructure
Read More
Arrow Right

Senior Big Data Engineer - Assistant Vice President

The Senior Data Engineer (C12 – AVP) is a senior-level position responsible for ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 9–12 years of relevant experience in data analysis and data engineering, preferably within the Financial Services or Banking industry
  • Proven interpersonal, diplomatic, management, and prioritization skills
  • Consistently demonstrates clear and concise written and verbal communication
  • Proven ability to manage multiple activities, build strong working relationships, and work effectively under pressure
  • Demonstrated strong problem-solving, analytical, and decision-making skills with a methodical attention to detail
  • Proven self-motivation to take initiative and master new tasks and technologies quickly
  • Education: Bachelor's degree / University degree in a technical or business discipline (Computer Science, Information Systems, Engineering, Finance, or equivalent experience)
  • Functional Skillset: Data Analysis: Extensive experience in analyzing and interpreting complex data from disparate sources to provide actionable insights
  • Financial/Banking Domain Expertise: Strong understanding of financial products, banking processes, and industry standards
  • Data Requirements Definition: Proven ability to analyze different data sources and datasets to create comprehensive data mapping documents and define data ingestion requirements
Job Responsibility
Job Responsibility
  • Consult with users and clients to solve complex data-related issues through in-depth evaluation of business processes, data sources, and industry standards
  • Analyze large and diverse datasets from various sources to identify trends, patterns, and anomalies, providing critical input for business and technology initiatives
  • Develop and document data mapping specifications, transformation logic, and ingestion requirements for new data pipelines and systems
  • Consult with business clients to determine functional specifications for data-centric systems and provide ongoing operational support
  • Design and implement scalable data pipelines and batch/streaming workflows using Apache Spark, Spark Streaming, Hive, and Hadoop within enterprise big data ecosystems
  • Develop and maintain backend services and automation scripts using Java, Spring Boot, JPA, and Shell Scripting to support data processing and operational workflows
  • Build and manage event-driven data architectures leveraging Apache Kafka for real-time data ingestion and streaming use cases
  • Automate job scheduling and dependency management using Autosys
  • manage and optimize Oracle database objects and queries to support analytical workloads
  • Develop supporting interfaces and data visualization components using JavaScript to enhance data accessibility and reporting capabilities
  • Fulltime
Read More
Arrow Right