Senior Big Data Engineer Job at Citi (Pune)

Senior Principal Data Platform Software Engineer

We’re looking for a Sr Principal Data Platform Software Engineer (P70) to be a k...

Location

Salary:

239400.00 - 312550.00 USD / Year

Atlassian

Expiration Date

Until further notice

Requirements

15+ years in Data Engineering, Software Engineering, or related roles, with substantial exposure to big data ecosystems
Demonstrated experience building and operating data platforms or large‑scale data services in production
Proven track record of building services from the ground up (requirements → design → implementation → deployment → ongoing ownership)
Hands‑on experience with AWS, GCP (e.g., compute, storage, data, and streaming services) and cloud‑native architectures
Practical experience with big data technologies, such as Databricks, Apache Spark, AWS EMR, Apache Flink, or StarRocks
Strong programming skills in one or more of: Kotlin, Scala, Java, Python
Experience leading cross‑team technical initiatives and influencing senior stakeholders
Experience mentoring Staff/Principal engineers and lifting the technical bar for a team or org
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience

Job Responsibility

Design, develop and own delivery of high quality big data and analytical platform solutions aiming to solve Atlassian’s needs to support millions of users with optimal cost, minimal latency and maximum reliability
Improve and operate large‑scale distributed data systems in the cloud (primarily AWS, with increasing integration with GCP and Kubernetes‑based microservices)
Drive the evolution of our high-performance analytical databases and its integrations with products, cloud infrastructures (AWS and GCP) and isolated cloud environments
Help define and uplift engineering and operational standards for petabyte scale data platforms, with sub‑second analytic queries and multi‑region availability (coding guidelines, code review practices, observability, incident response, SLIs/SLOs)
Partner across multiple product and platform teams (including Analytics, Marketplace/Ecosystem, Core Data Platform, ML Platform, Search, and Oasis/FedRAMP) to deliver company‑wide initiatives that depend on reliable, high‑quality data
Act as a technical mentor and multiplier, raising the bar on design quality, code quality, and operational excellence across the broader team
Design and implement self‑healing, resilient data platforms with strong observability, fault tolerance, and recovery characteristics
Own the long‑term architecture and technical direction of Atlassian’s product data platform with projects that are directly tied to Atlassian’s company-level OKRs
Be accountable for the reliability, cost efficiency, and strategic direction of Atlassian’s product analytical data platform
Partner with executives and influence senior leaders to align engineering efforts with Atlassian’s long-term business objectives

What we offer

health and wellbeing resources
paid volunteer days

Fulltime

Senior Data Engineer

We are looking for a Data Engineer to join our team and support with designing, ...

Location

Salary:

Not provided

Foundever

Expiration Date

Until further notice

Requirements

Minimum of 7 years plus experience in data engineering
Track record of deploying and maintaining complex data systems at an enterprise level within regulated environments
Expertise in implementing robust data security measures, access controls, and monitoring systems
Proficiency in data modeling and database management
Strong programming skills in Python and SQL
Knowledge of big data technologies like Hadoop, Spark, and NoSQL databases
Deep experience with ETL processes and data pipeline development
Strong understanding of data warehousing concepts and best practices
Experience with cloud platforms such as AWS and Azure
Excellent problem-solving skills and attention to detail

Job Responsibility

Design and optimize complex data storage solutions, including data warehouses and data lakes
Develop, automate, and maintain data pipelines for efficient and scalable ETL processes
Ensure data quality and integrity through data validation, cleansing, and error handling
Collaborate with data analysts, machine learning engineers, and software engineers to deliver relevant datasets or data APIs for downstream applications
Implement data security measures and access controls to protect sensitive information
Monitor data infrastructure for performance and reliability, addressing issues promptly
Stay abreast of industry trends and emerging technologies in data engineering
Document data pipelines, processes, and best practices for knowledge sharing
Lead data governance and compliance efforts to meet regulatory requirements
Collaborate with cross-functional teams to drive data-driven decision-making within the organization

What we offer

Impactful work
Professional growth
Competitive compensation
Collaborative environment
Attractive salary and benefits package
Continuous learning and development opportunities
A supportive team culture with opportunities for occasional travel for training and industry events

Senior ML Data Engineer

As a Senior Data Engineer, you will play a pivotal role in our AI/ML workstream,...

Location

Poland , Warsaw

Salary:

Not provided

Awin Global

Expiration Date

Until further notice

Requirements

Bachelor or Master’s degree in data science, data engineering, Computer Science with focus on math and statistics / Master’s degree is preferred
At least 5 years experience as AI/ML data engineer undertaking above task and accountabilities
Strong foundation in computer science principes and statistical methods
Strong experience with cloud technology (AWS or Azure)
Strong experience with creation of data ingestion pipeline and ET process
Strong knowledge of big data tool such as Spark, Databricks and Python
Strong understanding of common machine learning techniques and frameworks (e.g. mlflow)
Strong knowledge of Natural language processing (NPL) concepts
Strong knowledge of scrum practices and agile mindset
Strong Analytical and Problem-Solving Skills with attention to data quality and accuracy

Job Responsibility

Design and maintain scalable data pipelines and storage systems for both agentic and traditional ML workloads
Productionise LLM- and agent-based workflows, ensuring reliability, observability, and performance
Build and maintain feature stores, vector/embedding stores, and core data assets for ML
Develop and manage end-to-end traditional ML pipelines: data prep, training, validation, deployment, and monitoring
Implement data quality checks, drift detection, and automated retraining processes
Optimise cost, latency, and performance across all AI/ML infrastructure
Collaborate with data scientists and engineers to deliver production-ready ML and AI systems
Ensure AI/ML systems meet governance, security, and compliance requirements
Mentor teams and drive innovation across both agentic and classical ML engineering practices
Participate in team meetings and contribute to project planning and strategy discussions

What we offer

Flexi-Week and Work-Life Balance: We prioritise your mental health and well-being, offering you a flexible four-day Flexi-Week at full pay and with no reduction to your annual holiday allowance. We also offer a variety of different paid special leaves as well as volunteer days
Remote Working Allowance: You will receive a monthly allowance to cover part of your running costs. In addition, we will support you in setting up your remote workspace appropriately
Pension: Awin offers access to an additional pension insurance to all employees in Germany
Flexi-Office: We offer an international culture and flexibility through our Flexi-Office and hybrid/remote work possibilities to work across Awin regions
Development: We’ve built our extensive training suite Awin Academy to cover a wide range of skills that nurture you professionally and personally, with trainings conveniently packaged together to support your overall development
Appreciation: Thank and reward colleagues by sending them a voucher through our peer-to-peer program

Senior Data Engineer

Adtalem is a data driven organization. The Data Engineering team builds data sol...

Location

United States , Lisle

Salary:

84835.61 - 149076.17 USD / Year

Adtalem Global Education

Expiration Date

Until further notice

Requirements

Bachelor's Degree Computer Science, Computer Engineering, Software Engineering, or other related technical field.
Master's Degree Computer Science, Computer Engineering, Software Engineering, or other related technical field.
Two (2) plus years experience in Google cloud with services like BigQuery, Composer, GCS, DataStream, Dataflows, BQML, Vertex AI.
Six (6) plus years experience in data engineering solutions such as data platforms, ingestion, data management, or publication/analytics.
Hands-on experience working with real-time, unstructured, and synthetic data.
Experience in Real Time Data ingestion using GCP PubSub, Kafka, Spark or similar.
Expert knowledge on Python programming and SQL.
Experience with cloud platforms (AWS, GCP, Azure) and their data services
Experience working with Airflow as workflow management tools and build operators to connect, extract and ingest data as needed.
Familiarity with synthetic data generation and unstructured data processing

Job Responsibility

Architect, develop, and optimize scalable data pipelines handling real-time, unstructured, and synthetic datasets
Collaborate with cross-functional teams, including data scientists, analysts, and product owners, to deliver innovative data solutions that drive business growth.
Design, develop, deploy and support high performance data pipelines both inbound and outbound.
Model data platform by applying the business logic and building objects in the semantic layer of the data platform.
Leverage streaming technologies and cloud platforms to enable real-time data processing and analytics
Optimize data pipelines for performance, scalability, and reliability.
Implement CI/CD pipelines to ensure continuous deployment and delivery of our data products.
Ensure quality of critical data elements, prepare data quality remediation plans and collaborate with business and system owners to fix the quality issues at its root.
Document the design and support strategy of the data pipelines
Capture, store and socialize data lineage and operational metadata

What we offer

Health, dental, vision, life and disability insurance
401k Retirement Program + 6% employer match
Participation in Adtalem’s Flexible Time Off (FTO) Policy
12 Paid Holidays
Eligible to participate in an annual incentive program

Fulltime

Senior Crypto Data Engineer

Token Metrics is seeking a multi-talented Senior Big Data Engineer to facilitate...

Location

Vietnam , Hanoi

Salary:

Not provided

Token Metrics

Expiration Date

Until further notice

Requirements

Bachelor's degree in Data Engineering, Big Data Analytics, Computer Engineering, or related field
A Master's degree in a relevant field is an added advantage
3+ years of Python, Java or any programming language development experience
3+ years of SQL & No-SQL experience (Snowflake Cloud DW & MongoDB experience is a plus)
3+ years of experience with schema design and dimensional data modeling
Expert proficiency in SQL, NoSQL, Python, C++, Java, R
Expert with building Data Lake, Data Warehouse or suitable equivalent
Expert in AWS Cloud
Excellent analytical and problem-solving skills
A knack for independence and group work

Job Responsibility

Liaising with coworkers and clients to elucidate the requirements for each task
Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed
Reformulating existing frameworks to optimize their functioning
Testing such structures to ensure that they are fit for use
Building a data pipeline from different data sources using different data types like API, CSV, JSON, etc
Preparing raw data for manipulation by Data Scientists
Implementing proper data validation and data reconciliation methodologies
Ensuring that your work remains backed up and readily accessible to relevant coworkers
Remaining up-to-date with industry standards and technological advancements that will improve the quality of your outputs

Fulltime

Senior Data Engineer

Senior Data Engineer – Dublin (Hybrid) Contract Role | 3 Days Onsite. We are see...

Location

Ireland , Dublin

Salary:

Not provided

Solas IT Recruitment

Expiration Date

Until further notice

Requirements

7+ years of experience as a Data Engineer working with distributed data systems
4+ years of deep Snowflake experience, including performance tuning, SQL optimization, and data modelling
Strong hands-on experience with the Hadoop ecosystem: HDFS, Hive, Impala, Spark (PySpark preferred)
Oozie, Airflow, or similar orchestration tools
Proven expertise with PySpark, Spark SQL, and large-scale data processing patterns
Experience with Databricks and Delta Lake (or equivalent big-data platforms)
Strong programming background in Python, Scala, or Java
Experience with cloud services (AWS preferred): S3, Glue, EMR, Redshift, Lambda, Athena, etc.

Job Responsibility

Build, enhance, and maintain large-scale ETL/ELT pipelines using Hadoop ecosystem tools including HDFS, Hive, Impala, and Oozie/Airflow
Develop distributed data processing solutions with PySpark, Spark SQL, Scala, or Python to support complex data transformations
Implement scalable and secure data ingestion frameworks to support both batch and streaming workloads
Work hands-on with Snowflake to design performant data models, optimize queries, and establish solid data governance practices
Collaborate on the migration and modernization of current big-data workloads to cloud-native platforms and Databricks
Tune Hadoop, Spark, and Snowflake systems for performance, storage efficiency, and reliability
Apply best practices in data modelling, partitioning strategies, and job orchestration for large datasets
Integrate metadata management, lineage tracking, and governance standards across the platform
Build automated validation frameworks to ensure accuracy, completeness, and reliability of data pipelines
Develop unit, integration, and end-to-end testing for ETL workflows using Python, Spark, and dbt testing where applicable