CrawlJobs Logo

Senior Data Engineer - Modern Engineering Team

India, Pune · Job Posted May 16, 2026
Apply Position
Job Link Share

Job Description

The Senior Data Engineer is an experienced, hands-on engineer who is passionate about solving complex technical problems through innovation and modern engineering practices. This role is central to building and evolving our next-generation, event-driven data platform, working at the intersection of data engineering, software craftsmanship, and agile delivery.

Job Responsibility

  • Design, develop, and maintain high-quality, production-grade data pipelines built on Java, Spring Boot microservices, Kafka, Elasticsearch, and Oracle
  • Write clean, well-tested, and maintainable code, applying SOLID principles, TDD, BDD (Cucumber/Spock), and Trunk-Based Development practices
  • Actively participate in code reviews, pairing sessions with developers and AI coding agents (e.g., GitHub Copilot, Devin), and contribute to raising the overall engineering bar
  • Decompose acceptance criteria to understand backlog deliverables, complexities, and risks, contributing to sprint planning and agile ceremonies
  • Implement and manage feature toggles to enable safe, continuous delivery to production
  • Contribute to the design and documentation of scalable, highly available, and low-latency event-driven architectures
  • Apply modern software design patterns and contribute to architectural decisions, including CQRS, Data Mesh, and Medallion Architecture principles
  • Leverage knowledge of Cloud-native development, Container Orchestration (OpenShift/Kubernetes), and CI/CD pipelines (Tekton, Jenkins, Harness) to deliver robust solutions
  • Partner continuously with business stakeholders, architects, and cross-functional teams to deliver sprint goals and broader platform objectives
  • Serve as a Subject Matter Expert (SME) for senior stakeholders and team members on data engineering topics
  • Mentor and coach junior engineers, fostering a culture of continuous learning and engineering excellence
  • Contribute to defining and evolving technical standards, coding guidelines, and team ways of working

Requirements

  • 8+ years of experience as a Senior Technology/Data Engineer
  • Strong, hands-on experience with event-driven architecture using Apache Kafka and Spring Boot microservices
  • Proficiency in Java (JDK 17/21+), Gradle, SQL/Oracle and Elasticsearch
  • Practical experience with BDD/TDD using Cucumber/Spock, Trunk-Based Development, and feature toggles
  • Strong understanding of Agile/Scrum development and a craftsperson approach to engineering, applying SOLID and Clean Code principles
  • Clear understanding of Data Structures and Object-Oriented Principles in Java
  • Experience with CI/CD pipelines (Tekton, Jenkins, Harness, etc.), either on-premises or in the cloud
  • Strong experience with Cloud-native development and Container Orchestration tools (Docker, Kubernetes, OpenShift)
  • B.Tech/B.Eng degree or equivalent work experience

Nice to have

  • Exposure to horizontally scalable, highly available, and low-latency event-driven application architecture
  • Experience with Security, Observability, and Monitoring tools (Grafana, Prometheus, Splunk, ELK, CloudWatch, etc.)
  • Good understanding of Data Engineering concepts, including data storage, data quality, data governance, data modeling, and data management
  • Familiarity with AI coding agents (e.g., GitHub Copilot, Devin or equivalent tools) and their integration into the development workflow
  • Exposure to Data Mesh, Modern Data Architecture, or CQRS patterns

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Data Engineer - Modern Engineering Team

8 matching positions

Senior Data Engineer – Data Engineering & AI Platforms

We are looking for a highly skilled Senior Data Engineer (L2) who can design, bu...
Location
Location
India , Chennai, Madurai, Coimbatore
Salary
Salary:
Not provided
optisolbusiness.com Logo
OptiSol Business Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on expertise in cloud ecosystems (Azure / AWS / GCP)
  • Excellent Python programming skills with data engineering libraries and frameworks
  • Advanced SQL capabilities including window functions, CTEs, and performance tuning
  • Solid understanding of distributed processing using Spark/PySpark
  • Experience designing and implementing scalable ETL/ELT workflows
  • Good understanding of data modeling concepts (dimensional, star, snowflake)
  • Familiarity with GenAI/LLM-based integration for data workflows
  • Experience working with Git, CI/CD, and Agile delivery frameworks
  • Strong communication skills for interacting with clients, stakeholders, and internal teams
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ETL/ELT pipelines across cloud and big data platforms
  • Contribute to architectural discussions by translating business needs into data solutions spanning ingestion, transformation, and consumption layers
  • Work closely with solutioning and pre-sales teams for technical evaluations and client-facing discussions
  • Lead squads of L0/L1 engineers—ensuring delivery quality, mentoring, and guiding career growth
  • Develop cloud-native data engineering solutions using Python, SQL, PySpark, and modern data frameworks
  • Ensure data reliability, performance, and maintainability across the pipeline lifecycle—from development to deployment
  • Support long-term ODC/T&M projects by demonstrating expertise during technical discussions and interviews
  • Integrate emerging GenAI tools where applicable to enhance data enrichment, automation, and transformations
What we offer
What we offer
  • Opportunity to work at the intersection of Data Engineering, Cloud, and Generative AI
  • Hands-on exposure to modern data stacks and emerging AI technologies
  • Collaboration with experts across Data, AI/ML, and cloud practices
  • Access to structured learning, certifications, and leadership mentoring
  • Competitive compensation with fast-track career growth and visibility
  • Fulltime
Read More
Arrow Right

Senior Data Architect/Data Engineer, Aladdin Engineering - Vice President

About this Role: At BlackRock, technology is the foundation of our business. As ...
Location
Location
United States , New York
Salary
Salary:
162000.00 - 215000.00 USD / Year
blackrock.com Logo
BlackRock Investments
Expiration Date
July 11, 2026
Flip Icon
Requirements
Requirements
  • B.S./M.S. in Computer Science, Engineering, or related discipline (or equivalent practical experience)
  • 8+ years of experience building production data systems, with demonstrated ownership of data modeling and data pipeline engineering
  • Strong SQL skills (advanced querying, query plans, performance tuning) with hands-on experience in Snowflake and/or Microsoft SQL Server
  • Proven experience with data modeling for analytics (dimensional modeling / star schemas, conformed dimensions, slowly changing dimensions) and translating business concepts into robust schemas
  • Hands-on experience designing and implementing ELT/ETL pipelines, including batch and near-real-time patterns
  • Proficiency in at least one general-purpose language used for data engineering (e.g., Python, Java, or Scala) for automation, orchestration, and integrations
  • Working knowledge of modern data engineering practices: testing for transformations, CI/CD, environment promotion, and operational monitoring
  • Strong communication skills and comfort collaborating with domain experts to turn ambiguity into clear, implementable data products
Job Responsibility
Job Responsibility
  • Partner with domain experts, product, and engineering teams to design canonical data models (conceptual → logical → physical) that power trusted reporting, analytics, and downstream integrations
  • Build and evolve analytics-ready datasets in Snowflake (curated layers / data marts), including clear metric definitions (grain, dimensions, measures) that enable consistent enterprise reporting
  • Design and develop reliable ELT/ETL pipelines across Snowflake and SQL Server to support both scheduled batch loads and low-latency ingestion where needed
  • Implement robust pipeline patterns such as incremental processing, idempotency (replay-safe loads), deduplication, and backfill/reprocessing strategies
  • Establish and enforce data quality and observability practices (freshness, completeness, accuracy checks
  • alerting
  • runbooks
  • SLAs) to keep data products production-grade
  • Optimize analytical performance and cost by applying Snowflake best practices (clustering/partition strategies, materializations, query optimization) and SQL Server performance tuning where appropriate
  • Publish curated data to downstream systems and serving layers when needed (e.g., search indices like Elasticsearch and operational stores like Cosmos DB) with clear contracts and monitoring
What we offer
What we offer
  • Strong retirement plan
  • Tuition reimbursement
  • Comprehensive healthcare
  • Support for working parents
  • Flexible Time Off (FTO)
  • Fulltime
Read More
Arrow Right

Senior Data Engineer Lead / Architect - Senior Vice President

At Citi Services - Global Trade Technology Organization, we are on a mission to ...
Location
Location
India , Pune, Maharashtra, India, Chennai, Tamil Nadu, India
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of professional experience in data engineering, with a proven track record of designing and building large-scale data systems
  • 3+ years in a technical leadership or architect role, with experience mentoring junior and senior engineers
  • Expert-level proficiency in at least one programming language (Python or Scala preferred) and exceptional SQL skills
  • Proven hands-on experience with Python or Scala for data manipulation, scripting, machine learning, and backend development
  • Deep, hands-on experience with a major cloud platform (AWS, GCP, or Azure) and its data ecosystem (e.g., S3/GCS, Redshift/BigQuery, EMR/Dataproc, Kinesis/Dataflow)
  • Extensive hands-on experience with modern big data technologies and Data streaming (like Hadoop, Hive, Impala, Apache Spark, Kafka, or Flink)
  • Proficiency with workflow orchestration tools such as Airflow, Dagster, or Prefect
  • Proficiency in designing and implementing microservices architectures, RESTful APIs, and event-driven systems with 'Data as a Product' Principle
  • Solid understanding of data modeling concepts and database design for both analytical (OLAP) and transactional (OLTP) workloads
  • Deep understanding and hands-on experience with relational databases (e.g., PostgreSQL, Oracle), NoSQL databases (e.g., MongoDB, Cassandra), data warehousing, and big data technologies (e.g., Spark, Kafka)
Job Responsibility
Job Responsibility
  • Architect & Design: Design, architect, and oversee the development of robust, scalable, and reliable data infrastructure, including data lakes, data warehouses, and real-time streaming platforms on the cloud
  • Build & Code: Act as a senior individual contributor and hands-on technical leader. Write clean, maintainable, and high-performance code for data ingestion, transformation, and serving layers (e.g., using Python, Scala, SQL, and Spark)
  • Lead & Mentor: Lead a team of data engineers, providing technical guidance, mentorship, and career development support. Foster a collaborative and inclusive team environment
  • Champion Culture: Define, document, and champion data engineering best practices across the organization, including CI/CD, data quality, testing frameworks, observability, and code review standards
  • Drive Strategy: Partner with leadership, product managers, data scientists, and analysts to understand data needs and develop a long-term data strategy and roadmap
  • Innovate & Evaluate: Stay at the forefront of data engineering technologies. Evaluate, prototype, and recommend new tools and frameworks to continuously improve our data platform
  • Ensure Governance: Implement and enforce robust data governance, security, and privacy policies in partnership with our security and compliance teams
  • Fulltime
Read More
Arrow Right

Senior Data Engineering Lead-Senior Vice President

The Applications Development Group Manager is a senior management level position...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 13+ years of relevant experience
  • Extensive experience in applications development
  • Experience in Wealth Management / Private Bank data space
  • Familiarity with key investment flows like Post Trade Flows, Portfolio and Risk Modelling, Performance, Market and Reference data including Client and Advisor and IC interactions
  • Sound knowledge of core investment products like Cash Equity, Fixed Income, Alternatives, Structured Notes and Derivatives
  • Experience with data technologies such as RDBMS and Hadoop (Cloudera) database
  • Experience in building cloud native data platforms (AWS & GCP) and exposure to modern data architecture
  • Experience in building real-time and batch data pipelines to support wide range of Analytics, Regulatory, Business and Operational use cases
  • Familiar with the big data technologies such as Hadoop, HDFS, Hive, HBase, Spark and Flink
  • Exposure with Snowflake and Databricks is preferred
Job Responsibility
Job Responsibility
  • Manage multiple teams of professionals to accomplish established goals and conduct personnel duties for team (e.g. performance evaluations, hiring and disciplinary actions)
  • Provide strategic influence and exercise control over resources, budget management and planning while monitoring end results
  • Utilize in-depth knowledge of concepts and procedures within own area and basic knowledge of other areas to resolve issues
  • Ensure essential procedures are followed and contribute to defining standards
  • Integrate in-depth knowledge of applications development with overall technology function to achieve established goals
  • Provide evaluative judgement based on analysis of facts in complicated, unique, and dynamic situations including drawing from internal and external sources
  • Influence and negotiate with senior leaders across functions, as well as communicate with external parties as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency, as well as effectively supervise the activity of others and create accountability with those who fail to maintain these standards
  • Fulltime
Read More
Arrow Right

Senior Staff Data Engineer- Data Platform

At Marktplaats, data is at the heart of everything we do, but Intelligence is wh...
Location
Location
Netherlands , Amsterdam
Salary
Salary:
Not provided
adevinta.com Logo
Adevinta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of hands-on experience in Software Development or Data Engineering
  • at least 5+ years specifically focused on building Data Platforms
  • deep understanding of how Platform infra supports Analytics workloads
  • proven experience evolving complex platforms from legacy patterns to modern, cloud-native solutions
  • deep knowledge of Spark internals, JVM tuning, and performance optimization for high-scale batch and streaming datasets
  • deep expertise in Unity Catalog, Delta Lake internals, and optimizing high-volume workloads
  • strict software engineering discipline (CI/CD, Testing, OOP) applied to data pipelines
  • understanding of microservices architecture
  • understanding the needs of Analytics/DWH teams (Data Modeling, dbt)
  • strong background in building automated pipelines using Terraform/Terragrunt and ensuring system observability
Job Responsibility
Job Responsibility
  • Lead the evolution of our Data Platform and architect the "Data Exchange" strategy
  • define robust patterns for API-based ingestion, Event-Driven Architectures (Kafka), and Reverse ETL
  • ensure architectures are optimized for cost and performance on AWS
  • act as a catalyst for technical evolution
  • constantly scan the horizon for next-generation technologies
  • lead the implementation of new paradigms
  • design the strategy for Unity Catalog implementation and Data Contracts
  • champion FinOps, automating cost controls for our highest-volume workloads
  • build the underlying infrastructure that allows Analytics/DWH teams to run efficient transformations
  • elevate the technical bar of the team, mentoring Staff and Senior engineers
What we offer
What we offer
  • An attractive Base Salary
  • Participation in our Short Term Incentive plan (annual bonus)
  • Work From Anywhere: Enjoy up to 20 days a year of working from anywhere
  • A 24/7 Employee Assistance Program for you and your family
  • a collaborative environment with an opportunity to explore your potential and grow
  • a range of locally relevant benefits
  • Fulltime
Read More
Arrow Right

Senior Software Engineer and Principal Software Engineer - Power Point AI Team

The PowerPoint team is embarking on an exciting new chapter - evolving a product...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • 8+ years of experience in backend service engineering, including work on high-scale infrastructures
  • Proficiency in one or more systems programming languages such as C#, C++
  • 1+ years of experience in software engineering, designing and developing systems (and APIs) that deploy and integrate with AI models
  • 2+ years of experience working with rich telemetry, making data driven decisions, and carrying out rapid experimentation
  • 2+ years of experience building software for scale, performance, and reliability
  • Academic or industry experience with building, finetuning, deploying or building eval-driven systems utilizing the models (any category)
Job Responsibility
Job Responsibility
  • Lead design and delivery of complex, scalable AI features ensuring resilience and exceptional user experience
  • Drive technical strategy and architecture decisions across multiple services, influencing partner teams and aligning with compliance and security requirements
  • Champion modern engineering practices, including AI-driven approaches, automation, and cloud-native patterns, across the full development lifecycle
  • Mentor and guide engineers, fostering technical excellence and continuous improvement in security, reliability, and performance
  • Collaborate cross-org to solve challenging technical problems, streamline processes, and reduce operational costs while improving live-site health
  • Design and implement scalable backend services optimized for machine learning workflows and large language model integration
  • Develop and maintain evaluation-driven systems that leverage text and multimodal inputs (e.g., images) to power visual-creation experiences
  • Build and optimize APIs and infrastructure to support high-performance model inference and experimentation at scale
  • Collaborate with product, ML, and design teams to integrate models into user-facing features, ensuring seamless functionality and performance
  • Conduct model evaluations and experiments, analyze results, and iterate on improvements to enhance accuracy and user experience
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

This role is categorized as hybrid. This means the successful candidate is expec...
Location
Location
United States , Austin
Salary
Salary:
133000.00 - 188600.00 USD / Year
gm.com Logo
General Motors
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of relevant full-time experience
  • or equivalent knowledge
  • Bachelor’s degree in Computer Science, Data Science, Engineering, or a related technical field, or equivalent practical experience.
  • Strong proficiency in Scala, Python, SQL, Spark, and Databricks, with production-quality software engineering practices.
  • Strong experience building and supporting cloud-based ETL/ELT pipelines, data platforms, and pipeline orchestration patterns for both batch and streaming operations.
  • Experience deploying machine learning models into production environments and supporting model development through operational release.
  • Strong experience with Databricks jobs, workflows, repos, Unity Catalog, and MLflow-based experimentation and model lifecycle management.
  • Experience building CI/CD pipelines with Git-based workflows, automated testing, and deployment of data or ML products.
  • Strong troubleshooting, communication, and cross-functional collaboration skills.
Job Responsibility
Job Responsibility
  • Design and develop batch and streaming ETL/ELT pipelines in Azure Databricks to support ingestion, transformation, curation, and delivery of manufacturing and enterprise data
  • Build reusable Bronze, Silver, and Gold data pipelines, data quality frameworks, and governed data products in Unity Catalog
  • Build and operationalize ML workflows for data preparation, feature engineering, model training, evaluation, deployment, batch scoring, retraining, and inference
  • Establish and maintain MLflow-based model lifecycle controls, including experiment tracking, model registration, versioning, lineage, traceability, and controlled promotion across environments
  • Implement CI/CD pipelines for data products, ML code, workflows, and deployment bundles using Git-based development practices, automated testing, and release automation
  • Build and support Databricks jobs and workflows for orchestration, scheduled execution, retraining, scoring, and downstream operational consumption
  • Improve reliability, observability, and production readiness through testing, monitoring, validation, alerting, documentation, and automation that reduce deployment risk and manual intervention
  • Partner with data scientists, data engineers, platform teams, architecture, manufacturing, and business stakeholders to productionize scalable, supportable solutions aligned to production constraints
  • Support modernization of legacy data and ML workflows into standardized cloud-native Databricks patterns and help bridge proof-of-concept work into repeatable production solutions
What we offer
What we offer
  • Medical
  • Dental
  • Vision
  • Health Savings Account
  • Flexible Spending Accounts
  • Retirement savings plan
  • Sickness and accident benefits
  • Life insurance
  • Paid vacation & holidays
  • Tuition assistance programs
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

As a Data Engineer, you will join our Data Platform team and contribute to the d...
Location
Location
Poland , Cracow
Salary
Salary:
18000.00 - 23000.00 PLN / Month
medius.com Logo
Medius
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience developing with Python, including Pandas and PySpark
  • Strong SQL skills and experience working with large-scale datasets
  • Hands-on experience designing and implementing ETL/ELT processes in Data Lakehouse environments
  • Practical experience with Azure cloud services
  • Experience working with Apache Spark and Azure Databricks
  • Understanding of data modeling, data integration, and modern data engineering best practices
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable data processing infrastructure and ETL/ELT pipelines on Azure
  • Build and optimize data solutions within Azure Databricks and Data Lakehouse environments
  • Deliver high-quality, reliable data to support AI/ML applications, analytics, reporting, and other data consumers
  • Ensure data governance, security, integrity, and availability across the data platform
  • Monitor, troubleshoot, and improve data workflows and platform performance
  • Collaborate with cross-functional teams to understand data requirements and implement effective solutions
  • Stay current with emerging technologies and best practices in data engineering
  • Share knowledge and contribute to continuous learning within the team
What we offer
What we offer
  • 26 days PTO
  • medical, dental, and vision coverage
  • paid time off
  • retirement benefits
  • variable compensation (performance bonus or commission)
  • Fulltime
Read More
Arrow Right