Sign In Post Your Job

Senior Data Engineer

Tech Mahindra

Location:
India, Hyderabad

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

Not provided

Save Job

Job Description:

We are seeking a highly skilled and motivated Senior Data Engineer/s to architect and implement scalable ETL and data storage solutions using Microsoft Fabric and the broader Azure technology stack. This role will be pivotal in building a metadata driven data lake that ingests data from over 100 structured and semi structured sources, enabling rich insights through canned reports, conversational agents, and analytics dashboards.

Job Responsibility:

Design and implement ETL pipelines using Microsoft Fabric (Dataflows, Pipelines, Lakehouse ,warehouse, sql) and Azure Data Factory
Build and maintain a metadata driven Lakehouse architecture with threaded datasets to support multiple consumption patterns
Develop agent specific data lakes and an orchestration layer for an uber agent that can query across agents to answer customer questions
Enable interactive data consumption via Power BI, Azure OpenAI, and other analytics tools
Ensure data quality, lineage, and governance across all ingestion and transformation processes
Collaborate with product teams to understand data needs and deliver scalable solutions
Optimize performance and cost across storage and compute layers

Requirements:

7-10 years of experience in data engineering with a focus on Microsoft Azure and Fabric technologies
Strong expertise in: Microsoft Fabric (Lakehouse, Dataflows Gen2, Pipelines, Notebooks)
Strong expertise in: Azure Data Factory, Azure SQL, Azure Data Lake Storage Gen2
Strong expertise in: Power BI and/or other visualization tools
Strong expertise in: Azure Functions, Logic Apps, and orchestration frameworks
Strong expertise in: SQL, Python and PySpark/Scala
Experience working with structured and semi structured data (JSON, XML, CSV, Parquet)
Proven ability to build metadata driven architectures and reusable components
Strong understanding of data modeling, data governance, and security best practices

Nice to have:

Familiarity with agent based architectures and conversational AI integration is a plus
Experience with Azure OpenAI, Copilot Studio, or similar conversational platforms
Knowledge of CI/CD pipelines and DevOps practices for data engineering
Experience in building multi tenant data platforms or domain specific lakes

Additional Information:

Job Posted:
December 12, 2025

Expiration:
January 30, 2026

View All Jobs In This Company

Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Suggested Tags:

Frontend Developer , Secretary , Talent Manager , Salesforce Administrator , Data Analyst

Similar Jobs for Senior Data Engineer

Senior Data Engineer – Data Engineering & AI Platforms

We are looking for a highly skilled Senior Data Engineer (L2) who can design, bu...

Location

India , Chennai, Madurai, Coimbatore

Salary

OptiSol Business Solutions

Expiration Date

Until further notice

Requirements

Strong hands-on expertise in cloud ecosystems (Azure / AWS / GCP)
Excellent Python programming skills with data engineering libraries and frameworks
Advanced SQL capabilities including window functions, CTEs, and performance tuning
Solid understanding of distributed processing using Spark/PySpark
Experience designing and implementing scalable ETL/ELT workflows
Good understanding of data modeling concepts (dimensional, star, snowflake)
Familiarity with GenAI/LLM-based integration for data workflows
Experience working with Git, CI/CD, and Agile delivery frameworks
Strong communication skills for interacting with clients, stakeholders, and internal teams

Job Responsibility

Job Responsibility

Design, build, and maintain scalable ETL/ELT pipelines across cloud and big data platforms
Contribute to architectural discussions by translating business needs into data solutions spanning ingestion, transformation, and consumption layers
Work closely with solutioning and pre-sales teams for technical evaluations and client-facing discussions
Lead squads of L0/L1 engineers—ensuring delivery quality, mentoring, and guiding career growth
Develop cloud-native data engineering solutions using Python, SQL, PySpark, and modern data frameworks
Ensure data reliability, performance, and maintainability across the pipeline lifecycle—from development to deployment
Support long-term ODC/T&M projects by demonstrating expertise during technical discussions and interviews
Integrate emerging GenAI tools where applicable to enhance data enrichment, automation, and transformations

What we offer

Opportunity to work at the intersection of Data Engineering, Cloud, and Generative AI
Hands-on exposure to modern data stacks and emerging AI technologies
Collaboration with experts across Data, AI/ML, and cloud practices
Access to structured learning, certifications, and leadership mentoring
Competitive compensation with fast-track career growth and visibility

Fulltime

Senior Software Engineer, Data Engineering

Join us in building the future of finance. Our mission is to democratize finance...

Location

United States , Menlo Park

Salary

146000.00 - 198000.00 USD / Year

Expiration Date

Until further notice

Requirements

5+ years of professional experience building end-to-end data pipelines
Hands-on software engineering experience, with the ability to write production-level code in Python for user-facing applications, services, or systems (not just data scripting or automation)
Expert at building and maintaining large-scale data pipelines using open source frameworks (Spark, Flink, etc)
Strong SQL (Presto, Spark SQL, etc) skills
Experience solving problems across the data stack (Data Infrastructure, Analytics and Visualization platforms)
Expert collaborator with the ability to democratize data through actionable insights and solutions

Job Responsibility

Job Responsibility

Help define and build key datasets across all Robinhood product areas. Lead the evolution of these datasets as use cases grow
Build scalable data pipelines using Python, Spark and Airflow to move data from different applications into our data lake
Partner with upstream engineering teams to enhance data generation patterns
Partner with data consumers across Robinhood to understand consumption patterns and design intuitive data models
Ideate and contribute to shared data engineering tooling and standards
Define and promote data engineering best practices across the company

What we offer

Market competitive and pay equity-focused compensation structure
100% paid health insurance for employees with 90% coverage for dependents
Annual lifestyle wallet for personal wellness, learning and development, and more
Lifetime maximum benefit for family forming and fertility benefits
Dedicated mental health support for employees and eligible dependents
Generous time away including company holidays, paid time off, sick time, parental leave, and more
Lively office environment with catered meals, fully stocked kitchens, and geo-specific commuter benefits
Bonus opportunities
Equity

Fulltime

Senior Data Engineer

As a Data Software Engineer, you will leverage your skills in data science, mach...

Location

United States , Arlington; Woburn

Salary

134000.00 - 184000.00 USD / Year

Expiration Date

Until further notice

Requirements

Ability to obtain a Top Secret (TS) security clearance, for which U.S citizenship is needed by the U.S government
5+ years of experience in one or more high level programming languages, like Python
Experience working with large datasets for machine learning applications
Experience in navigating and contributing to complex, large code bases
Experience with containerization practices and CI/CD workflows
BS, MS, or PhD in a related field or equivalent experience

Job Responsibility

Job Responsibility

Explore a variety of data sources to build and integrate machine learning models into software
Develop machine learning and deep learning algorithms for large-scale multi-modal problems
Work with large data sets and develop software solutions for scalable analysis
Collaborate to create and maintain software for data pipelines, algorithms, storage, and access
Monitor software deployments, create logging frameworks, and design APIs
Build analytic tools that utilize data pipelines to provide actionable insights into customer requests
Develop and execute plans to improve software robustness and ensure system performance

Fulltime

Senior Data Engineer - Platform Enablement

SoundCloud empowers artists and fans to connect and share through music. Founded...

Location

United States , New York; Atlanta; East Coast

Salary

160000.00 - 210000.00 USD / Year

Expiration Date

Until further notice

Requirements

7+ years of experience in data engineering, analytics engineering, or similar roles
Expert-level SQL skills, including performance tuning, advanced joins, CTEs, window functions, and analytical query design
Proven experience with Apache Airflow (designing DAGs, scheduling, task dependencies, monitoring, Python)
Familiarity with event-driven architectures and messaging systems (Pub/Sub, Kafka, etc.)
Knowledge of data governance, schema management, and versioning best practices
Understanding observability practices: logging, metrics, tracing, and incident response
Experience deploying and managing services in cloud environments, preferably GCP, AWS
Excellent communication skills and a collaborative mindset

Job Responsibility

Job Responsibility

Develop and optimize SQL data models and queries for analytics, reporting, and operational use cases
Design and maintain ETL/ELT workflows using Apache Airflow, ensuring reliability, scalability, and data integrity
Collaborate with analysts and business teams to translate data needs into efficient, automated data pipelines and datasets
Own and enhance data quality and validation processes, ensuring accuracy and completeness of business-critical metrics
Build and maintain reporting layers, supporting dashboards and analytics tools (e.g. Looker, or similar)
Troubleshoot and tune SQL performance, optimizing queries and data structures for speed and scalability
Contribute to data architecture decisions, including schema design, partitioning strategies, and workflow scheduling
Mentor junior engineers, advocate for best practices and promote a positive team culture

What we offer

Comprehensive health benefits including medical, dental, and vision plans, as well as mental health resources
Robust 401k program
Employee Equity Plan
Generous professional development allowance
Creativity and Wellness benefit
Flexible vacation and public holiday policy where you can take up to 35 days of PTO annually
16 paid weeks for all parents (birthing and non-birthing), regardless of gender, to welcome newborns, adopted and foster children
Various snacks, goodies, and 2 free lunches weekly when at the office

Fulltime

Senior Data Engineer

SoundCloud is looking for a Senior Data Engineer to join our growing Content Pla...

Location

Germany; United Kingdom , Berlin; London

Salary

Expiration Date

Until further notice

Requirements

Proven experience in backend engineering (Scala/Go/Python) with strong design and data modeling skills
Hands-on experience building ETL/ELT pipelines and streaming solutions on cloud platforms (GCP preferred)
Proficient in SQL and experienced with relational and NoSQL databases
Familiarity with event-driven architectures and messaging systems (Pub/Sub, Kafka, etc.)
Knowledge of data governance, schema management, and versioning best practices
Understanding observability practices: logging, metrics, tracing, and incident response
Experience with containerization and orchestration (Docker, Kubernetes)
Experience deploying and managing services in cloud environments, preferably GCP, AWS
Strong collaboration skills and ability to work across backend, data, and product teams

Job Responsibility

Job Responsibility

Design, build, and maintain high-performance services for content modeling, serving, and integration
Develop data pipelines (batch & streaming) with cloud native tools
Collaborate on rearchitecting the content model to support rich metadata
Implement APIs and data services that power internal products, external integrations, and real-time features
Ensure data quality, governance, and validation across ingestion, storage, and serving layers
Optimize system performance, scalability, and cost efficiency for both backend services and data workflows
Work with infrastructure-as-code (Terraform) and CI/CD pipelines for deployment and automation
Monitor, debug, and improve reliability using various observability tools (logging, tracing, metrics)
Collaborate with product leadership, music industry experts, and engineering teams across SoundCloud

What we offer

Extensive relocation support including allowances, one way flights, temporary accommodation and on the ground support on arrival
Creativity and Wellness benefit
Employee Equity Plan
Generous professional development allowance
Flexible vacation and public holiday policy where you can take up to 35 days of PTO annually
16 paid weeks for all parents (birthing and non-birthing), regardless of gender, to welcome newborns, adopted and foster children
Free German courses at beginning, intermediate and advanced
Various snacks, goodies, and 2 free lunches weekly when at the office

Fulltime

Senior Data Engineer

Senior Data Engineer – Dublin (Hybrid) Contract Role | 3 Days Onsite. We are see...

Location

Ireland , Dublin

Salary

Solas IT Recruitment

Expiration Date

Until further notice

Requirements

7+ years of experience as a Data Engineer working with distributed data systems
4+ years of deep Snowflake experience, including performance tuning, SQL optimization, and data modelling
Strong hands-on experience with the Hadoop ecosystem: HDFS, Hive, Impala, Spark (PySpark preferred)
Oozie, Airflow, or similar orchestration tools
Proven expertise with PySpark, Spark SQL, and large-scale data processing patterns
Experience with Databricks and Delta Lake (or equivalent big-data platforms)
Strong programming background in Python, Scala, or Java
Experience with cloud services (AWS preferred): S3, Glue, EMR, Redshift, Lambda, Athena, etc.

Job Responsibility

Job Responsibility

Build, enhance, and maintain large-scale ETL/ELT pipelines using Hadoop ecosystem tools including HDFS, Hive, Impala, and Oozie/Airflow
Develop distributed data processing solutions with PySpark, Spark SQL, Scala, or Python to support complex data transformations
Implement scalable and secure data ingestion frameworks to support both batch and streaming workloads
Work hands-on with Snowflake to design performant data models, optimize queries, and establish solid data governance practices
Collaborate on the migration and modernization of current big-data workloads to cloud-native platforms and Databricks
Tune Hadoop, Spark, and Snowflake systems for performance, storage efficiency, and reliability
Apply best practices in data modelling, partitioning strategies, and job orchestration for large datasets
Integrate metadata management, lineage tracking, and governance standards across the platform
Build automated validation frameworks to ensure accuracy, completeness, and reliability of data pipelines
Develop unit, integration, and end-to-end testing for ETL workflows using Python, Spark, and dbt testing where applicable

Senior Data Engineer

We are looking for a foundational member of the Data Team to enable Skydio to ma...

Location

United States , San Mateo

Salary

170000.00 - 230000.00 USD / Year

Expiration Date

Until further notice

Requirements

5+ years of professional experience
2+ years in software engineering
2+ years in data engineering with a bias towards getting your hands dirty
Deep experience with Databricks or Palantir Foundry, including building pipelines, managing datasets, and developing dashboards or analytical applications
Proven track record of operating scalable data platforms, defining company-wide patterns that ensure reliability, performance, and cost effectiveness
Proficiency in SQL and at least one modern programming language (for example, Python or Java)
Strong communication skills, with the ability to collaborate effectively across all levels and functions
Demonstrated ability to lead technical direction, mentor teammates, and promote engineering excellence and best practices across the organization
Familiarity with AI-assisted data workflows, including tools that accelerate data transformations or enable natural-language interfaces for analytics

Job Responsibility

Job Responsibility

Design and scale the data infrastructure that ingests live telemetry from tens of thousands of autonomous drones
Build and evolve our Databricks and Palantir Foundry environments
Develop data systems that make our products truly data-driven
Create and integrate AI-powered tools for data analysis, transformation, and pipeline generation
Champion a data-driven culture by defining and enforcing best practices for data quality, lineage, and governance
Collaborate with autonomy, manufacturing, and operations teams to unify how data flows across the company
Lead and mentor data engineers, analysts, and stakeholders across Skydio
Ensure platform reliability by implementing robust monitoring, observability, and contributing to the on-call rotation for critical data systems

What we offer

Equity in the form of stock options
Comprehensive benefits packages
Relocation assistance may also be provided for eligible roles
Group health insurance plans
Paid vacation time
Sick leave
Holiday pay
401K savings plan

Fulltime

Senior Data Engineer

Are you an experienced Data Engineer ready to tackle complex, high-load, and dat...

Location

Salary

Sigma Software Group

Expiration Date

Until further notice

Requirements

Apache Spark / expert
Python / expert
SQL / expert
Kafka / good
Data Governance (Apache Ranger/Atlas) / good

What we offer

Diversity of Domains & Businesses
Variety of technology
Health & Legal support
Active professional community
Continuous education and growing
Flexible schedule
Remote work
Outstanding offices (if you choose it)
Sports and community activities

Fulltime

Senior Data Engineer

We are looking for a Senior Data Engineer to join one of the best team at Sigma ...

Location

Salary

Sigma Software Group

Expiration Date

Until further notice

Requirements

Python / strong
SQL / strong
Snowflake / good
English / strong

What we offer

Diversity of Domains & Businesses
Variety of technology
Health & Legal support
Active professional community
Continuous education and growing
Flexible schedule
Remote work
Outstanding offices (if you choose it)
Sports and community activities

Senior Data Engineer

At Relatient, we’re on a mission to simplify access to care – intelligently. As ...

Location

Salary

Expiration Date

Until further notice

Requirements

Bachelor's degree, B.E./ B. Tech, computer engineering, or equivalent work experience in lieu of a degree is required, Master’s degree preferred
7+ years of experience in database engineering, data warehousing, or data architecture
Proven expertise with at least one major data warehouse platform (e.g. Postgres, Snowflake, Redshift, BigQuery)
Strong SQL and ETL/ELT development skills
Deep understanding of data modeling
Experience with cloud data ecosystems (AWS)
Hands-on experience with orchestration tools and version control (Git)
Experience in data governance, security, and compliance best practices
Experience building/generating analytical reports using Power BI

Job Responsibility

Job Responsibility

Architect, design, and implement robust end-to-end data warehouse (DW) solutions using modern technologies (e.g. Postgres or on-prem solutions)
Define data modeling standards (dimensional and normalized) and build ETL/ELT pipelines for efficient data flow and transformation
Integrate data from multiple sources (ERP, CRM. APIs, flat files, real-time streams)
Develop and maintain scalable and reliable data ingestion, transformation, and storage pipelines
Ensure data quality, consistency, and lineage across all data systems
Analyst and tune SQL queries, schemas, indexes, and ETL process to maximize database and warehouse performance
Monitor data systems and optimize storage costs and query response times
Implement high availability, backup, disaster recovery, and data security strategies
Collaborate with DevOps and Infrastructure teams to ensure optimal deployment, scaling, and performance of DW environments
Work closely with Data Scientists, Analysts, and Business Teams to translate business needs into technical data solutions

What we offer

INR 5,00,000/- of life insurance coverage for all full-time employees and their immediate family
INR 15,00,000/- of group accident insurance
Education reimbursement
10 national and state holidays, plus 1 floating holiday
Flexible working hours and a hybrid policy

Fulltime

Senior Data Engineer

At Rearc, we're committed to empowering engineers to build awesome products and ...

Location

India , Bangalore

Salary

Expiration Date

Until further notice

Requirements

8+ years of experience in data engineering, showcasing expertise in diverse architectures, technology stacks, and use cases
Strong expertise in designing and implementing data warehouse and data lake architectures, particularly in AWS environments
Extensive experience with Python for data engineering tasks, including familiarity with libraries and frameworks commonly used in Python-based data engineering workflows
Proven experience with data pipeline orchestration using platforms such as Airflow, Databricks, DBT or AWS Glue
Hands-on experience with data analysis tools and libraries like Pyspark, NumPy, Pandas, or Dask
Proficiency with Spark and Databricks is highly desirable
Experience with SQL and NoSQL databases, including PostgreSQL, Amazon Redshift, Delta Lake, Iceberg and DynamoDB
In-depth knowledge of data architecture principles and best practices, especially in cloud environments
Proven experience with AWS services, including expertise in using AWS CLI, SDK, and Infrastructure as Code (IaC) tools such as Terraform, CloudFormation, or AWS CDK
Exceptional communication skills, capable of clearly articulating complex technical concepts to both technical and non-technical stakeholders

Job Responsibility

Job Responsibility

Strategic Data Engineering Leadership: Provide strategic vision and technical leadership in data engineering, guiding the development and execution of advanced data strategies that align with business objectives
Architect Data Solutions: Design and architect complex data pipelines and scalable architectures, leveraging advanced tools and frameworks (e.g., Apache Kafka, Kubernetes) to ensure optimal performance and reliability
Drive Innovation: Lead the exploration and adoption of new technologies and methodologies in data engineering, driving innovation and continuous improvement across data processes
Technical Expertise: Apply deep expertise in ETL processes, data modelling, and data warehousing to optimize data workflows and ensure data integrity and quality
Collaboration and Mentorship: Collaborate closely with cross-functional teams to understand requirements and deliver impactful data solutions—mentor and coach junior team members, fostering their growth and development in data engineering practices
Thought Leadership: Contribute to thought leadership in the data engineering domain through technical articles, conference presentations, and participation in industry forums

Senior Data Engineer

As a Senior Data Engineer at Rearc, you'll play a pivotal role in establishing a...

Location

United States , New York

Salary

160000.00 - 200000.00 USD / Year

Expiration Date

Until further notice

Requirements

8+ years of professional experience in data engineering across modern cloud architectures and diverse data systems
Expertise in designing and implementing data warehouses and data lakes across modern cloud environments (e.g., AWS, Azure, or GCP), with experience in technologies such as Redshift, BigQuery, Snowflake, Delta Lake, or Iceberg
Strong Python experience for data engineering, including libraries like Pandas, PySpark, NumPy, or Dask
Hands-on experience with Spark and Databricks (highly desirable)
Experience building and orchestrating data pipelines using Airflow, Databricks, DBT, or AWS Glue
Strong SQL skills and experience with both SQL and NoSQL databases (PostgreSQL, DynamoDB, Redshift, Delta Lake, Iceberg)
Solid understanding of data architecture principles, data modeling, and best practices for scalable data systems
Experience with cloud provider services (AWS, Azure, or GCP) and comfort using command-line interfaces or SDKs as part of development workflows
Familiarity with Infrastructure as Code (IaC) tools such as Terraform, CloudFormation, ARM/Bicep, or AWS CDK
Excellent communication skills, able to explain technical concepts to technical and non-technical stakeholders

Job Responsibility

Job Responsibility

Provide strategic data engineering leadership by shaping the vision, roadmap, and technical direction of data initiatives to align with business goals
Architect and build scalable, reliable data solutions, including complex data pipelines and distributed systems, using modern frameworks and technologies (e.g., Spark, Kafka, Kubernetes, Databricks, DBT)
Drive innovation by evaluating, proposing, and adopting new tools, patterns, and methodologies that improve data quality, performance, and efficiency
Apply deep technical expertise in ETL/ELT design, data modeling, data warehousing, and workflow optimization to ensure robust, high-quality data systems
Collaborate across teams—partner with engineering, product, analytics, and customer stakeholders to understand requirements and deliver impactful, scalable solutions
Mentor and coach junior engineers, fostering growth, knowledge-sharing, and best practices within the data engineering team
Contribute to thought leadership through knowledge-sharing, writing technical articles, speaking at meetups or conferences, or representing the team in industry conversations

What we offer

Health Benefits
Generous time away
Maternity and Paternity leave
Educational resources and reimbursements
401(k) plan with a company contribution

Fulltime

Welcome to CrawlJobs.com –
Your Global Job Discovery Platform

At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.