CrawlJobs Logo
Tech Mahindra Logo Tech Mahindra · IT - Software Development

Sr. Azure Data Engineer

India, Pune · Job Posted June 15, 2026
Apply Position
Job Link Share

Job Description

Senior Data Engineer. Role: Senior Data Engineer and Migration Execution. Location: Offshore (India). Experience: 5+ years in data engineering with cloud platforms. Role Summary: Execute priority workload migrations across all 3 tracks (L&S, Refactor, New Build) building production grade pipelines on Databricks and Snowflake, running dual run validations, and remediating technical debt across schemas, naming, and lineage.

Job Responsibility

  • Execute automated Lift & Shift migrations schema mapping, Iceberg table creation, pipeline generation for Bronze/Silver tables
  • Develop and refactor ETL/ELT pipelines decompose monolithic PSQL/Glue jobs into modular PySpark/Snowpark patterns
  • Review, validate, and refine AI assisted code conversions (PySpark/Snowpark)
  • label AI generated lines in code comments per EchoStar Addendum
  • Configure and execute dual run validation scripts row count, hash-based integrity, business rule comparisons
  • Rationalize schemas, naming conventions, and lineage for AI Factory consumption patterns and Unity Catalog / Horizon Catalog governance
  • Develop and deploy Airflow DAGs replacing Control M orchestration
  • Produce before/after metrics (complexity, runtime, cost) for Technical Debt Remediation Log
  • Create operations runbooks, SOPs, and procedural guides for each migrated workload
  • Support EchoStar FTE shadowing and reverse shadowing during KT phases

Requirements

  • Bachelor's degree
  • SQL
  • Python
  • Azure
  • Databricks
  • Total Experience: 4.00 to 7.00 Years
  • Strong proficiency in SQL, Python, PySpark, Apache Spark
  • Hands-on experience with Databricks (Delta, DLT, Autoloader, Unity Catalog) and/or Snowflake (Snowpark, Snowpipe, Streams, Tasks)
  • Experience with AWS services S3, Redshift, Athena, Glue, EMR, Lambda
  • Hands on with Apache Airflow DAG development
  • Experience with GitLab for version control and CI/CD
  • Strong SQL performance tuning and optimization skills
  • Experience with data quality validation and reconciliation
  • 5+ years in data engineering with cloud platforms

Nice to have

  • Experience migrating from Redshift/Athena to Databricks or Snowflake
  • Familiarity with Control M and Control M and Airflow migration
  • Experience with Qlik Replication / AWS DMS
  • Shell scripting (Linux/Unix)
  • AWS or Databricks certifications
Tech Mahindra - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Sr. Azure Data Engineer

8 matching positions

New

Data Engineer Sr (Databricks)

Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in databricks with understanding complex legacy data models and getting that data into the cloud: Extracting DB2/AS400: Experience with Change Data Capture (CDC) or scheduled batch extractions from DB2 into cloud storage. Involves working through JDBC connections, mapping table dependencies, and re-platforming legacy SQL to distributed computing standards
  • Handling Guidewire Data: Integrating with Guidewire Cloud Data Access (CDA) or InsuranceSuite to replicate complex P&C (Property & Casualty) insurance schemas. Senior engineers parse these highly normalized operational databases and transform them into analytical-friendly schemas in the cloud
  • 2. Architecture & Pipeline Development: The core of the experience involves transitioning these legacy, row-based stores into a scalable Medallion Architecture (Bronze, Silver, Gold layers)
  • Delta Lake Optimization: Using Databricks and Apache Spark to build ETL/ELT data pipelines with ACID transactions. Senior engineers handle schema evolution, upserts, and slowly changing dimensions (SCD Type 2)
  • Business Logic Refactoring: Translating rigid legacy procedural code (e.g., RPG/COBOL background logic, stored procedures) into scalable distributed patterns (PySpark, Spark SQL, and Scala)
  • 3. Data Governance & Observability: A senior engineer is expected to govern vast amounts of incoming and generated data across the enterprise
  • Unity Catalog: Implementing strict data governance, lineage tracing, and table-level security
  • Data Quality: Automating data validation frameworks to ensure a seamless transition from legacy to modern systems without data loss or corruption
  • 4. Integration with the Databricks Platform Ecosystem: Moving beyond basic storage to utilizing the full power of the Databricks Data Intelligence Platform
  • Serverless Compute: Managing Databricks serverless resources, ensuring optimal cluster sizing, and reducing compute costs
Job Responsibility
Job Responsibility
  • Develop Databricks notebooks, jobs, and workflows to replicate and enhance DB2/Guidewire-based pipelines and transformations
  • Implement Delta Lake tables and patterns (bronze/silver/gold, ACID, time travel, schema evolution) for migrated data
  • Integrate Databricks with AWS/S3 or Azure ADLS, ADF/Synapse, Key Vault, and Snowflake as required
  • Optimize Databricks clusters, jobs, and queries for performance and cost
  • Implement incremental loads, CDC patterns, and batch schedules for large datasets
  • Collaborate with Snowflake and dbt teams to ensure consistent data models and data contracts
  • Participate in data validation and reconciliation between DB2 400 / Guiderwire and Databricks outputs
  • Follow coding standards, version control, and CI/CD practices using Git/Azure DevOps
  • Provide defect fixes and support during SIT/UAT and post go-live stabilization
Read More
Arrow Right

Sr Data Engineer

We’re seeking a Data Engineer with strong Databricks and Azure experience.
Location
Location
United States , Jacksonville
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Databricks + Spark
  • Azure Data Factory or similar
  • SQL and Python
Job Responsibility
Job Responsibility
  • Build and maintain data pipelines
  • Work with large-scale data processing frameworks
  • Optimize data flows and storage solutions
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • Company 401(k) plan
  • Free online training
Read More
Arrow Right

Sr. Data Engineer

The Senior Data Engineer supports enterprise data unification and analytics init...
Location
Location
United States , Deerfield
Salary
Salary:
Not provided
tier4group.com Logo
Tier4 Group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience designing, developing, and supporting data pipelines (ETL/ELT) that integrate data from multiple systems
  • Strong SQL skills, including writing and optimizing complex queries, joins, and stored procedures in Microsoft SQL Server or comparable relational databases
  • Hands-on experience with Azure Synapse Analytics, Azure Data Factory, or similar cloud-based data warehousing and integration platforms
  • Experience working with large datasets in cloud or hybrid data environments
  • Working knowledge of data modeling concepts, including fact and dimension tables and schema design for analytics and reporting
  • Experience supporting business intelligence tools, particularly Microsoft Power BI, including datasets and dataflows
  • Ability to use scripting or programming languages such as SQL, Python, or PySpark for data transformation and automation
  • Approximately 3–5 years of professional experience in data engineering, analytics engineering, or a related role
  • Approximately 1–3 years of experience in data modeling or database design for analytics use cases
  • Undergraduate degree or equivalent experience in Computer Science, Information Systems, or a related field
Job Responsibility
Job Responsibility
  • Design, build, and maintain automated ETL/ELT data pipelines that ingest and transform data from Microsoft Dynamics 365 and legacy systems into an Azure Synapse data lake and enterprise data warehouse
  • Monitor, optimize, and support data pipeline performance to ensure reliable, timely data refreshes and efficient resource utilization
  • Implement data quality checks, validation rules, and cleansing processes to ensure data accuracy, consistency, and readiness for enterprise-wide analysis
  • Support data unification efforts by integrating data from multiple business units and systems without altering source system integrity
  • Contribute to the design and evolution of enterprise data models, including dimensional and star schemas, to support standardized reporting and unified business definitions
  • Define and maintain master data structures and relationships that enable analysis across both ERP and non-ERP data sources
  • Prepare curated and optimized datasets for business intelligence and analytics use cases, including Power BI dashboards and self-service reporting
  • Write and optimize SQL queries and develop new pipeline components to support reporting, analytics, and ad hoc data needs
  • Collaborate with business analysts, business intelligence developers, ERP specialists, and other stakeholders to translate reporting and analytics requirements into technical solutions
  • Apply data engineering and analytics best practices, including version control, documentation, code review, and performance tuning
  • Fulltime
Read More
Arrow Right

Sr Engineer, Data

This role is essential for designing and developing data architectures across on...
Location
Location
United States , Bothell
Salary
Salary:
105100.00 - 189600.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree plus 5 years of related work experience OR Advanced degree with 3 years of related experience (Required)
  • Acceptable areas of study include Computer Engineering, Computer Science, a related subject area (Required)
  • 4-7 years Developing cloud solutions using data series
  • experience with cloud platforms (Amazon Web Services, Azure, or Google Cloud) (Required)
  • 4-7 years Hands-on development using and migrating data to cloud platforms (Required)
  • 4-7 years Experience in SQL, NoSQL, and/or relational database design and development (Required)
  • 4-7 years Advanced knowledge and experience in building complex data pipelines with Python, Experience in languages such as SQL, DAX Python, Java, Scala, and/or Go (Required)
  • Cloud Computing (Required)
  • Collaboration (Required)
  • Data Analysis (Required)
Job Responsibility
Job Responsibility
  • Develop data engineering solutions that enable data pipelines, visualization, and analytical tools to support business requirements
  • Design and develop data architectures across on-premise, cloud, and hybrid platforms to ensure scalable data infrastructure
  • Architect, build, and maintain reliable data pipelines that ingest, transform, and deliver data from diverse customer and business data sources into enterprise data lake, warehouse, and analytics platforms
  • Design reusable data models and standardized schemas, including Customer 360 and customer behavioral data structures, to support consistent metric definitions and trusted downstream use by analytics, marketing, and AI/ML teams
  • Perform data wrangling, exploration, and discovery of heterogeneous data to generate new business insights
  • Embed data quality practices into pipeline development, including automated checks for completeness, accuracy, timeliness, and governance alignment
  • Build and curate self-service data products, metadata, and documentation that improve data discoverability and enable approved users to access trusted datasets efficiently
  • Partner with analytics, marketing, AI/ML, and business stakeholders to understand data needs, define data contracts, and support scalable customer data use cases
  • Contribute to team knowledge sharing and drive the advancement of new data engineering capabilities
  • Mentor team members to build and enhance their data engineering skillsets and professional growth
What we offer
What we offer
  • Medical, dental and vision insurance
  • flexible spending account
  • 401(k)
  • employee stock grants
  • employee stock purchase plan
  • paid time off
  • up to 12 paid holidays
  • paid parental and family leave
  • family building benefits
  • back-up care
  • Fulltime
Read More
Arrow Right

Sr. Data Engineer III

Piper Companies is seeking a Sr. Data Engineer III to join an enterprise technol...
Location
Location
United States
Salary
Salary:
130000.00 - 150000.00 USD / Year
pipercompanies.com Logo
Piper Companies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of data engineering or related experience
  • Strong expertise in SQL, Python, dbt, and or C#
  • Experience with Azure Data Factory or similar orchestration tools
  • Hands-on experience with Microsoft Azure and Fabric
  • Experience with Databricks or Snowflake is acceptable
  • Strong understanding of data modeling and ETL concepts
  • Experience with Git and version control best practices
  • Ability to work independently and solve complex problems
  • Background working in small or mid-sized environments with leadership experience
  • Must be authorized to work in the United States
Job Responsibility
Job Responsibility
  • Design and maintain scalable data pipelines and ETL workflows
  • Develop and optimize advanced SQL queries for performance and reliability
  • Leverage Azure, Fabric, Spark, and Python to automate complex data processes
  • Build and support REST and SOAP APIs for system integrations
  • Monitor and troubleshoot data pipelines to ensure high availability
  • Identify and resolve bottlenecks in data workflows
  • Lead engineering efforts supporting AI and machine learning initiatives
  • Collaborate with stakeholders and cross-functional teams
  • Conduct code reviews and contribute to technical documentation
  • Provide Tier 3 support for reporting and integrated systems
What we offer
What we offer
  • medical
  • dental
  • vision
  • 401k
  • PTO
  • sick leave as required by law
  • Fulltime
Read More
Arrow Right

Sr Data Engineer

The purpose of the Data Engineer position is to leverage their technical experti...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
July 03, 2026
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Data Science, Information Technology, or a related field with a focus on data engineering or data analytics
  • Strong proficiency in programming languages such as Python, SQL, Java, or Scala for data processing and analysis
  • Experience with data modeling, ETL processes, data warehousing, data integration, and data pipeline development
  • Proficiency in relational databases (e.g., SQL, PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra)
  • Working knowledge of cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform for data storage and processing
  • Experience with data visualization tools (e.g., Power BI) to create meaningful insights from data
  • Understanding of data quality principles, data governance, and data validation processes
  • Ability to analyze complex data sets, identify trends, patterns, and insights to drive data-driven decision-making
  • Proficiency in troubleshooting data-related issues, identifying root causes, and implementing solutions
  • Familiarity with project management methodologies to contribute effectively to project planning and execution
Job Responsibility
Job Responsibility
  • Build pipelines using standard patterns for data pipelines and workflows utilizing Streamsets, Kestra, dbt, Git
  • Design and Implement data storage and processing solutions employing Snowflake
  • Utilize AWS services for cloud-based platform tooling infrastructure including but not limited to Lambda,ECS,MSK,RDS,EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge
  • Utilize Terraform for AWS and Azure deployments
  • Leverage and integrate APIs for data access and manipulation
  • Write Python scripts for data common processing and automation tasks
  • Leverage Platform API’s and Web Applications to enforce Platform Security
  • Create and execute comprehensive test plans to ensure the pipelines functionality and performance
  • Develop unit tests, integration tests, and end-to-end tests for data pipelines and workflows
  • Ensure data accuracy and consistency through rigorous testing processes
What we offer
What we offer
  • Commitment to your ongoing development, including on the job opportunities and formal programs
  • Inclusive parental leave entitlements for both parents
  • Values led culture
  • Flexible work options
  • Generous annual leave, sick leave and casual leave
  • Cultural and religious leave with flexible public holiday opportunities
  • A competitive remuneration package featuring performance based incentives with uncapped Employer Provident Fund
  • Fulltime
Read More
Arrow Right

Sr. Data Engineer - Python Developer

Seeking a hands-on Senior Data Engineer (ETL / Python Developer) to support the ...
Location
Location
United States , Springfield
Salary
Salary:
52.00 - 54.00 USD / Hour
myticas.com Logo
Myticas Consulting
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of data engineering experience with a focus on enterprise data warehousing
  • 5+ years of hands-on ETL development using Informatica PowerCenter, Azure Data Factory, or similar tools
  • 5+ years of Python development for data engineering and automation
  • 3+ years of experience with Spark-based processing frameworks (Databricks or equivalent)
  • Strong SQL expertise and experience with relational databases (such as Teradata, Snowflake, Oracle, SQL Server)
  • Experience with source control and DevOps practices (Azure DevOps, GitHub, CI/CD)
  • Bachelor's degree or higher in Computer Science, Engineering, Analytics, or a related field
  • Strong analytical, problem-solving, and troubleshooting skills
Job Responsibility
Job Responsibility
  • Design, develop, and maintain enterprise ETL pipelines using Azure Data Factory (ADF), Informatica PowerCenter, and Python-based frameworks
  • Build and optimize scalable data processing solutions using Python, Spark, and Databricks
  • Support Medicaid analytics and federal reporting initiatives (e.g., T-MSIS, PERM, MARS, Quality of Care)
  • Develop robust data validation, reconciliation, and audit-traceable data pipelines
  • Write and optimize SQL and stored procedures across relational platforms such as Snowflake, Oracle, and SQL Server
  • Participate in cloud migration and modernization initiatives within Azure-based architectures
  • Collaborate with analysts, QA, and reporting teams to ensure data quality, accuracy, and timeliness
  • Follow data engineering best practices for performance, reliability, reusability, and security
  • Support production operations, incident resolution, and root-cause analysis
  • Participate in code reviews, source control, and CI/CD processes using Azure DevOps and GitHub
  • Fulltime
Read More
Arrow Right

Sr Data Engineer

We are looking for a full stack Azure Data Engineer to serve as the primary tech...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.
Job Responsibility
Job Responsibility
  • Technology Stack Ownership & Operations
  • Proof-of-Concept Management & Evaluation
  • Data Engineering & Pipeline Development
  • Stakeholder Partnership & Communication
  • Fulltime
Read More
Arrow Right