CrawlJobs Logo
Tech Mahindra Logo Tech Mahindra · IT - Software Development

Senior Azure Data Engineer

India, Pune · Job Posted June 15, 2026
Apply Position
Job Link Share

Job Description

Senior Data Engineer Role: Senior Data Engineer ‿ Migration Execution Location: Offshore (India) Experience: 5+ years in data engineering with cloud platforms Role Summary: Execute priority workload migrations across all 3 tracks (L&S, Refactor, New Build) ‿ building production grade pipelines on Databricks and Snowflake, running dual run validations, and remediating technical debt across schemas, naming, and lineage. Key Responsibilities: Execute automated Lift & Shift migrations ‿ schema mapping, Iceberg table creation, pipeline generation for Bronze/Silver tables Develop and refactor ETL/ELT pipelines ‿ decompose monolithic PSQL/Glue jobs into modular PySpark/Snowpark patterns Review, validate, and refine AI assisted code conversions (PySpark/Snowpark); label AI generated lines in code comments per EchoStar Addendum Configure and execute dual run validation scripts ‿ row count, hash based integrity, business rule comparisons Rationalize schemas, naming conventions, and lineage for AI Factory consumption patterns and Unity Catalog / Horizon Catalog governance Develop and deploy Airflow DAGs replacing Control M orchestration Produce before/after metrics (complexity, runtime, cost) for Technical Debt Remediation Log Create operations runbooks, SOPs, and procedural guides for each migrated workload Support EchoStar FTE shadowing and reverse shadowing during KT phases Must Have Skills: Strong proficiency in SQL, Python, PySpark, Apache Spark Hands on experience with Databricks (Delta, DLT, Autoloader, Unity Catalog) and/or Snowflake (Snowpark, Snowpipe, Streams, Tasks) Experience with AWS services ‿ S3, Redshift, Athena, Glue, EMR, Lambda Hands on with Apache Airflow DAG development Experience with GitLab for version control and CI/CD Strong SQL performance tuning and optimization skills Experience with data quality validation and reconciliation Nice to Have: Experience migrating from Redshift/Athena to Databricks or Snowflake Familiarity with Control M and Control M ‿ Airflow migration Experience with Qlik Replication / AWS DMS Shell scripting (Linux/Unix) AWS or Databricks certifications

Job Responsibility

  • Execute automated Lift & Shift migrations ‿ schema mapping, Iceberg table creation, pipeline generation for Bronze/Silver tables
  • Develop and refactor ETL/ELT pipelines ‿ decompose monolithic PSQL/Glue jobs into modular PySpark/Snowpark patterns
  • Review, validate, and refine AI assisted code conversions (PySpark/Snowpark)
  • label AI generated lines in code comments per EchoStar Addendum
  • Configure and execute dual run validation scripts ‿ row count, hash based integrity, business rule comparisons
  • Rationalize schemas, naming conventions, and lineage for AI Factory consumption patterns and Unity Catalog / Horizon Catalog governance
  • Develop and deploy Airflow DAGs replacing Control M orchestration
  • Produce before/after metrics (complexity, runtime, cost) for Technical Debt Remediation Log
  • Create operations runbooks, SOPs, and procedural guides for each migrated workload
  • Support EchoStar FTE shadowing and reverse shadowing during KT phases

Requirements

  • Strong proficiency in SQL, Python, PySpark, Apache Spark
  • Hands on experience with Databricks (Delta, DLT, Autoloader, Unity Catalog) and/or Snowflake (Snowpark, Snowpipe, Streams, Tasks)
  • Experience with AWS services ‿ S3, Redshift, Athena, Glue, EMR, Lambda
  • Hands on with Apache Airflow DAG development
  • Experience with GitLab for version control and CI/CD
  • Strong SQL performance tuning and optimization skills
  • Experience with data quality validation and reconciliation
  • A Bachelor’s or Higher Degree is the minimum entry required for the position

Nice to have

  • Experience migrating from Redshift/Athena to Databricks or Snowflake
  • Familiarity with Control M and Control M ‿ Airflow migration
  • Experience with Qlik Replication / AWS DMS
  • Shell scripting (Linux/Unix)
  • AWS or Databricks certifications
Tech Mahindra - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Azure Data Engineer

8 matching positions

Senior Azure Data Engineer

Parexel is looking for changemakers who are driven by curiosity, passion, and op...
Location
Location
Canada , Remote
Salary
Salary:
Not provided
parexel.com Logo
Parexel
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 7 years of experience in software development
  • Minimum of 2 years of experience in cloud computing
  • Proficiency with Cloud Platform, Azure, DataBricks, Azure Data Factory, Azure Dev Ops, and SQL
  • Python and Power BI are preferred
  • Bachelor of Science in Computer Science, Engineering, or related technical field is required
  • Solid knowledge of SDLC and medallion architecture
  • Excellent problem-solving and analytical skills
  • Impeccable communication skills
Job Responsibility
Job Responsibility
  • Using Microsoft Azure data PaaS services, design, build, modify, and support data pipelines leveraging DataBricks and PowerBI in a medallion architecture setting
  • Create prototypes to validate proposed ideas and solicit input from stakeholders
  • Collaborate with Team Leads to define/clarify business requirements, estimate development costs, and finalize work plans
  • Run unit and integration tests on all created code – Create and run unit and integration tests throughout the development lifecycle
What we offer
What we offer
  • Health, Vision & Dental Insurance
  • Tuition Reimbursement
  • Vacation/Holiday/Sick Time
  • Flexible Spending & Health Savings Accounts
  • Work/Life Balance
  • 401(k) with Company match
  • Pet Insurance
  • Opportunity to work on innovative projects at the forefront of the industry
  • Collaborative and inclusive work environment that values your expertise
  • Professional advancement and development opportunities
  • Fulltime
Read More
Arrow Right

Senior Azure Data Engineer

Seeking a Lead AI DevOps Engineer to oversee design and delivery of advanced AI/...
Location
Location
Poland
Salary
Salary:
Not provided
lingarogroup.com Logo
Lingaro
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 6 years of professional experience in the Data & Analytics area
  • 1+ years of experience (or acting as) in the Senior Consultant or above role with a strong focus on data solutions build in Azure and Databricks/Synapse/(MS Fabric is nice to have)
  • Proven experience in Azure cloud-based infrastructure, Databricks and one of SQL implementation (e.g., Oracle, T-SQL, MySQL, etc.)
  • Proficiency in programming languages such as SQL, Python, PySpark is essential (R or Scala nice to have)
  • Very good level of communication including ability to convey information clearly and specifically to co-workers and business stakeholders
  • Working experience in the agile methodologies – supporting tools (JIRA, Azure DevOps)
  • Experience in leading and managing a team of data engineers, providing guidance, mentorship, and technical support
  • Knowledge of data management principles and best practices, including data governance, data quality, and data integration
  • Good project management skills, with the ability to prioritize tasks, manage timelines, and deliver high-quality results within designated deadlines
  • Excellent problem-solving and analytical skills, with the ability to identify and resolve complex data engineering issues
Job Responsibility
Job Responsibility
  • Act as a senior member of the Data Science & AI Competency Center, AI Engineering team, guiding delivery and coordinating workstreams
  • Develop and execute a cloud data strategy aligned with organizational goals
  • Lead data integration efforts, including ETL processes, to ensure seamless data flow
  • Implement security measures and compliance standards in cloud environments
  • Continuously monitor and optimize data solutions for cost-efficiency
  • Establish and enforce data governance and quality standards
  • Leverage Azure services, as well as tools like dbt and Databricks, for efficient data pipelines and analytics solutions
  • Work with cross-functional teams to understand requirements and provide data solutions
  • Maintain comprehensive documentation for data architecture and solutions
  • Mentor junior team members in cloud data architecture best practices
What we offer
What we offer
  • Stable employment
  • “Office as an option” model
  • Workation
  • Great Place to Work® certified employer
  • Flexibility regarding working hours and your preferred form of contract
  • Comprehensive online onboarding program with a “Buddy” from day 1
  • Cooperation with top-tier engineers and experts
  • Unlimited access to the Udemy learning platform from day 1
  • Certificate training programs
  • Upskilling support
Read More
Arrow Right

Senior Azure Data Engineer

The Department of Customer Service is seeking a highly motivated Senior Data Eng...
Location
Location
Australia , Sydney
Salary
Salary:
Not provided
darumatic.com Logo
Darumatic
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum Bachelor’s degree in Computer Science, Software Engineering, or Information Systems
  • 5+ years of professional experience as a Data Engineer
  • Hands-on experience building scalable data pipelines and ELT/ETL workflows specifically using Microsoft Fabric to optimise datasets for PowerBI
  • Strong proficiency in SQL and at least one other programming language commonly used for data engineering (e.g., Python, Java, Scala)
  • Proven background in documentation, security, and compliance, specifically with experience in Microsoft Purview and Azure Key Vault
  • Advanced skills in Excel, including Power Query, VBA automation, and connectors
  • A creative problem solver who can think critically to resolve complex data challenges
  • Ability to stay updated with industry trends and integrate new best practices
  • Strong ability to collaborate with non-technical stakeholders and technical teams alike
Job Responsibility
Job Responsibility
  • Drive the end-to-end data engineering strategy using Microsoft technologies
  • Build, maintain, and scale data pipelines and platforms to ensure efficient data flow
  • Design and execute ELT/ETL workflows to transform raw data from disparate systems into structured, high-quality datasets optimised for PowerBI, reporting, and machine learning
  • Work cross-functionally with Data Analysts and Business Analysts to understand database requirements, partitioning strategies, and business logic
  • Implement strong data governance and compliance measures
  • Stay abreast of industry trends and best practices to drive continuous improvement initiatives within the data engineering function
  • Utilise Excel, Power Query, and VBA to handle automation tasks and connectors where required
Read More
Arrow Right

Senior Azure Data Engineer

We are seeking an experienced Azure Data Engineer with Databricks expertise to j...
Location
Location
Poland , Warszawa
Salary
Salary:
150.00 - 180.00 PLN / Hour
cyclad.pl Logo
Cyclad Sp. z o.o.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 3+years' hands-on experience with Azure Data Factory and Databricks (modern features, including Unity Catalog and 2025 capabilities)
  • At least 5+ years' experience in data engineering or backend/full-stack software development
  • Solid software engineering background: writing unit tests in Python, proficient with Git, and CI/CD pipelines
  • Strong SQL skills and experience structuring and modelling data in both relational and non-relational formats
  • Experience with data transformation tools, Spark, and real-time analytics solutions on Azure
  • Strong verbal and written communication skills in English
Job Responsibility
Job Responsibility
  • Design, build, and deploy scalable data pipelines using Azure Databricks and the Azure Analytics stack
  • Curate structured, semi-structured, and unstructured data by creating efficient, cost-effective, and scalable pipelines
  • Create and maintain robust data pipeline architecture ensuring data quality, reliability, and scalability
  • Assemble and manage large, complex datasets to meet functional and non-functional business requirements
  • Identify, design, and implement process improvements, including automation and optimization of data delivery
  • Work with real-time and streaming analytics solutions where applicable
  • Collaborate closely with Product Owners, Scrum Masters, architects, and data leadership to ensure timely, high-quality delivery
  • Align with Data Engineering chapter standards, processes, and best practices
  • Apply solid software engineering practices, including unit testing, CI/CD, and version control
  • Troubleshoot complex data-related issues and perform root cause analysis
What we offer
What we offer
  • Private medical care with dental care (covering 70% of costs) + rehabilitation package.
  • family package option possible
  • Multisport card (also for an accompanying person)
  • Life insurance
  • Flexibility and international environment
  • Fulltime
Read More
Arrow Right

Software engineer 2 / Senior Software engineer - Azure Data

Microsoft's Azure Data engineering team is leading the transformation of analyti...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 3+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience
  • Experience with the Azure stack including Storage, Compute, Networking, Fabric, Purview, Synapse, AKS, DevOps, Data Factory, or Power BI
  • Experience with big data technologies such as Spark, Kafka, Hadoop, or HBase
  • Experience building data lake or data engineering products, tools, or pipelines
  • Familiarity with container-based architectures (Docker, Kubernetes)
  • Ability to debug complex distributed systems on Linux and/or Windows platforms
Job Responsibility
Job Responsibility
  • Write extensible, maintainable code in C#, Java, Scala, or Python for Fabric Materialized Lake View services and HDInsight components
  • Use AI tools and coding best practices across the development lifecycle
  • Design data refresh, scheduling, and query optimisation features with minimal supervision
  • Review code from teammates for correctness, test coverage, security risks, and adherence to team standards
  • Coach junior engineers through code reviews
  • Debug complex issues in distributed systems running on Azure, Linux, and Windows
  • Run live site operations on a rotational, on-call basis
  • Integrate logging and instrumentation to gather telemetry on system health, performance, reliability, and security
  • Work with product managers, technical leads, and partners across geographies to define customer requirements for Materialized Lake View features
  • Fulltime
Read More
Arrow Right

Senior Data Engineer (Azure)

Location
Location
India , Bangalore
Salary
Salary:
Not provided
mercedes-benz.com Logo
Mercedes-Benz AG
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6 to 9 years of professional experience in data engineering or related roles
  • Proven hands-on experience with Azure Databricks and cloud-based data pipeline development
  • Strong knowledge of ETL/ELT tools and processes, preferably within Azure ecosystems
  • Expertise in data modeling (star/snowflake schema, normalization, dimensional modeling)
  • Advanced programming skills in PySpark, Python, and SQL
  • Experience working with Azure SQL Data Warehouse, Azure Synapse, and Azure Data Factory
  • Understanding of data governance, data quality, and security best practices
  • Familiarity with Power BI for report and dashboard development (preferred)
  • Excellent analytical, problem-solving, and communication skills
  • Ability to work independently and in cross-functional teams in agile environments
Job Responsibility
Job Responsibility
  • Design, develop, and maintain robust, scalable, and efficient data pipelines and ETL processes on Azure cloud platform
  • Build and optimize data workflows using Azure Data Factory, Databricks, Azure Synapse Analytics, and related tools
  • Implement data ingestion from multiple sources (databases, APIs, event streams) ensuring data quality and integrity
  • Collaborate with cross-functional teams to understand data requirements and deliver timely solutions
  • Develop and maintain data models and data warehouse solutions optimized for performance and scalability
  • Monitor and troubleshoot data pipeline failures, ensuring high availability and reliability
  • Ensure data security, compliance, and governance in line with organizational policies
What we offer
What we offer
  • Mitarbeiterrabatte möglich
  • Gesundheitsmaßnahmen
  • Mitarbeiterhandy möglich
  • Essenszulagen
  • Betriebliche Altersversorgung
  • Hybrides Arbeiten möglich
  • Mobilitätsangebote
  • Mitarbeiter Events
  • Coaching
  • Flexible Arbeitszeit möglich
  • Fulltime
Read More
Arrow Right

Senior Azure Data Engineer with Fabric

Senior Azure Data Engineer with Microsoft Fabric Experience
Location
Location
United States , Newark
Salary
Salary:
130000.00 USD / Year
realign-llc.com Logo
Realign
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 10+ years experience in modern data engineering/data warehousing/data lakes technologies on cloud platforms like Azure, Microsoft Fabric etc.
  • Azure experience is preferred over other cloud platforms
  • Minimum 10 + years of proven experience with SQL, schema design and dimensional data modelling
  • Solid knowledge of data warehouse best practices, development standards and methodologies
  • Experience with ETL/ELT tools like ADF, Azure, MS Fabric etc., and data warehousing technologies like Azure Synapse, Azure SQL, ADLS etc.
  • Strong experience with big data tools(Microsoft Fabric, Databricks , Spark etc..) and programming skills in PySpark and Spark SQL
  • Be an independent self-learner with a let s get this done approach and ability to work in Fast paced and Dynamic environment
  • Knowledge in Microsoft Fabric Data Platform
  • Expertise in designing application solution on Azure Data Platform for complex business needs
  • Serves as key technical expert engaged on designing, architecting, deploying and maintaining cloud solutions, Data Management & Analytics
Job Responsibility
Job Responsibility
  • Build and maintain new and existing applications in preparation for a large-scale architectural migration within an Agile function
  • Align with the Product Owner and Scrum Master in assessing business needs and transforming them into scalable applications
  • Build and maintain code to manage data received from heterogenous data formats including web- based sources, internal/external databases, flat files, heterogenous data formats (binary, ASCII)
  • Help build new enterprise Datawarehouse and maintain the existing one
  • Design and support effective storage and retrieval of very large internal and external data set and be forward think about the convergence strategy with our Azure cloud migration
  • Assess the impact of scaling up and scaling out and ensure sustained data management and data delivery performance
  • Fulltime
Read More
Arrow Right

Azure Data Engineer- Senior Consultant

Location
Location
Salary
Salary:
Not provided
lingarogroup.com Logo
Lingaro
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A Bachelor's or master's degree in computer science, Information Systems, or a related field is typically required. Additional certifications in cloud are advantageous
  • Minimum of 9+ years of experience in data engineering or a related field
  • Strong technical skills in data engineering, including proficiency in programming languages such as Python, SQL, Pyspark
  • Familiarity with Azure cloud platform viz. Azure Databricks, Data Factory, Data Lake etc., and experience in implementing data solutions in a cloud environment
  • Expertise in working with various data tools and technologies, such as ETL frameworks, data pipelines, and data warehousing solutions
  • In-depth knowledge of data management principles and best practices, including data governance, data quality, and data integration
  • Excellent problem-solving and analytical skills, with the ability to identify and resolve complex data engineering issues
  • Knowledge of data security and privacy regulations, and the ability to ensure compliance within data engineering projects
  • Excellent communication and interpersonal skills, with the ability to effectively collaborate with cross-functional teams, stakeholders, and senior management
  • Continuous learning mindset, staying updated with the latest advancements and trends in data engineering and related technologies
Job Responsibility
Job Responsibility
  • Provide technical expertise and direction in data engineering, guiding the team in selecting appropriate tools, technologies, and methodologies
  • Stay updated with the latest advancements in data engineering and ensure the team follows best practices and industry standards
  • Collaborate with stakeholders to understand project requirements, define scope, and create project plans
  • Support project managers to ensure that projects are executed effectively, meeting timelines, budgets, and quality standards
  • Monitor progress, identify risks, and implement mitigation strategies
  • Oversee the design and architecture of data solutions, collaborating with data architects and other stakeholders
  • Ensure data solutions are scalable, efficient, and aligned with business requirements
  • Provide guidance in areas such as data modeling, database design, and data integration
  • Align coding standards, conduct code reviews to ensure proper code quality level
  • Identify and introduce quality assurance processes for data pipelines and workflows
What we offer
What we offer
  • Stable employment
  • 100% remote
  • Flexibility regarding working hours
  • Comprehensive online onboarding program with a “Buddy” from day 1
  • Cooperation with top-tier engineers and experts
  • Unlimited access to the Udemy learning platform from day 1
  • Certificate training programs
  • Upskilling support
  • Grow as we grow as a company
  • A diverse, inclusive, and values-driven community
  • Fulltime
Read More
Arrow Right