CrawlJobs Logo
Tech Mahindra Logo Tech Mahindra · IT - Software Development

Advanced Data Engineer

Canada, Toronto 100000.00 - 120000.00 USD / Year · Job Posted June 09, 2026
Apply Position
Job Link Share

Job Description

We are seeking an Advanced Data Engineer with strong expertise in the Databricks ecosystem to join our data engineering team. The ideal candidate will be responsible for designing, developing, and optimizing robust data pipelines and frameworks that support data analytics, machine learning, and reporting initiatives. You will play a key role in ensuring data governance, observability, and automation within a modern data stack. In addition, the role requires strong skills in SQL, DBX (Databricks), PySpark, Data Engineering fundamentals, and experience with workflow orchestration tools such as Apache Airflow.

Job Responsibility

  • Understand, analyze, and contribute to the current Databricks architecture and design principles, ensuring scalability and performance
  • Develop and maintain efficient data processing scripts using Python and PySpark, ensuring clean, reusable, and scalable code
  • Demonstrate a deep understanding of datasets, including structure, lineage, semantics, and business context
  • Use GitHub for version control and collaborate effectively using GitHub Actions for automating workflows and CI/CD pipelines
  • Configure and maintain CI/CD pipelines in a DevOps environment for seamless code integration and deployment
  • Leverage AI coding assistants like GitHub Copilot and Databricks Assistant to improve development efficiency and code quality
  • Collaborate with cross-functional teams including data scientists, analysts, and platform engineers
  • Utilize advanced SQL for data transformation, analysis, and troubleshooting across large-scale datasets
  • Apply strong data engineering principles to design, optimize, and maintain scalable ETL/ELT processes
  • Build and manage data workflows using Apache Airflow or similar orchestration tools to ensure reliable automation and scheduling
  • Work extensively within the DBX (Databricks) environment to develop scalable pipelines and enforce best practices across the platform

Requirements

  • 5+ years of experience in data engineering or related roles
  • Proficient in Python and PySpark, with a strong foundation in distributed data processing
  • Hands-on experience working with Databricks (DBX), including workspace administration and Unity Catalog integration
  • Strong understanding of data security and governance best practices
  • Proficiency in SQL, including complex queries, optimization, and performance tuning
  • Experience with monitoring tools such as Datadog for data system observability
  • Proficiency in Git/GitHub, including pull requests, branching strategies, and GitHub Actions
  • Experience with DevOps practices related to CI/CD, especially in data pipeline deployments
  • Familiarity with AI-powered coding tools such as GitHub Copilot and Databricks Assistant
  • Strong problem-solving skills and ability to work in a fast-paced, collaborative environment
  • Experience in workflow orchestration, preferably with Apache Airflow

Nice to have

  • Databricks or Azure certifications are a plus
  • Experience in cloud platforms (Azure) in a data engineering context
  • Familiarity with modern data stack tools and frameworks
  • Excellent communication and documentation skills

What we offer

  • medical
  • vision
  • dental
  • life
  • disability insurance
  • paid time off (including holidays, parental leave, and sick leave, as required by law)
Tech Mahindra - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Advanced Data Engineer

8 matching positions

Advanced Packaging Data Analytics Engineer

We are seeking an experienced and motivated engineer in the semiconductor and ad...
Location
Location
Taiwan , Hsinchu
Salary
Salary:
Not provided
amd.com Logo
AMD
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's in Electrical Engineering, Chemical Engineering, Mechanical Engineering, Software Engineering, Data Analytics, Computer Science or similar
  • In-depth experience with challenges and yield analysis in advanced semiconductor packaging, including 2.5D and 3D heterogeneous integration
  • Expertise in analytics systems, data management, and software-driven solutions
  • Proven track record of collaboration with manufacturing partners on yield improvement
  • Python programming and software engineering experience in web app development
  • Strong problem-solving skills and the ability to manage the complexities of heterogeneous integration
  • Familiarity with electronic systems and solution implementation in advanced manufacturing
  • Experience influencing analytics software vendors roadmap
Job Responsibility
Job Responsibility
  • Develop, deploy, and maintain advanced analytics frameworks and systems designed for yield analysis and variability reduction in heterogeneous integration and packaging
  • Work closely with manufacturing partners to identify key yield issues and deliver actionable analytics solutions
  • Design and implement tools and methodologies to manage and analyze advanced packaging yield data
  • Act as a technical liaison between manufacturing partners and internal development teams to ensure that systems are functional, efficient, and aligned with strategic goals
  • Collaborate with cross-functional teams to improve processes, address gaps, and enhance the robustness of advanced analytics solutions
  • Promote innovation by integrating cutting-edge methodologies and solutions for enhanced yield analytics
Read More
Arrow Right

Senior Data Engineer – Data Engineering & AI Platforms

We are looking for a highly skilled Senior Data Engineer (L2) who can design, bu...
Location
Location
India , Chennai, Madurai, Coimbatore
Salary
Salary:
Not provided
optisolbusiness.com Logo
OptiSol Business Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on expertise in cloud ecosystems (Azure / AWS / GCP)
  • Excellent Python programming skills with data engineering libraries and frameworks
  • Advanced SQL capabilities including window functions, CTEs, and performance tuning
  • Solid understanding of distributed processing using Spark/PySpark
  • Experience designing and implementing scalable ETL/ELT workflows
  • Good understanding of data modeling concepts (dimensional, star, snowflake)
  • Familiarity with GenAI/LLM-based integration for data workflows
  • Experience working with Git, CI/CD, and Agile delivery frameworks
  • Strong communication skills for interacting with clients, stakeholders, and internal teams
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ETL/ELT pipelines across cloud and big data platforms
  • Contribute to architectural discussions by translating business needs into data solutions spanning ingestion, transformation, and consumption layers
  • Work closely with solutioning and pre-sales teams for technical evaluations and client-facing discussions
  • Lead squads of L0/L1 engineers—ensuring delivery quality, mentoring, and guiding career growth
  • Develop cloud-native data engineering solutions using Python, SQL, PySpark, and modern data frameworks
  • Ensure data reliability, performance, and maintainability across the pipeline lifecycle—from development to deployment
  • Support long-term ODC/T&M projects by demonstrating expertise during technical discussions and interviews
  • Integrate emerging GenAI tools where applicable to enhance data enrichment, automation, and transformations
What we offer
What we offer
  • Opportunity to work at the intersection of Data Engineering, Cloud, and Generative AI
  • Hands-on exposure to modern data stacks and emerging AI technologies
  • Collaboration with experts across Data, AI/ML, and cloud practices
  • Access to structured learning, certifications, and leadership mentoring
  • Competitive compensation with fast-track career growth and visibility
  • Fulltime
Read More
Arrow Right

Senior Data Engineer, Big Data

This role is essential for designing and developing data architectures across on...
Location
Location
United States , New York; Philadelphia
Salary
Salary:
105100.00 - 189600.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree plus 5 years of related work experience OR Advanced degree with 3 years of related experience
  • 4-7 years Developing cloud solutions using data series
  • experience with cloud platforms (Amazon Web Services, Azure, or Google Cloud)
  • 4-7 years Hands-on development using and migrating data to cloud platforms
  • 4-7 years Experience in SQL, NoSQL, and/or relational database design and development
  • 4-7 years Advanced knowledge and experience in building complex data pipelines with Python, Experience in languages such as SQL, DAX Python, Java, Scala, and/or Go
  • At least 18 years of age
  • Legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Develop data engineering solutions that enable data pipelines, visualization, and analytical tools to support business requirements
  • Design and develop data architectures across on-premise, cloud, and hybrid platforms to ensure scalable data infrastructure
  • Perform data wrangling, exploration, and discovery of heterogeneous data to generate new business insights
  • Contribute to team knowledge sharing and drive the advancement of new data engineering capabilities
  • Mentor team members to build and enhance their data engineering skillsets and professional growth
  • Assist management in project definition, including estimating, planning, and scoping work to meet objectives
  • Also responsible for other duties/projects as assigned by business management as needed
What we offer
What we offer
  • Medical, dental and vision insurance
  • Flexible spending account
  • 401(k)
  • Employee stock grants
  • Employee stock purchase plan
  • Paid time off
  • Up to 12 paid holidays
  • Paid parental and family leave
  • Family building benefits
  • Back-up care
  • Fulltime
Read More
Arrow Right

Senior Data Engineer, Big Data

This role is essential for designing and developing data architectures across on...
Location
Location
United States , New York; Philadelphia
Salary
Salary:
105100.00 - 189600.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree plus 5 years of related work experience OR Advanced degree with 3 years of related experience
  • 4-7 years Developing cloud solutions using data series
  • experience with cloud platforms (Amazon Web Services, Azure, or Google Cloud)
  • 4-7 years Hands-on development using and migrating data to cloud platforms
  • 4-7 years Experience in SQL, NoSQL, and/or relational database design and development
  • 4-7 years Advanced knowledge and experience in building complex data pipelines with Python, Experience in languages such as SQL, DAX Python, Java, Scala, and/or Go
  • At least 18 years of age
  • Legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Develop data engineering solutions that enable data pipelines, visualization, and analytical tools to support business requirements
  • Design and develop data architectures across on-premise, cloud, and hybrid platforms to ensure scalable data infrastructure
  • Perform data wrangling, exploration, and discovery of heterogeneous data to generate new business insights
  • Contribute to team knowledge sharing and drive the advancement of new data engineering capabilities
  • Mentor team members to build and enhance their data engineering skillsets and professional growth
  • Assist management in project definition, including estimating, planning, and scoping work to meet objectives
  • Also responsible for other duties/projects as assigned by business management as needed
What we offer
What we offer
  • annual stock grant
  • employee stock purchase plan
  • 401(k)
  • free, year-round money coaches
  • medical insurance
  • dental insurance
  • vision insurance
  • flexible spending account
  • paid time off
  • up to 12 paid holidays
  • Fulltime
Read More
Arrow Right

Data Engineer (Big Data, Python, Databricks) - Assistant Vice President

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Chennai, Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of relevant handson experience in Big Data technologies like Cloudera, Python, HQL, Java/PySpark
  • Knowledge on Machine Learning, AI would be added advantage
  • Experience in systems analysis, data analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right
New

Data Engineer Lead (OT Data)

Data Engineer (OT Data) (Category - Engineer) Sector: Oil and Gas Location: Doha...
Location
Location
Qatar , Doha
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's in engineering, Information Systems, or a related quantitative field
  • 5+ years of proven experience in a data engineering role
  • Experience within oil and gas industry is highly preferred
  • Demonstrable experience building and operationalizing large-scale data pipelines and applications
Job Responsibility
Job Responsibility
  • Architect & Build Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines
  • Integrate Diverse Data Sources: Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians (like OSIsoft PI, Aspen InfoPlus.21), DCS, PLC, and IoT sensors
  • Cloud Data Platform Development: Implement and manage data solutions on the Microsoft Azure cloud platform, Leveraging services like Azure IoT Hub, Azure Event Hubs, and Azure Stream Analytics for real-time ingestion and processing of operational technology (OT) data
  • Data Modelling & Warehousing: Design and implement data models optimized for time-series data from industrial assets, supporting operational dashboards and real-time analytics
  • Enable Advanced AI: Build the data infrastructure to support AI/ML models for predictive maintenance, operational anomaly detection, and process optimization using real-time OT data
  • Champion Master Data Management (MDM): Design and implement MDM strategies and solutions to create a single, authoritative source of truth for critical data domains such as wells, equipment, and assets, ensuring data consistency across the enterprise
  • Ensure Data Quality & Governance: Implement robust data quality checks, validation rules, and monitoring to ensure the accuracy, consistency, and reliability of our data. Adhere to and help shape our data governance policies
  • Embrace Industry Standards: Champion and implement industry-specific data standards and models, such as the OSDU™ Data Platform, to ensure interoperability and a unified data view across the upstream lifecycle
  • Collaborate & Innovate: Work closely with a cross-functional team of geoscientists, drilling engineers, data scientists, and business analysts to understand their data needs and deliver effective solutions
  • Automate & Optimize: Identify opportunities for process automation and infrastructure optimization to improve data delivery, scalability, and cost-effectiveness
  • Fulltime
Read More
Arrow Right

Forward Deployed Engineer - Data Migration & Data Consolidation Platforms

As a Forward Deployed Engineer (FDE) for Data Migration & Data Consolidation Pla...
Location
Location
United States
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7-10+ years of progressive experience in enterprise data engineering, data migration, or large-scale system integration roles within complex, multi-platform environments
  • 3-5+ years directly leading end-to-end data migration or multi-system consolidation programs for Global Enterprises and Industry Leaders, with full ownership of technical delivery and client outcomes
  • Demonstrated client-facing experience serving as a trusted technical advisor to C-level executives, enterprise architecture teams, and cross-functional business stakeholders
  • Proven industry depth in at least two of the following verticals: Healthcare, Financial Services, Manufacturing, Retail, Energy & Utilities, or Public Sector
  • Hands-on migration complexity: successfully delivered programs involving at least 3+ heterogeneous source systems, 100M+ records, complex master data harmonization, and multi-phase cutover execution
  • Advanced proficiency in Python and SQL with working experience in PySpark and TypeScript/JavaScript
  • Hands-on expertise with modern ETL/ELT and data integration platforms (Informatica, Talend, Matillion, Fivetran, AWS Glue, Azure Data Factory)
  • Proven ability to build scalable, version-controlled data pipelines with error handling, incremental loading, and Change Data Capture (CDC)
  • Strong working knowledge of at least one major cloud provider (AWS, Azure, or GCP), including core infrastructure, managed data services, and security configurations
  • Experience with enterprise data warehouse and lakehouse platforms (Snowflake, Databricks, BigQuery, Redshift, Synapse Analytics, Delta Lake)
Job Responsibility
Job Responsibility
  • Migration Execution & Cloud Architecture: Lead end-to-end delivery of enterprise data migrations from corporate systems (SAP, Oracle, Epic ERP) to target cloud data platforms, including the design of cloud landing zones, data governance frameworks, and system rationalization strategies. Establish migration compliance controls, automated rollback procedures, and operational readiness gates while owning full technical accountability for 12–18+ month migration roadmaps
  • Data Pipeline Engineering & Transformation: Build production-grade data connectors to SAP (RFC, IDoc, BAPI, OData), Oracle (AQ, GoldenGate, APIs), and SQL/non-relational sources. Develop ETL/ELT pipelines with LLM-enabled transformation logic, multi-layer validation and reconciliation frameworks, and optimized throughput for datasets scaling from tens of millions to billions of records with built-in CDC and incremental loading
  • Ontology Layer Development & Schema Automation: Construct semantic ontology layers translating raw ERP structures into business-consumable objects (Customer, Order, Invoice, Product, Vendor, Asset). Deploy automated schema mapping agents for source-to-target analysis and transformation logic generation. Build unified master data models with row/column-level security, cross-system lineage tracking, and AI-ready semantic structures
  • Application & Workflow Delivery: Build operational dashboards, migration control centers, and agent-driven workflows for automated validation, exception handling, and anomaly detection using low-code platform tools. Generate TypeScript/Python SDKs for custom integrations and deliver real-time monitoring and self-service interfaces for migration progress, data quality KPIs, and compliance tracking
  • Multi-System Consolidation & Master Data Management: Lead consolidation of 5–15+ fragmented ERP instances into standardized master data models. Resolve complex entity resolution challenges including customer matching, product harmonization, and chart of accounts unification. Establish golden record frameworks, data quality scorecards, survivorship rules, and data stewardship workflows for post-migration governance
  • Client Engagement, Discovery & Modernization Advisory: Serve as primary technical advisor to C-suite and enterprise architecture stakeholders across all engagement phases. Deploy discovery agents to analyze legacy data estates, conduct assessment workshops, facilitate solution design sessions, and deliver executive briefings, go/no-go readiness assessments, and prioritized modernization roadmaps
  • Knowledge Transfer, Enablement & IP Development: Build reusable migration accelerators, playbooks, and reference architectures that scale across engagements. Lead knowledge transfer to upskill client teams for post-migration ownership and collaborate with internal product and sales engineering teams to feed field insights back into platform development and delivery methodology
  • Leadership & Executive Engagement: Operate autonomously in ambiguous, high-stakes client environments, driving outcomes with minimal oversight
  • translate deeply technical concepts into clear, business-level narratives for C-suite audiences through executive briefings and stakeholder communications
  • navigate organizational complexity, competing stakeholder priorities, and enterprise change management dynamics to maintain momentum across multi-workstream engagements
Read More
Arrow Right

Senior Data Engineer / Analytic Engineer (Microsoft Fabric)

As a Senior Data Engineer / Analytic Engineer (Microsoft Fabric), you will lead ...
Location
Location
United States
Salary
Salary:
Not provided
velvetech.com Logo
Velvetech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in data engineering, with strong expertise in Microsoft Fabric or Azure Data Factory, Azure Synapse Analytics and Azure ecosystems
  • Practical knowledge of Python, SQL, Javascript
  • Proficiency in building and maintaining Azure Blob Storage solutions
  • Experience with Power BI for data visualization and integration
  • Advanced programming skills in Python and DAX
  • Familiarity with BPMN and data governance frameworks
  • Microsoft certifications in relevant tools (e.g., PL-600, PL-400, DP-500)
  • Excellent problem-solving and communication skills
Job Responsibility
Job Responsibility
  • Architect and implement ETL workflows using Microsoft Fabric, including Data Pipelines, Dataflows, and Notebooks
  • Integrate Azure Blob Storage as the primary data staging area, ensuring seamless compatibility with Microsoft Fabric
  • Design template mapping systems with versioning to support dynamic file processing and metadata management
  • Build and optimize data processing pipelines for various source file formats and carrier templates
  • Ensure the system adheres to high availability, scalability, and performance standards
  • Collaborate with Power Apps and Power Automate developers for smooth workflow integration
  • Develop and implement comprehensive error handling and logging mechanisms
What we offer
What we offer
  • FLEXIBLE working conditions and a COOPERATIVE environment
  • Competitive salary
  • Many CHALLENGING and exciting projects with new opportunities and learning
  • GROWTH opportunities, skills and competencies improvement, and professional certification
  • In-company TRAINING (English, Software / DevOps / Project management / Design / Business)
  • Fulltime
Read More
Arrow Right