Data Engineer – Lead Job at NTT DATA (Bengaluru Urban)

Job Description

Data Engineer – Lead

Lead the design and implementation of scalable data pipelines and data processing frameworks in Microsoft Fabric
Define data engineering standards, development practices, naming conventions, coding guidelines, and reusable technical patterns
Lead implementation of Bronze, Silver, and Gold layers in the Medallion Architecture
Oversee ingestion, transformation, orchestration, validation, and publication of data from multiple enterprise, clinical, operational, and cloud-based sources
Guide development of Fabric Pipelines, Dataflows Gen2, Notebooks, Lakehouse tables, Warehouse objects, and curated datasets
Ensure scalability, performance, reliability, maintainability, security, monitoring, and optimization of data solutions
Define standards for data quality, reconciliation, logging, error handling, auditability, and lineage
Conduct technical design reviews, code reviews, performance reviews, and deployment readiness reviews
Mentor and guide data engineering teams across onshore/offshore locations
Collaborate with architects, platform engineers, BI teams, QA teams, AI/ML teams, functional consultants, and stakeholders
Support deployment planning, release activities, production support, and operational handover

Strong hands-on experience with Microsoft Fabric, including Lakehouse, Warehouse, OneLake, Pipelines, Dataflows Gen2, Notebooks, and Power BI integration
Expertise in ETL/ELT, data pipelines, distributed data processing, and cloud-scale data engineering
Strong SQL, Python, PySpark, and data modeling skills
Experience with Lakehouse, Warehouse, and Medallion Architecture
Understanding of Delta tables, dimensional modeling, star schema, facts, dimensions, and curated analytical datasets
Experience integrating structured, semi-structured, file-based, API-based, enterprise application, and cloud data sources
Experience with data quality, reconciliation, logging, monitoring, and error-handling frameworks
Experience leading technical teams and coordinating onshore/offshore delivery
Experience with Git, CI/CD, Azure DevOps, branching, code reviews, and release management
Good to Have: Experience with Azure Data Factory, Synapse, Databricks, ADLS Gen2, Azure SQL, Microsoft Purview, or related Azure services
Good to Have: Experience with CRO, clinical research, pharma, healthcare and clinical systems

Experience with Azure Data Factory, Synapse, Databricks, ADLS Gen2, Azure SQL, Microsoft Purview, or related Azure services
Experience with CRO, clinical research, pharma, healthcare and clinical systems