CrawlJobs Logo
Hewlett Packard Enterprise Logo Hewlett Packard Enterprise · IT - Software Development

Principal Data Engineer

India, Bengaluru · Job Posted June 04, 2026
Apply Position
Job Link Share

Job Description

Principal Data Engineer. This role has been designed as 'Onsite' with an expectation that you will primarily work from an HPE office. We are looking for a highly skilled Principal Data Engineer with strong architectural expertise to design and evolve our next-generation data platform. You will help to define the technical vision, build scalable and reliable data systems, and guide the long-term architecture that powers analytics, operational decision-making, and data-driven products across the organization. This role is both strategic and hands-on. You will evaluate modern data technologies, define engineering best practices, and lead the implementation of robust, high-performance data solutions—including the design, build, and lifecycle management of data pipelines that support batch, streaming, and near-real-time workloads.

Job Responsibility

  • Architecture & Strategy: Own a significant portion of the architecture of our data platform, ensuring scalability, performance, reliability, and security
  • Define standards and best practices for data modeling, transformation, orchestration, governance, and lifecycle management
  • Evaluate and integrate modern data technologies and frameworks that align with our long-term platform strategy
  • Collaborate with engineering and product leadership to shape the technical roadmap
  • Engineering & Delivery: Design, build, and manage scalable, resilient data pipelines for batch, streaming, and event-driven workloads
  • Develop clean, high-quality data models and schemas to support analytics, BI, operational systems, and ML workflows
  • Implement data quality, lineage, observability, and automated testing frameworks
  • Build ingestion patterns for APIs, event streams, files, and third-party data sources
  • Optimize compute, storage, and transformation layers for performance and cost efficiency
  • Leadership & Collaboration: Serve as a senior technical leader and mentor within the data engineering team
  • Lead architecture reviews, design discussions, and cross-team engineering initiatives
  • Work closely with analysts, data scientists, software engineers, and product owners to define and deliver data solutions
  • Communicate architectural decisions and trade-offs to technical and non-technical stakeholders

Requirements

  • 8+ years of experience in Data Engineering, with demonstrated architectural ownership
  • Expert-level experience with Snowflake (mandatory), including performance optimization, data modeling, security, and ecosystem components
  • Expert proficiency in SQL and strong Python skills for pipeline development and automation
  • Experience with modern orchestration tools (Airflow, Dagster, Prefect, or equivalent)
  • Strong understanding of ELT/ETL patterns, distributed processing, and data lifecycle management
  • Familiarity with streaming/event technologies (Kafka, Kinesis, Pub/Sub, etc.)
  • Experience implementing data quality, observability, and lineage solutions
  • Solid understanding of cloud infrastructure (AWS, GCP, or Azure)
  • Strong background in DataOps practices: CI/CD, testing, version control, automation
  • Proven leadership in driving architectural direction and mentoring engineering teams

Nice to have

  • Experience with data governance or metadata management tools
  • Hands-on experience with DBT, including modeling, testing, documentation, and advanced features
  • Exposure to machine learning pipelines, feature stores, or MLOps
  • Experience with Terraform, CloudFormation, or other IaC tools
  • Background designing systems for high scale, security, or regulated environments

What we offer

  • Health & Wellbeing: comprehensive suite of benefits supporting physical, financial and emotional wellbeing
  • Personal & Professional Development: programs to help reach career goals
  • Unconditional Inclusion: inclusive work environment
Hewlett Packard Enterprise - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Principal Data Engineer

8 matching positions

Principal Data Engineer

Our Client is seeking a Principal Data Engineer to roll up their sleeves and bui...
Location
Location
United States , New York
Salary
Salary:
175000.00 - 220000.00 USD / Year
solomonpage.com Logo
Solomon Page
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong foundation in software engineering principles, including SOLID design, modularity, and scalability.
  • Expert proficiency in Databricks, including Delta Lake, Unity Catalog, Delta Live Tables, MLflow, and Databricks Workflows.
  • Deep experience with Snowflake, including data modeling, performance optimization, and integration with upstream/downstream pipeline tooling.
  • Expert proficiency in Python for data pipeline and automation development.
  • Advanced SQL skills with experience optimizing complex queries and data models at scale.
  • Proven experience designing and maintaining cloud-native data pipelines on AWS (e.g., MWAA/Airflow, Lambda, ECS, SQS, Glue, S3, Redshift, etc.).
  • Experience implementing and managing Terraform or similar IaC frameworks.
  • Strong understanding of lakehouse architecture patterns, data ingestion, transformation, and orchestration, including familiarity with ML/AI pipeline integration patterns.
  • Familiarity with CI/CD pipelines, automated testing, and modern DevOps practices.
  • 8+ years of experience in data engineering or backend development, with a focus on scalable data solutions.
Job Responsibility
Job Responsibility
  • Build and maintain end-to-end data pipelines — from raw ingestion through transformation and delivery — across diverse data sources (APIs, web data, internal feeds, etc.).
  • Hands-on development of scalable, production-grade pipelines within Databricks, including Delta Lake table management, Workflows, and cluster optimization.
  • Build and maintain data models, schemas, and transformation logic in Snowflake, optimizing for performance and reliability.
  • Develop and manage Databricks environments including Unity Catalog, Delta Live Tables, and integration patterns that support both internal data consumers and external sharing use cases.
  • Build and manage orchestration workflows using AWS services (MWAA/Airflow, Lambda, ECS, SQS, MSK) and Databricks-native orchestration where appropriate.
  • Implement and maintain infrastructure as code (IaC) using Terraform, ensuring reproducibility and compliance with cloud standards.
  • Establish and enforce best practices in data modeling, schema design, and ETL/ELT processes for high-volume structured and semi-structured data across Snowflake and Databricks.
  • Ensure data quality, lineage, and observability through automated testing, monitoring, and alerting across all pipeline layers.
  • Collaborate closely with technology leadership to align data platform development with business strategy and product goals.
  • Stay at the forefront of industry trends in data engineering, lakehouse architecture, and cloud-native data platforms.
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

Capstone IT is seeking a Principal Data Engineer to lead enterprise-scale data e...
Location
Location
United States , Springfield
Salary
Salary:
Not provided
capstonec.com Logo
Capstone IT Staffing
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Snowflake
  • Google Cloud Platform (GCP)
  • BigQuery
  • ELT/ETL Pipeline Development
  • Kafka and/or NiFi
  • Cloud Data Architecture
  • SQL & Data Modeling
  • Real-Time Data Streaming
  • Business Intelligence & Reporting Tools
  • Executive Stakeholder Communication
Job Responsibility
Job Responsibility
  • Lead enterprise data engineering initiatives across eCommerce, customer analytics, and omnichannel data environments
  • Design, develop, and optimize scalable ELT/ETL pipelines supporting enterprise analytics and reporting
  • Build and maintain high-performance Snowflake and BigQuery data architectures
  • Develop and support real-time and batch data processing solutions using cloud-native technologies and streaming platforms
  • Partner with Product Owners, Business Analysts, marketing teams, and executive leadership to translate business requirements into scalable technical solutions
  • Deliver actionable analytics, dashboards, and executive-level reporting insights
  • Ensure data quality through governance, cleansing, transformation, integration, and validation processes
  • Drive architectural decisions related to enterprise data platforms, cloud modernization, and streaming ecosystems
  • Support BI and analytics environments utilizing tools such as Domo, Sigma, Tableau, and related visualization technologies
  • Manage offshore engineering resources and vendor relationships, including deliverables, budgeting, performance management, and capacity planning
What we offer
What we offer
  • Medical Insurance
  • Dental Insurance
  • Vision Insurance
  • Company Matching 401(k)
  • Paid Relocation Assistance
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

We are on the lookout for a Principal Data Engineer to help define and lead the ...
Location
Location
South Africa; United Kingdom; Poland
Salary
Salary:
Not provided
dotdigital.com Logo
Dotdigital
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Extensive experience delivering python-based projects in the data engineering space
  • Extensive experience working with SQL and NoSQL database technologies
  • Proven experience with modern data warehousing and large-scale data processing tools
  • Hands on experience with data orchestration tools
  • Experience using cloud environments (e.g. Azure, AWS, GCP) to process, store and surface large scale data
  • Experience using Kafka or similar event-based architectures
  • Strong grasp of data architecture and data modelling principles for both OLAP and OLTP workloads
  • Capable in the wider software development lifecycle in terms of agile ways of working and continuous integration/deployment of data solutions
  • Experience as a lead or Principal Engineer on large-scale data initiative or product builds
  • Demonstrated ability to architect data systems and data structures for high volume, high throughput systems
Job Responsibility
Job Responsibility
  • Lead the design and implementation of scalable, secure and resilient data systems across streaming, batch and real-time use cases
  • Architect data pipelines, model and storage solutions that power analytical and product use cases
  • using primarily Python and SQL via orchestration tooling that run workloads in the cloud
  • Leverage AI to automate both data processing and engineering processes
  • Assure and drive best practices relating to data infrastructure, governance, security and observability
  • Work with technologists across multiple teams to deliver coherent features and data outcomes
  • Support the data team to help adopt data engineering principles
  • Identify, validate and promote new tools and technologies that improve the performance and stability of data services
What we offer
What we offer
  • Parental leave
  • Medical benefits
  • Paid sick leave
  • Dotdigital day
  • Share reward
  • Wellbeing reward
  • Wellbeing Days
  • Loyalty reward
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

We are looking for a hands-on Principal Data Engineer who is passionate about so...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 15+ years’ experience of implementing data-intensive solutions using agile methodologies
  • Proficient in one or more programming languages commonly used in data engineering such as Python, Java, or Scala
  • Multiple years of experience with software engineering best practices (unit testing, automation, design patterns, peer review, etc.)
  • Strong proficiency in working with relational databases and using SQL for data querying, transformation, and manipulation
  • Expertise in Quantexa Entity resolution product, Quantexa certifications will be preferred
  • Clear understanding of Data Structures and Object Oriented Principles
  • Experience of modelling data for analytical consumers
  • Ability to automate and streamline the build, test and deployment of data pipelines
  • Experience with Hadoop for data storage and processing is valuable, as is exposure to modern data platforms such as Snowflake and Databricks
  • Multiple years of experience architecting and building horizontally scalable, highly available, highly resilient, and low latency applications
Job Responsibility
Job Responsibility
  • Code contributing member of multiple Agile teams, working to deliver sprint goals
  • Demonstrating deep technical knowledge and expertise in software development, including programming languages, frameworks, and best practices. Providing guidance and mentorship to junior team members
  • Actively contributes to the implementation of critical features and complex technical solutions. Write clean, efficient, and maintainable code that meets the highest standards of quality
  • Collaborate with other Principal Engineers to define and evolve the overall system architecture and design
  • Provide guidance on scalable, robust, and efficient solutions that align with business requirements and industry best practices
  • Offer expert engineering guidance and support to multiple teams, helping them overcome technical challenges, make informed decisions, and deliver high-quality software solutions. Foster a culture of technical excellence and continuous improvement
  • Stay up to date with emerging technologies, tools, and industry trends. Evaluate their potential impact on the organization and provide recommendations for technology adoption and innovation
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

The Principal Data Engineer at NTT DATA is responsible for transforming data int...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree or equivalent in computer science, software engineering, information technology, or a related field
  • Extended demonstrated experience in data engineering, data mining within a fast-paced environment
  • Extended experience in building modern data analytics solutions that delivers insights from large and complex data sets with multi-terabyte scale
  • Extended demonstrated experience with architecture and design of secure, highly available and scalable systems
  • Extended demonstrated experience in automation, scripting and proven examples of successful implementation
  • Extended demonstrated experience using scripting language (Perl, bash, Shell Scripting, Python, etc.)
  • Extended demonstrated experience with big data tools like Hadoop, Cassandra, Storm etc.
  • Extended demonstrated experience in any applicable language, preferably .NET
  • Extended demonstrated experience in SAP, SQL, MySQL databases and Microsoft SQL
  • Extended demonstrated experience working with data sets and ordering data through MS Excel functions, e.g. macros, pivots
Job Responsibility
Job Responsibility
  • Leads on designing data models in a structured data format to enable analysis thereof
  • Design. and develop. scalable extract, transformation and loading (ETL) packages from the business source systems and the development of ETL routines to populate data from sources
  • Participates in the transformation of object and data models into appropriate database schemas within design constraints
  • Interprets installation standards to meet project needs and produce database components as required
  • Advises on test scenarios and be responsible for participating in thorough testing and validation to support the accuracy of data transformations
  • Accountable for running data migrations across different databases and applications, e.g. MS Dynamics, Oracle, SAP and other ERP systems
  • Works across multiple IT and business teams to define and implement data table structures and data models based on requirements
  • Accountable for analysis, and development of ETL and migration documentation
  • Works with cross functionally stakeholders to evaluate potential data requirements
  • Owns the definition and management of scoping, requirements, definition, and prioritization activities for small-scale changes and assist with more complex change initiatives
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

Microsoft’s Path team helps customers along their journey from the initial idea ...
Location
Location
United States , Redmond
Salary
Salary:
139900.00 - 274800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience.
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Design, implement, monitor, and optimize large-scale, robust data pipelines and architectures that support Microsoft Path products data analytics initiatives.
  • Develop scalable ETL/ELT workflows, ensure reliable ingestion, transformation, and storage of high-volume, high-velocity data from diverse sources with different schemas.
  • Enforce best practices for data quality, lineage, security, and compliance.
  • Collaborate with stakeholders to define trustworthy data sets and implement rigorous data validation protocols, ensuring analytics are accurate and auditable.
  • Work closely with product managers, software engineers, AI researchers, and business stakeholders to build data solutions that accomplish business goals.
  • Enable self-service analytics and empower teams by building data models, semantic layers, and tools that streamline access to trusted information.
  • Translate ambitious vision into actionable roadmaps and measurable outcomes.
  • Mentor senior data engineers and technical leads, guide complex design decisions, and foster a culture of engineering excellence, operational rigor, and continuous learning across the organization.
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

As Google Cloud's premier partner in AI, Datatonic provides world-class business...
Location
Location
Croatia , Zagreb
Salary
Salary:
Not provided
datatonic.com Logo
Datatonic
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong commercial awareness of Data Managed Services solutions, particularly with scoping and pitching to large enterprise customers including pricing, contracts and proposals
  • Strong interpersonal skills with the ability to work with clients to establish requirements in non-technical language
  • Ability to translate business requirements into plausible technical solutions for articulation to engineering teams
  • Ability to understand existing target architecture and adjust accordingly to accommodate new business requirements
  • Strong experience in monitoring and measuring platform performance and availability e.g SLO and SLI
  • Experience managing across relationships across internal and external customers, stakeholders and executives
  • Experience with Google Data Products tools (e.g., BigQuery, Dataflow, DataProc, Dataplex, Composer, Vertex, Looker, etc.)
  • Experience building and deploying solutions to Cloud (Google Cloud) including Cloud provisioning tools and management of releases through various environments into production
  • Experience and knowledge of application containerisation e.g. Docker, Kubernetes, Cloud Run, etc
  • Deep expertise with data warehousing (Kimball, Lakehouse, Data Mesh), particularly Big Query built using dbt/Dataform
Job Responsibility
Job Responsibility
  • Leadership and accountability for the technical roadmap of services
  • Implement and execute on the tooling strategy required to delivery a highly automated and scalable data managed service
  • Support the sales team with technical pre-sales enablement including build out pitch decks, architecture documents and sales collateral
  • Advise consulting teams on best practices for build of data platforms that are considerate of managed services
  • Working closely with OCTO (Office of the CTO) to ensure close alignment on existing and new standards
  • Being an advocate for and keeping up to date with new technologies, Google services, and the evolution of GenAI
  • Senior escalation point for intractable issues with customer platforms
  • Work with the Head of Managed Services on the strategic objectives of the Data Operations team
  • Contributing to development and evolution of the services portfolio
  • Coaching and mentoring junior technical staff, including responsibility for defining learning paths for engineering teams
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

We are on the lookout for a Principal Data Engineer to help define and lead the ...
Location
Location
United Kingdom
Salary
Salary:
Not provided
dotdigital.com Logo
Dotdigital
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Extensive experience delivering python-based projects in the data engineering space
  • Extensive experience working with SQL and NoSQL database technologies (e.g. SQL Server, MongoDB & Cassandra)
  • Proven experience with modern data warehousing and large-scale data processing tools (e.g. Snowflake, DBT, BiqQuery, Clickhouse)
  • Hands on experience with data orchestration tools like Airflow, Dagster or Prefect
  • Experience using cloud environments (e.g. Azure, AWS, GCP) to process, store and surface large scale data
  • Experience using Kafka or similar event-based architectures e.g. (Pub/Sub via AWS SQS, Azure EventHubs, AWS Kinesis)
  • Strong grasp of data architecture and data modelling principles for both OLAP and OLTP workloads
  • Capable in the wider software development lifecycle in terms of agile ways of working and continuous integration/deployment of data solutions
  • Experience as a lead or Principal Engineer on large-scale data initiative or product builds
  • Demonstrated ability to architect data systems and data structures for high volume, high throughput systems
Job Responsibility
Job Responsibility
  • Lead the design and implementation of scalable, secure and resilient data systems across streaming, batch and real-time use cases
  • Architect data pipelines, model and storage solutions that power analytical and product use cases
  • using primarily Python and SQL via orchestration tooling that run workloads in the cloud
  • Leverage AI to automate both data processing and engineering processes
  • Assure and drive best practices relating to data infrastructure, governance, security and observability
  • Work with technologists across multiple teams to deliver coherent features and data outcomes
  • Support the data team to help adopt data engineering principles
  • Identify, validate and promote new tools and technologies that improve the performance and stability of data services
What we offer
What we offer
  • Parental leave
  • Medical benefits
  • Paid sick leave
  • Dotdigital day
  • Share reward
  • Wellbeing reward
  • Wellbeing Days
  • Loyalty reward
  • Fulltime
Read More
Arrow Right