Principal Data Engineer Job at Dotdigital

Principal Data Engineer

Our Client is seeking a Principal Data Engineer to roll up their sleeves and bui...

Location

United States , New York

Salary:

175000.00 - 220000.00 USD / Year

Solomon Page

Expiration Date

Until further notice

Requirements

Strong foundation in software engineering principles, including SOLID design, modularity, and scalability.
Expert proficiency in Databricks, including Delta Lake, Unity Catalog, Delta Live Tables, MLflow, and Databricks Workflows.
Deep experience with Snowflake, including data modeling, performance optimization, and integration with upstream/downstream pipeline tooling.
Expert proficiency in Python for data pipeline and automation development.
Advanced SQL skills with experience optimizing complex queries and data models at scale.
Proven experience designing and maintaining cloud-native data pipelines on AWS (e.g., MWAA/Airflow, Lambda, ECS, SQS, Glue, S3, Redshift, etc.).
Experience implementing and managing Terraform or similar IaC frameworks.
Strong understanding of lakehouse architecture patterns, data ingestion, transformation, and orchestration, including familiarity with ML/AI pipeline integration patterns.
Familiarity with CI/CD pipelines, automated testing, and modern DevOps practices.
8+ years of experience in data engineering or backend development, with a focus on scalable data solutions.

Job Responsibility

Build and maintain end-to-end data pipelines — from raw ingestion through transformation and delivery — across diverse data sources (APIs, web data, internal feeds, etc.).
Hands-on development of scalable, production-grade pipelines within Databricks, including Delta Lake table management, Workflows, and cluster optimization.
Build and maintain data models, schemas, and transformation logic in Snowflake, optimizing for performance and reliability.
Develop and manage Databricks environments including Unity Catalog, Delta Live Tables, and integration patterns that support both internal data consumers and external sharing use cases.
Build and manage orchestration workflows using AWS services (MWAA/Airflow, Lambda, ECS, SQS, MSK) and Databricks-native orchestration where appropriate.
Implement and maintain infrastructure as code (IaC) using Terraform, ensuring reproducibility and compliance with cloud standards.
Establish and enforce best practices in data modeling, schema design, and ETL/ELT processes for high-volume structured and semi-structured data across Snowflake and Databricks.
Ensure data quality, lineage, and observability through automated testing, monitoring, and alerting across all pipeline layers.
Collaborate closely with technology leadership to align data platform development with business strategy and product goals.
Stay at the forefront of industry trends in data engineering, lakehouse architecture, and cloud-native data platforms.

Fulltime

New

Principal Data Engineer

Principal Data Engineer. This role has been designed as 'Onsite' with an expecta...

Location

India , Bengaluru

Salary:

Not provided

Hewlett Packard Enterprise

Expiration Date

Until further notice

Requirements

8+ years of experience in Data Engineering, with demonstrated architectural ownership
Expert-level experience with Snowflake (mandatory), including performance optimization, data modeling, security, and ecosystem components
Expert proficiency in SQL and strong Python skills for pipeline development and automation
Experience with modern orchestration tools (Airflow, Dagster, Prefect, or equivalent)
Strong understanding of ELT/ETL patterns, distributed processing, and data lifecycle management
Familiarity with streaming/event technologies (Kafka, Kinesis, Pub/Sub, etc.)
Experience implementing data quality, observability, and lineage solutions
Solid understanding of cloud infrastructure (AWS, GCP, or Azure)
Strong background in DataOps practices: CI/CD, testing, version control, automation
Proven leadership in driving architectural direction and mentoring engineering teams

Job Responsibility

Architecture & Strategy: Own a significant portion of the architecture of our data platform, ensuring scalability, performance, reliability, and security
Define standards and best practices for data modeling, transformation, orchestration, governance, and lifecycle management
Evaluate and integrate modern data technologies and frameworks that align with our long-term platform strategy
Collaborate with engineering and product leadership to shape the technical roadmap
Engineering & Delivery: Design, build, and manage scalable, resilient data pipelines for batch, streaming, and event-driven workloads
Develop clean, high-quality data models and schemas to support analytics, BI, operational systems, and ML workflows
Implement data quality, lineage, observability, and automated testing frameworks
Build ingestion patterns for APIs, event streams, files, and third-party data sources
Optimize compute, storage, and transformation layers for performance and cost efficiency
Leadership & Collaboration: Serve as a senior technical leader and mentor within the data engineering team

What we offer

Health & Wellbeing: comprehensive suite of benefits supporting physical, financial and emotional wellbeing
Personal & Professional Development: programs to help reach career goals
Unconditional Inclusion: inclusive work environment

Fulltime

Principal Data Engineer

Capstone IT is seeking a Principal Data Engineer to lead enterprise-scale data e...

Location

United States , Springfield

Salary:

Not provided

Capstone IT Staffing

Expiration Date

Until further notice

Requirements

Snowflake
Google Cloud Platform (GCP)
BigQuery
ELT/ETL Pipeline Development
Kafka and/or NiFi
Cloud Data Architecture
SQL & Data Modeling
Real-Time Data Streaming
Business Intelligence & Reporting Tools
Executive Stakeholder Communication

Job Responsibility

Lead enterprise data engineering initiatives across eCommerce, customer analytics, and omnichannel data environments
Design, develop, and optimize scalable ELT/ETL pipelines supporting enterprise analytics and reporting
Build and maintain high-performance Snowflake and BigQuery data architectures
Develop and support real-time and batch data processing solutions using cloud-native technologies and streaming platforms
Partner with Product Owners, Business Analysts, marketing teams, and executive leadership to translate business requirements into scalable technical solutions
Deliver actionable analytics, dashboards, and executive-level reporting insights
Ensure data quality through governance, cleansing, transformation, integration, and validation processes
Drive architectural decisions related to enterprise data platforms, cloud modernization, and streaming ecosystems
Support BI and analytics environments utilizing tools such as Domo, Sigma, Tableau, and related visualization technologies
Manage offshore engineering resources and vendor relationships, including deliverables, budgeting, performance management, and capacity planning

What we offer

Medical Insurance
Dental Insurance
Vision Insurance
Company Matching 401(k)
Paid Relocation Assistance

Fulltime

Principal Data Engineer

We are looking for a hands-on Principal Data Engineer who is passionate about so...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

15+ years’ experience of implementing data-intensive solutions using agile methodologies
Proficient in one or more programming languages commonly used in data engineering such as Python, Java, or Scala
Multiple years of experience with software engineering best practices (unit testing, automation, design patterns, peer review, etc.)
Strong proficiency in working with relational databases and using SQL for data querying, transformation, and manipulation
Expertise in Quantexa Entity resolution product, Quantexa certifications will be preferred
Clear understanding of Data Structures and Object Oriented Principles
Experience of modelling data for analytical consumers
Ability to automate and streamline the build, test and deployment of data pipelines
Experience with Hadoop for data storage and processing is valuable, as is exposure to modern data platforms such as Snowflake and Databricks
Multiple years of experience architecting and building horizontally scalable, highly available, highly resilient, and low latency applications

Job Responsibility

Code contributing member of multiple Agile teams, working to deliver sprint goals
Demonstrating deep technical knowledge and expertise in software development, including programming languages, frameworks, and best practices. Providing guidance and mentorship to junior team members
Actively contributes to the implementation of critical features and complex technical solutions. Write clean, efficient, and maintainable code that meets the highest standards of quality
Collaborate with other Principal Engineers to define and evolve the overall system architecture and design
Provide guidance on scalable, robust, and efficient solutions that align with business requirements and industry best practices
Offer expert engineering guidance and support to multiple teams, helping them overcome technical challenges, make informed decisions, and deliver high-quality software solutions. Foster a culture of technical excellence and continuous improvement
Stay up to date with emerging technologies, tools, and industry trends. Evaluate their potential impact on the organization and provide recommendations for technology adoption and innovation

Fulltime

Principal Data Engineer

The Principal Data Engineer at NTT DATA is responsible for transforming data int...

Location

India , Bangalore

Salary:

Not provided

NTT DATA

Expiration Date

Until further notice

Requirements

Bachelor's degree or equivalent in computer science, software engineering, information technology, or a related field
Extended demonstrated experience in data engineering, data mining within a fast-paced environment
Extended experience in building modern data analytics solutions that delivers insights from large and complex data sets with multi-terabyte scale
Extended demonstrated experience with architecture and design of secure, highly available and scalable systems
Extended demonstrated experience in automation, scripting and proven examples of successful implementation
Extended demonstrated experience using scripting language (Perl, bash, Shell Scripting, Python, etc.)
Extended demonstrated experience with big data tools like Hadoop, Cassandra, Storm etc.
Extended demonstrated experience in any applicable language, preferably .NET
Extended demonstrated experience in SAP, SQL, MySQL databases and Microsoft SQL
Extended demonstrated experience working with data sets and ordering data through MS Excel functions, e.g. macros, pivots

Job Responsibility

Leads on designing data models in a structured data format to enable analysis thereof
Design. and develop. scalable extract, transformation and loading (ETL) packages from the business source systems and the development of ETL routines to populate data from sources
Participates in the transformation of object and data models into appropriate database schemas within design constraints
Interprets installation standards to meet project needs and produce database components as required
Advises on test scenarios and be responsible for participating in thorough testing and validation to support the accuracy of data transformations
Accountable for running data migrations across different databases and applications, e.g. MS Dynamics, Oracle, SAP and other ERP systems
Works across multiple IT and business teams to define and implement data table structures and data models based on requirements
Accountable for analysis, and development of ETL and migration documentation
Works with cross functionally stakeholders to evaluate potential data requirements
Owns the definition and management of scoping, requirements, definition, and prioritization activities for small-scale changes and assist with more complex change initiatives

Fulltime

Principal Data Engineer

Microsoft’s Path team helps customers along their journey from the initial idea ...

Location

United States , Redmond

Salary:

139900.00 - 274800.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
OR equivalent experience.
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Job Responsibility

Design, implement, monitor, and optimize large-scale, robust data pipelines and architectures that support Microsoft Path products data analytics initiatives.
Develop scalable ETL/ELT workflows, ensure reliable ingestion, transformation, and storage of high-volume, high-velocity data from diverse sources with different schemas.
Enforce best practices for data quality, lineage, security, and compliance.
Collaborate with stakeholders to define trustworthy data sets and implement rigorous data validation protocols, ensuring analytics are accurate and auditable.
Work closely with product managers, software engineers, AI researchers, and business stakeholders to build data solutions that accomplish business goals.
Enable self-service analytics and empower teams by building data models, semantic layers, and tools that streamline access to trusted information.
Translate ambitious vision into actionable roadmaps and measurable outcomes.
Mentor senior data engineers and technical leads, guide complex design decisions, and foster a culture of engineering excellence, operational rigor, and continuous learning across the organization.

Fulltime

Principal Data Engineer

As Google Cloud's premier partner in AI, Datatonic provides world-class business...

Location

Croatia , Zagreb

Salary:

Not provided

Datatonic

Expiration Date

Until further notice

Requirements

Strong commercial awareness of Data Managed Services solutions, particularly with scoping and pitching to large enterprise customers including pricing, contracts and proposals
Strong interpersonal skills with the ability to work with clients to establish requirements in non-technical language
Ability to translate business requirements into plausible technical solutions for articulation to engineering teams
Ability to understand existing target architecture and adjust accordingly to accommodate new business requirements
Strong experience in monitoring and measuring platform performance and availability e.g SLO and SLI
Experience managing across relationships across internal and external customers, stakeholders and executives
Experience with Google Data Products tools (e.g., BigQuery, Dataflow, DataProc, Dataplex, Composer, Vertex, Looker, etc.)
Experience building and deploying solutions to Cloud (Google Cloud) including Cloud provisioning tools and management of releases through various environments into production
Experience and knowledge of application containerisation e.g. Docker, Kubernetes, Cloud Run, etc
Deep expertise with data warehousing (Kimball, Lakehouse, Data Mesh), particularly Big Query built using dbt/Dataform

Job Responsibility

Leadership and accountability for the technical roadmap of services
Implement and execute on the tooling strategy required to delivery a highly automated and scalable data managed service
Support the sales team with technical pre-sales enablement including build out pitch decks, architecture documents and sales collateral
Advise consulting teams on best practices for build of data platforms that are considerate of managed services
Working closely with OCTO (Office of the CTO) to ensure close alignment on existing and new standards
Being an advocate for and keeping up to date with new technologies, Google services, and the evolution of GenAI
Senior escalation point for intractable issues with customer platforms
Work with the Head of Managed Services on the strategic objectives of the Data Operations team
Contributing to development and evolution of the services portfolio
Coaching and mentoring junior technical staff, including responsibility for defining learning paths for engineering teams

Fulltime

Principal Data Engineer

We are on the lookout for a Principal Data Engineer to help define and lead the ...

Location

United Kingdom

Salary:

Not provided

Dotdigital

Expiration Date

Until further notice

Requirements

Extensive experience delivering python-based projects in the data engineering space
Extensive experience working with SQL and NoSQL database technologies (e.g. SQL Server, MongoDB & Cassandra)
Proven experience with modern data warehousing and large-scale data processing tools (e.g. Snowflake, DBT, BiqQuery, Clickhouse)
Hands on experience with data orchestration tools like Airflow, Dagster or Prefect
Experience using cloud environments (e.g. Azure, AWS, GCP) to process, store and surface large scale data
Experience using Kafka or similar event-based architectures e.g. (Pub/Sub via AWS SQS, Azure EventHubs, AWS Kinesis)
Strong grasp of data architecture and data modelling principles for both OLAP and OLTP workloads
Capable in the wider software development lifecycle in terms of agile ways of working and continuous integration/deployment of data solutions
Experience as a lead or Principal Engineer on large-scale data initiative or product builds
Demonstrated ability to architect data systems and data structures for high volume, high throughput systems

Job Responsibility

Lead the design and implementation of scalable, secure and resilient data systems across streaming, batch and real-time use cases
Architect data pipelines, model and storage solutions that power analytical and product use cases
using primarily Python and SQL via orchestration tooling that run workloads in the cloud
Leverage AI to automate both data processing and engineering processes
Assure and drive best practices relating to data infrastructure, governance, security and observability
Work with technologists across multiple teams to deliver coherent features and data outcomes
Support the data team to help adopt data engineering principles
Identify, validate and promote new tools and technologies that improve the performance and stability of data services

What we offer

Parental leave
Medical benefits
Paid sick leave
Dotdigital day
Share reward
Wellbeing reward
Wellbeing Days
Loyalty reward

Fulltime

Select Country

Principal Data Engineer

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Our AI answers in your language