Principal Data Engineer Job at Capstone IT Staffing (Springfield)

Principal Data Engineer

We are on the lookout for a Principal Data Engineer to help define and lead the ...

Location

United Kingdom

Salary:

Not provided

Dotdigital

Expiration Date

Until further notice

Requirements

Extensive experience delivering python-based projects in the data engineering space
Extensive experience working with SQL and NoSQL database technologies (e.g. SQL Server, MongoDB & Cassandra)
Proven experience with modern data warehousing and large-scale data processing tools (e.g. Snowflake, DBT, BiqQuery, Clickhouse)
Hands on experience with data orchestration tools like Airflow, Dagster or Prefect
Experience using cloud environments (e.g. Azure, AWS, GCP) to process, store and surface large scale data
Experience using Kafka or similar event-based architectures e.g. (Pub/Sub via AWS SQS, Azure EventHubs, AWS Kinesis)
Strong grasp of data architecture and data modelling principles for both OLAP and OLTP workloads
Capable in the wider software development lifecycle in terms of agile ways of working and continuous integration/deployment of data solutions
Experience as a lead or Principal Engineer on large-scale data initiative or product builds
Demonstrated ability to architect data systems and data structures for high volume, high throughput systems

Job Responsibility

Lead the design and implementation of scalable, secure and resilient data systems across streaming, batch and real-time use cases
Architect data pipelines, model and storage solutions that power analytical and product use cases
using primarily Python and SQL via orchestration tooling that run workloads in the cloud
Leverage AI to automate both data processing and engineering processes
Assure and drive best practices relating to data infrastructure, governance, security and observability
Work with technologists across multiple teams to deliver coherent features and data outcomes
Support the data team to help adopt data engineering principles
Identify, validate and promote new tools and technologies that improve the performance and stability of data services

What we offer

Parental leave
Medical benefits
Paid sick leave
Dotdigital day
Share reward
Wellbeing reward
Wellbeing Days
Loyalty reward

Fulltime

Principal Data Engineer

Our Principal Data Engineers are responsible for leading and delivering strategi...

Location

United Kingdom , Bristol; London; Manchester; Swansea

Salary:

100000.00 - 115000.00 GBP / Year

Made Tech

Expiration Date

Until further notice

Requirements

Understanding of the issues and challenges that the public sector faces in delivering services that make the best use of data and digital capabilities, transforming legacy infrastructure, and taking an innovative and user-centric approach
Ability to innovate and take learnings from the commercial sector, other countries and advances in technology and apply them to UK Public Sector challenges to create tangible solutions for our clients
Experience building trusted advisor relationships with senior client stakeholders within the public sector.
Experience of building and leading high performing, consulting teams and creating the leveraged engagements to provide a cost-effective, profitable, successful client-facing delivery
Leadership of bids and solution shaping to produce compelling proposals that help Made Tech win new business and grow the industry
Experience of managing third-party partnerships and suppliers (in conjunction with Made Tech colleagues) to provide a consolidated and seamless delivery team to clients.
Experience in delivering complex and difficult engagements that span multiple capabilities for user-facing digital and data services in the public sector
Experience in identifying opportunities based on client needs and developing targeted solutions to progress the development of the opportunity
Experience of working with sales professionals and commercial responsibility for strategic organisational goals.
Experience working directly with customers and users within a technology consultancy

Job Responsibility

Collaborate with clients to understand their needs, provide solution advice in your role as a trusted advisor and shape solutions that leverage Made Tech's wider capabilities and credentials
Assess project performance as a part of the billable delivery team, Quality Assure (QA) the deliverables and outcomes, and ensure client satisfaction. Coach and mentor team members as well as providing direction to enable them to achieve their engagement outcomes and to develop their careers
Act as a Technical Authority of the Data & AI capability to provide oversight and ensure alignment with internal and industry best practices. Ensure engagement experience is captured and used to improve standards and contribute to Made Tech knowledge
Participate in business development activities, including bids and pre-sales within the account, industry and practice. Coach team members on their contributions and oversee the relevant technical aspects of the proposal submission
Undertake people management responsibilities, including performance reviews and professional development of your engagement and practice colleagues
Serve as a thought leader within Made Tech, our account engagements and the wider public sector and represent the company at industry events

What we offer

30 days of paid annual leave + bank holidays
Flexible Parental Leave
Remote Working
Paid counselling as well as financial and legal advice
Flexible benefit platform which includes a Smart Tech scheme, Cycle to work scheme, and an individual benefits allowance which you can invest in a Health care cash plan or Pension plan
Optional social and wellbeing calendar of events

Fulltime

Principal Data Engineer

A VC-backed conversational AI scale-up is expanding its engineering team and is ...

Location

United States

Salary:

Not provided

Orbis Consultants

Expiration Date

Until further notice

Requirements

5+ years in software development and data engineering with ownership of production-grade systems
Proven expertise in Spark/PySpark
Strong knowledge of distributed computing and modern data modeling approaches
Solid programming skills in Python, with an emphasis on clean, maintainable code
Hands-on experience with SQL and NoSQL databases (e.g., PostgreSQL, DynamoDB, Cassandra)
Excellent communicator who can influence and partner across teams

Job Responsibility

Design and evolve distributed, cloud-based data infrastructure that supports both real-time and batch processing at scale
Build high-performance data pipelines that power analytics, AI/ML workloads, and integrations with third-party platforms
Champion data reliability, quality, and observability, introducing automation and monitoring across pipelines
Collaborate closely with engineering, product, and AI teams to deliver data solutions for business-critical initiatives

What we offer

Fully remote
great equity
the chance to join a rocketship

Fulltime

Principal Data Engineer

We are seeking an experienced Principal Data Engineer to define, lead, and scale...

Location

United Kingdom , Thame; Leeds

Salary:

80000.00 - 100000.00 GBP / Year

PEXA UK

Expiration Date

Until further notice

Requirements

Broad experience as a Data Engineer including technical leadership
Broad cloud experience, ideally both Azure and AWS
Deep expertise in PySpark and distributed data processing at scale
Extensive experience designing and optimising in Databricks
Advanced SQL optimisation and schema design for analytical workloads
Strong understanding of data security, privacy, and GDPR/PII compliance
Experience implementing and leading data governance frameworks
Proven experience leading the design and operation of a complex data platform
Track record of mentoring engineers and raising technical standards
Ability to influence senior stakeholders and align data initiatives with wider business goals

Job Responsibility

Design and oversee scalable, performant, and secure architectures on Databricks and distributed systems
Anticipate scaling challenges and ensure platforms are future-proof
Lead the design and development of robust, high-performance data pipelines using PySpark and Databricks
Define and ensure testing frameworks for data workflows
Ensure end-to-end data quality from raw ingestion to curated, trusted datasets powering analytics
Establish and enforce best practices for data governance, lineage, metadata, and security controls
Ensure compliance with GDPR and other regulatory frameworks
Act as a technical authority and mentor, guiding data engineers
Influence cross-functional teams to align on data strategy, standards, and practices
Partner with product, engineering, and business leaders to prioritise and deliver high-impact data initiatives

Fulltime

Principal Data Engineer

PointClickCare is searching for a Principal Data Engineer who will contribute to...

Location

United States

Salary:

183200.00 - 203500.00 USD / Year

PointClickCare

Expiration Date

Until further notice

Requirements

Principal Data Engineer with at least 10 years of professional experience in software or data engineering, including a minimum of 4 years focused on streaming and real-time data systems
Proven experience driving technical direction and mentoring engineers while delivering complex, high-scale solutions as a hands-on contributor
Deep expertise in streaming and real-time data technologies, including frameworks such as Apache Kafka, Flink, and Spark Streaming
Strong understanding of event-driven architectures and distributed systems, with hands-on experience implementing resilient, low-latency pipelines
Practical experience with cloud platforms (AWS, Azure, or GCP) and containerized deployments for data workloads
Fluency in data quality practices and CI/CD integration, including schema management, automated testing, and validation frameworks (e.g., dbt, Great Expectations)
Operational excellence in observability, with experience implementing metrics, logging, tracing, and alerting for data pipelines using modern tools
Solid foundation in data governance and performance optimization, ensuring reliability and scalability across batch and streaming environments
Experience with Lakehouse architectures and related technologies, including Databricks, Azure ADLS Gen2, and Apache Hudi
Strong collaboration and communication skills, with the ability to influence stakeholders and evangelize modern data practices within your team and across the organization

Job Responsibility

Lead and guide the design and implementation of scalable streaming data pipelines
Engineer and optimize real-time data solutions using frameworks like Apache Kafka, Flink, Spark Streaming
Collaborate cross-functionally with product, analytics, and AI teams to ensure data is a strategic asset
Advance ongoing modernization efforts, deepening adoption of event-driven architectures and cloud-native technologies
Drive adoption of best practices in data governance, observability, and performance tuning for streaming workloads
Embed data quality in processing pipelines by defining schema contracts, implementing transformation tests and data assertions, enforcing backward-compatible schema evolution, and automating checks for freshness, completeness, and accuracy across batch and streaming paths before production deployment
Establish robust observability for data pipelines by implementing metrics, logging, and distributed tracing for streaming jobs, defining SLAs and SLOs for latency and throughput, and integrating alerting and dashboards to enable proactive monitoring and rapid incident response
Foster a culture of quality through peer reviews, providing constructive feedback and seeking input on your own work

What we offer

Benefits starting from Day 1!
Retirement Plan Matching
Flexible Paid Time Off
Wellness Support Programs and Resources
Parental & Caregiver Leaves
Fertility & Adoption Support
Continuous Development Support Program
Employee Assistance Program
Allyship and Inclusion Communities
Employee Recognition … and more!

Fulltime

Principal Data Engineer

The Principal Data Engineer will lead the design, development, and optimization ...

Location

United States , Washington DC; Philadelphia PA; Wilmington DE

Salary:

113200.00 - 146664.00 USD / Year

AMTRAK

Expiration Date

Until further notice

Requirements

Bachelor’s Degree or equivalent combination of education, training and/or relevant experience
Plus 7 years of relevant work experience
Proficiency and hands on experience designing and implementing end-to-end data solutions in AWS (e.g., S3, EMR, Glue, Redshift, Kenesis) and/or Azure (e.g. Azure Data Factory, Synaps Analytics, Azure Data Lakre Storage)
Experience with some of the following technologies, Databricks, Python, Apche Sparks, IDMC, Talend, CI/CD pipelines, Jenkins, Gitlab, Power BI, SQL and NoSQL

Job Responsibility

Architect and Design: Define and implement the architectural strategy for our enterprise data platform, focusing on scalability, security, and performance. Design and build robust, high-volume, and performant data pipelines using cloud-native services like AWS Data Pipelines (Glue, EMR, S3, Redshift, etc.) or Azure Data Factory/Synapse Analytics
Technical Leadership: Act as a subject matter expert and mentor for junior and mid-level data engineers, setting best practices for code quality, testing, and deployment
Data Governance & Management: Utilize tools like IDMC (Informatica Data Management Cloud) for data integration, quality, governance, and cataloging across the enterprise
Data Processing and Analysis: Develop, optimize, and manage large-scale data processing jobs using Databricks(Spark/Delta Lake) for ETL/ELT workflows and advanced analytics
Coding and Scripting: Write high-quality, efficient, and well-documented code primarily in Python for data manipulation, automation, and pipeline orchestration
Deployment and Automation: Implement and maintain robust CI/CD pipelines and infrastructure-as-code (e.g., Terraform/CloudFormation) for automated deployment and management of data solutions
Business Intelligence: Ensure data readiness for reporting and analytics and possess working knowledge/experience with BI tools like Tableau and PowerBI to facilitate data consumption
Performance and Optimization: Monitor, troubleshoot, and tune existing data infrastructure and pipelines to ensure optimal performance and cost efficiency

What we offer

health, dental, and vision plans
health savings accounts
wellness programs
flexible spending accounts
401K retirement plan with employer match
life insurance
short and long term disability insurance
paid time off
back-up care
adoption assistance

Fulltime

Principal Consulting AI / Data Engineer

As a Principal Consulting AI / Data Engineer, you will design, build, and optimi...

Location

Australia , Sydney

Salary:

Not provided

DyFlex Solutions

Expiration Date

Until further notice

Requirements

Proven expertise in delivering enterprise-grade data engineering and AI solutions in production environments
Strong proficiency in Python and SQL, plus experience with Spark, Airflow, dbt, Kafka, or Flink
Experience with cloud platforms (AWS, Azure, or GCP) and Databricks
Ability to confidently communicate and present at C-suite level, simplifying technical concepts into business impact
Track record of engaging senior executives and influencing strategic decisions
Strong consulting and stakeholder management skills with client-facing experience
Background in MLOps, ML pipelines, or AI solution delivery highly regarded
Degree in Computer Science, Engineering, Data Science, Mathematics, or a related field

Job Responsibility

Design, build, and maintain scalable data and AI solutions using Databricks, cloud platforms, and modern frameworks
Lead solution architecture discussions with clients, ensuring alignment of technical delivery with business strategy
Present to and influence executive-level stakeholders, including boards, C-suite, and senior directors
Translate highly technical solutions into clear business value propositions for non-technical audiences
Mentor and guide teams of engineers and consultants to deliver high-quality solutions
Champion best practices across data engineering, MLOps, and cloud delivery
Build DyFlex’s reputation as a trusted partner in Data & AI through thought leadership and client advocacy

What we offer

Work with SAP’s latest technologies on cloud as S/4HANA, BTP and Joule, plus Databricks, ML/AI tools and cloud platforms
A flexible and supportive work environment including work from home
Competitive remuneration and benefits including novated lease, birthday leave, salary packaging, wellbeing programme, additional purchased leave, and company-provided laptop
Comprehensive training budget and paid certifications (Databricks, SAP, cloud platforms)
Structured career advancement pathways with opportunities to lead large-scale client programs
Exposure to diverse industries and client environments, including executive-level engagement

Fulltime

Senior Principal Data Platform Software Engineer

We’re looking for a Sr Principal Data Platform Software Engineer (P70) to be a k...

Location

Salary:

239400.00 - 312550.00 USD / Year

Atlassian

Expiration Date

Until further notice

Requirements

15+ years in Data Engineering, Software Engineering, or related roles, with substantial exposure to big data ecosystems
Demonstrated experience building and operating data platforms or large‑scale data services in production
Proven track record of building services from the ground up (requirements → design → implementation → deployment → ongoing ownership)
Hands‑on experience with AWS, GCP (e.g., compute, storage, data, and streaming services) and cloud‑native architectures
Practical experience with big data technologies, such as Databricks, Apache Spark, AWS EMR, Apache Flink, or StarRocks
Strong programming skills in one or more of: Kotlin, Scala, Java, Python
Experience leading cross‑team technical initiatives and influencing senior stakeholders
Experience mentoring Staff/Principal engineers and lifting the technical bar for a team or org
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience

Job Responsibility

Design, develop and own delivery of high quality big data and analytical platform solutions aiming to solve Atlassian’s needs to support millions of users with optimal cost, minimal latency and maximum reliability
Improve and operate large‑scale distributed data systems in the cloud (primarily AWS, with increasing integration with GCP and Kubernetes‑based microservices)
Drive the evolution of our high-performance analytical databases and its integrations with products, cloud infrastructures (AWS and GCP) and isolated cloud environments
Help define and uplift engineering and operational standards for petabyte scale data platforms, with sub‑second analytic queries and multi‑region availability (coding guidelines, code review practices, observability, incident response, SLIs/SLOs)
Partner across multiple product and platform teams (including Analytics, Marketplace/Ecosystem, Core Data Platform, ML Platform, Search, and Oasis/FedRAMP) to deliver company‑wide initiatives that depend on reliable, high‑quality data
Act as a technical mentor and multiplier, raising the bar on design quality, code quality, and operational excellence across the broader team
Design and implement self‑healing, resilient data platforms with strong observability, fault tolerance, and recovery characteristics
Own the long‑term architecture and technical direction of Atlassian’s product data platform with projects that are directly tied to Atlassian’s company-level OKRs
Be accountable for the reliability, cost efficiency, and strategic direction of Atlassian’s product analytical data platform
Partner with executives and influence senior leaders to align engineering efforts with Atlassian’s long-term business objectives

What we offer

health and wellbeing resources
paid volunteer days

Fulltime

Select Country

Principal Data Engineer

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Data Engineer

Principal Consulting AI / Data Engineer

Senior Principal Data Platform Software Engineer

Our AI answers in your language