CrawlJobs Logo

Senior Principal Data Engineer

https://www.atlassian.com Logo

Atlassian

Location Icon

Location:
United States, San Francisco

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

Atlassian is looking for a Senior Principal Data Engineer to join the Go-To Market Data Engineering (GTM-DE) team to build data lakes, maintain big data pipelines/services, and facilitate the movement of billions of messages daily. Work directly with business stakeholders and platform/engineering teams to enable growth and retention strategies. Use technical expertise to manage and orchestrate multi-petabyte scale data lakes and build scalable, efficient data pipelines and services.

Job Responsibility:

  • Help stakeholder teams ingest data faster into our data lake
  • find ways to make data pipelines more efficient
  • come up with ideas to help instigate self-serve data engineering within the company
  • building micro-services, architecting, designing, and enabling self-serve capabilities at scale to help Atlassian grow

Requirements:

  • 18+ years of experience in a Data Engineer role as an individual contributor
  • at least 7 years of experience as a tech lead for Data Engineering teams, and delivered complex, cross-team initiatives
  • built durable relationships with executives/senior leaders across Sales, Marketing, Finance, Commerce and related organizations, and understand complexities of data in these organizations
  • a track record of driving and delivering large complex, multi-team efforts
  • a great communicator and maintain many of the essential cross-team and cross-functional relationships necessary for the team's success
  • experience with building streaming pipelines with a micro-services architecture for low-latency analytics
  • experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, dbt, and column stores (e.g. Redshift)
  • experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools
  • experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka)
  • understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team
  • industry experience working with large-scale, high-performance data processing systems (batch and streaming) with a 'Streaming First' mindset to drive Atlassian's business growth and improve the product experience

Nice to have:

  • built and designed Kappa or Lambda architecture data platforms and services
  • experience implementing Master Data Management (MDM), Customer Relationship Management (CRM) solutions
  • built pipelines using Databricks and well-versed with their APIs
  • contributed to open source projects
  • well-versed in Python, & Spark and scaling data pipelines
What we offer:
  • health coverage
  • paid volunteer days
  • wellness resources

Additional Information:

Job Posted:
April 30, 2025

Employment Type:
Fulltime
Work Type:
Remote work
Icon
View All Jobs In This Company
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Senior Principal Data Engineer

New

Principal Data Engineer

We are seeking an experienced Principal Data Engineer to define, lead, and scale...
Location
Location
United Kingdom , Thame; Leeds
Salary
Salary:
80000.00 - 100000.00 GBP / Year
pexa.co.uk Logo
PEXA UK
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Broad experience as a Data Engineer including technical leadership
  • Broad cloud experience, ideally both Azure and AWS
  • Deep expertise in PySpark and distributed data processing at scale
  • Extensive experience designing and optimising in Databricks
  • Advanced SQL optimisation and schema design for analytical workloads
  • Strong understanding of data security, privacy, and GDPR/PII compliance
  • Experience implementing and leading data governance frameworks
  • Proven experience leading the design and operation of a complex data platform
  • Track record of mentoring engineers and raising technical standards
  • Ability to influence senior stakeholders and align data initiatives with wider business goals
Job Responsibility
Job Responsibility
  • Design and oversee scalable, performant, and secure architectures on Databricks and distributed systems
  • Anticipate scaling challenges and ensure platforms are future-proof
  • Lead the design and development of robust, high-performance data pipelines using PySpark and Databricks
  • Define and ensure testing frameworks for data workflows
  • Ensure end-to-end data quality from raw ingestion to curated, trusted datasets powering analytics
  • Establish and enforce best practices for data governance, lineage, metadata, and security controls
  • Ensure compliance with GDPR and other regulatory frameworks
  • Act as a technical authority and mentor, guiding data engineers
  • Influence cross-functional teams to align on data strategy, standards, and practices
  • Partner with product, engineering, and business leaders to prioritise and deliver high-impact data initiatives
  • Fulltime
Read More
Arrow Right
New

Principal Data Engineer

Our Principal Data Engineers are responsible for leading and delivering strategi...
Location
Location
United Kingdom , Bristol; London; Manchester; Swansea
Salary
Salary:
100000.00 - 115000.00 GBP / Year
madetech.com Logo
Made Tech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Understanding of the issues and challenges that the public sector faces in delivering services that make the best use of data and digital capabilities, transforming legacy infrastructure, and taking an innovative and user-centric approach
  • Ability to innovate and take learnings from the commercial sector, other countries and advances in technology and apply them to UK Public Sector challenges to create tangible solutions for our clients
  • Experience building trusted advisor relationships with senior client stakeholders within the public sector.
  • Experience of building and leading high performing, consulting teams and creating the leveraged engagements to provide a cost-effective, profitable, successful client-facing delivery
  • Leadership of bids and solution shaping to produce compelling proposals that help Made Tech win new business and grow the industry
  • Experience of managing third-party partnerships and suppliers (in conjunction with Made Tech colleagues) to provide a consolidated and seamless delivery team to clients.
  • Experience in delivering complex and difficult engagements that span multiple capabilities for user-facing digital and data services in the public sector
  • Experience in identifying opportunities based on client needs and developing targeted solutions to progress the development of the opportunity
  • Experience of working with sales professionals and commercial responsibility for strategic organisational goals.
  • Experience working directly with customers and users within a technology consultancy
Job Responsibility
Job Responsibility
  • Collaborate with clients to understand their needs, provide solution advice in your role as a trusted advisor and shape solutions that leverage Made Tech's wider capabilities and credentials
  • Assess project performance as a part of the billable delivery team, Quality Assure (QA) the deliverables and outcomes, and ensure client satisfaction. Coach and mentor team members as well as providing direction to enable them to achieve their engagement outcomes and to develop their careers
  • Act as a Technical Authority of the Data & AI capability to provide oversight and ensure alignment with internal and industry best practices. Ensure engagement experience is captured and used to improve standards and contribute to Made Tech knowledge
  • Participate in business development activities, including bids and pre-sales within the account, industry and practice. Coach team members on their contributions and oversee the relevant technical aspects of the proposal submission
  • Undertake people management responsibilities, including performance reviews and professional development of your engagement and practice colleagues
  • Serve as a thought leader within Made Tech, our account engagements and the wider public sector and represent the company at industry events
What we offer
What we offer
  • 30 days of paid annual leave + bank holidays
  • Flexible Parental Leave
  • Remote Working
  • Paid counselling as well as financial and legal advice
  • Flexible benefit platform which includes a Smart Tech scheme, Cycle to work scheme, and an individual benefits allowance which you can invest in a Health care cash plan or Pension plan
  • Optional social and wellbeing calendar of events
  • Fulltime
Read More
Arrow Right

Senior Principal Engineer Core Data Platform

As an engineer well into your career, we know you're an expert at what you do an...
Location
Location
United States , Seattle; San Francisco; Mountain View
Salary
Salary:
198300.00 - 318600.00 USD / Year
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related technical field
  • 12+ years of experience in backend software development, with a focus on distributed systems and large-scale storage solutions
  • 8+ years of experience designing and managing highly available, large-scale storage architectures in cloud environments
  • 5+ years of hands-on experience working with AWS storage services (S3, EBS, EFS, FSx, Glacier, DynamoDB)
  • Proficiency in system design, performance optimization, and cost-efficient architecture for exabyte-scale storage
  • Expertise in at least one major backend programming language (Kotlin, Java, Go, Rust, or Python)
  • Experience leading technical strategy and architectural decisions in large, multi-team engineering organizations
  • Strong understanding of distributed systems principles, including consistency models, replication, sharding, and consensus algorithms (Raft, Paxos)
  • Deep knowledge of security best practices, including encryption, access control (IAM), and compliance standards (SOC2, GDPR, HIPAA)
  • Experience mentoring senior engineers and driving high-impact engineering initiatives
Job Responsibility
Job Responsibility
  • Collaborate with partner teams and internal customers to help define technical direction and OKRs for the Core Data platform organization
  • Regularly tackle the largest and most complex problems on the team, from technical design to implementation and launch
  • Partner across engineering teams to take on company-wide initiatives spanning multiple projects
  • Routinely tackle complex architecture challenges and apply architectural standards and start using them on new projects
  • Work across senior engineering and product leaders to build strategy and design solutions to earn customers trust and business
  • Own key OKRs and end-to-end outcomes of critical projects in a microservices environment
  • Champion best practices and innovative techniques for scalability, reliability, and performance optimizations
  • Own engineering and operational excellence for the health of our systems and processes
  • Proactively drive opportunities for continuous improvements and own key operational metrics
  • Continually drive developer productivity initiatives to ensure that we unleash the potential of our own teams
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Principal Consulting AI / Data Engineer

As a Principal Consulting AI / Data Engineer, you will design, build, and optimi...
Location
Location
Australia , Sydney
Salary
Salary:
Not provided
dyflex.com.au Logo
DyFlex Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven expertise in delivering enterprise-grade data engineering and AI solutions in production environments
  • Strong proficiency in Python and SQL, plus experience with Spark, Airflow, dbt, Kafka, or Flink
  • Experience with cloud platforms (AWS, Azure, or GCP) and Databricks
  • Ability to confidently communicate and present at C-suite level, simplifying technical concepts into business impact
  • Track record of engaging senior executives and influencing strategic decisions
  • Strong consulting and stakeholder management skills with client-facing experience
  • Background in MLOps, ML pipelines, or AI solution delivery highly regarded
  • Degree in Computer Science, Engineering, Data Science, Mathematics, or a related field
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable data and AI solutions using Databricks, cloud platforms, and modern frameworks
  • Lead solution architecture discussions with clients, ensuring alignment of technical delivery with business strategy
  • Present to and influence executive-level stakeholders, including boards, C-suite, and senior directors
  • Translate highly technical solutions into clear business value propositions for non-technical audiences
  • Mentor and guide teams of engineers and consultants to deliver high-quality solutions
  • Champion best practices across data engineering, MLOps, and cloud delivery
  • Build DyFlex’s reputation as a trusted partner in Data & AI through thought leadership and client advocacy
What we offer
What we offer
  • Work with SAP’s latest technologies on cloud as S/4HANA, BTP and Joule, plus Databricks, ML/AI tools and cloud platforms
  • A flexible and supportive work environment including work from home
  • Competitive remuneration and benefits including novated lease, birthday leave, salary packaging, wellbeing programme, additional purchased leave, and company-provided laptop
  • Comprehensive training budget and paid certifications (Databricks, SAP, cloud platforms)
  • Structured career advancement pathways with opportunities to lead large-scale client programs
  • Exposure to diverse industries and client environments, including executive-level engagement
  • Fulltime
Read More
Arrow Right

Senior Principal Engineer

The Core Engineering Compute Platform team is responsible for powering all of At...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong technical expertise and 12+ years experience in designing/building enterprise grade public cloud or platform as a service solutions
  • Proven experience in a senior engineering leadership role, preferably in a large-scale, distributed systems environment
  • Deep understanding of cloud computing, Kubernetes, microservices architecture, and container orchestration technologies
  • Experience building and operating large scale distributed systems using cloud technologies from AWS/Azure/GCP
  • Focus on business outcomes and the 80/20 rule
  • very high bar for output quality, but recognize the business benefit of 'ship small, ship fast and iterate’ vs ‘building for perfection’
  • Passion for engineering rigor and operational excellence (design principles and patterns, unit testing, best practices for security and privacy, CI/CD etc)
  • You will be a role model for operational responsibility and data-driven engineering rigor
  • Excellent collaboration, communication and leadership skills
  • Ability to get engineers to follow their lead as well as mentor/coach
Job Responsibility
Job Responsibility
  • Collaborate with other core engineering teams and internal customers to help define technical direction and OKRs for the Compute Platform team
  • Collaborate with cross-functional teams to ensure seamless integration and deliver a seamless compute platform experience
  • Evaluate and implement new technologies and frameworks to enhance platform capabilities, designed with security, performance, and scalability in mind
  • Own key OKRs and end-to-end outcomes of critical projects in a globally distributed environment
  • Develop and execute a strategic vision for the compute platform that aligns with the company’s goals and objectives
  • Identify opportunities for innovation and improvement in platform services and developer tools
  • Own engineering and operational excellence for the health of our systems and processes
  • Proactively drive opportunities for continuous improvements and own key operational metrics
  • Continually drive developer productivity initiatives to ensure that we unleash the potential of our own team
  • Help hire and mentor engineers to build an open, innovative and operationally responsible culture
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Senior Principal Engineer

As a senior technical leader, you will work closely with the Cloud Transition Ca...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong technical expertise and 14+ years experience in designing/building enterprise grade solutions using microservices
  • Fluency in at least one modern object-oriented programming language (preferably Java)
  • Broad experience architecting, designing, and building large-scale systems with multiple dependencies
  • Passion for engineering rigor and operational excellence (design principles and patterns, unit testing, best practices for security and privacy, CI/CD etc)
  • Success with building, expressing, and pitching a technical vision to stakeholders
  • Driving consensus with other cross team technical leaders
  • Excellent collaboration, communication and leadership skills
  • Ability to get engineers to follow their lead as well as mentor/coach
  • Success with leading the long-term strategy for software architecture
  • Experience with Enterprise Customers
Job Responsibility
Job Responsibility
  • Define and execute the vision and strategy for the group
  • Lead other engineers to define strategic technical direction
  • Execute projects from design to launch
  • Collaborate with other teams and leaders globally to influence, set expectations, gather inputs and communicate results
  • Identify and drive step-change initiatives
  • Build world class platforms to power Backup/Restore, Sandbox, Data Portability and Wall-to-wall expansion from competitors
  • Inspire other engineers in professional development
  • Contribute to engineering and operational practices
  • Bring in a continuous improvement mindset
  • Fulltime
Read More
Arrow Right

Senior Principal Software Engineer

We're hiring for Senior Principal Engineers to join our ITSM Engineering Organis...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 15 years of hands-on experience in software/technology roles, in Enterprise Products & Solutions in the cloud
  • Proficiency with cloud-native technologies, ideally AWS
  • At least 6 years of experience architecting and implementing multiple features of a SaaS product at a high scale, performance and reliability of at least 4 nines
  • Experience with building and delivering multi-tenant cloud-native SaaS software
  • Strong understanding of distributed systems, modern cloud-native architectures, and distributed data management
  • Experience communicating updates and resolutions to senior and executive level stakeholders and other partners to drive the mission forward
  • Passion for maintaining high-quality standards in software development
  • Success in articulating and pitching a technical vision to stakeholders
  • Ability to balance short- and long-term objectives, focusing on both technical and business outcomes
Job Responsibility
Job Responsibility
  • Shape the forward-looking technical direction and long-term architecture for pillars in the ITSM Organisation
  • Collaborate with product, engineering, and design leaders to influence the broader department-level strategy
  • Ensure that the technical strategy you build is aligned with the technical strategy of Atlassian products and platforms
  • Partner with principal engineers and architects from other teams and guide exploration of large-scale projects spanning multiple teams in ITSM and beyond
  • Propel technical evolution while enhancing the engineering culture across the org and beyond, promoting cross-team collaboration and delivering measurable results
  • Be influential within your team and work with peers and senior leaders to define and revise the standards for operational excellence across Atlassian
  • Mentor, hire and develop other engineers
What we offer
What we offer
  • Health and wellbeing resources
  • Paid volunteer days
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Transactional Data Platform

As a Senior Software Engineer, you will play a critical role in designing, build...
Location
Location
Australia , Sydney
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related technical field
  • 5+ years of experience in backend software development
  • 3+ years of hands-on experience working with AWS cloud services, particularly AWS storage technologies (S3, DynamoDB, EBS, EFS, FSx, or Glacier)
  • 3+ years of experience in designing and developing distributed systems or high-scale backend services
  • Strong programming skills in Kotlin
  • Experience working in agile environments following DevOps and CI/CD best practices
  • Strong Backend Development Skills
  • Proficiency in Kotlin, Java for backend development
  • Experience building high-performance, scalable microservices and APIs
  • Strong understanding of RESTful APIs, gRPC, and event-driven architectures
Job Responsibility
Job Responsibility
  • Designing, building, and optimizing high-performance, scalable, and resilient backend storage solutions on AWS cloud infrastructure
  • Developing distributed storage systems, APIs, and backend services that power mission-critical applications, ensuring low-latency, high-throughput, and fault-tolerant data storage
  • Collaborating closely with principal engineers, architects, SREs, and product teams to define technical roadmaps, improve storage efficiency, and optimize access patterns
  • Driving performance tuning, data modeling, caching strategies, and cost optimization across AWS storage services like S3, DynamoDB, EBS, EFS, FSx, and Glacier
  • Contributing to infrastructure automation, security best practices, and monitoring strategies using tools like Terraform, CloudWatch, Prometheus, and OpenTelemetry
  • Troubleshooting and resolving production incidents related to data integrity, latency spikes, and storage failures, ensuring high availability and disaster recovery preparedness
  • Mentoring junior engineers, participating in design reviews and architectural discussions, and advocating for engineering best practices such as CI/CD automation, infrastructure as code, and observability-driven development
What we offer
What we offer
  • Atlassians can choose where they work – whether in an office, from home, or a combination of the two
  • Flexibility for eligible candidates to work remotely across the West US
  • Fulltime
Read More
Arrow Right
Welcome to CrawlJobs.com –
Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Australia Jobs Canada Jobs Poland Jobs Ireland Jobs Spain Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
© 2025 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy