CrawlJobs Logo
Optiver Logo Optiver · IT - Software Development

Principal Engineer (Data)

Australia, Sydney · Job Posted March 26, 2026
Apply Position
Job Link Share

Job Description

As a Principal Engineer, you will play a key role in building and evolving Optiver’s next-generation data platform that supports research and real-time trading. The platform spans orchestration, storage, access and governance of data at scale, enabling engineers, researchers and traders to work efficiently with high-quality data. This is a hands-on role with meaningful contribution to system design and production code.

Job Responsibility

  • Designing and developing scalable data platform components (e.g. orchestration, storage formats, metadata and streaming systems)
  • Working across the full lifecycle: problem understanding, design, implementation, testing, deployment and monitoring
  • Contributing to architectural decisions that improve performance, scalability and reliability
  • Standardising data ingestion and access patterns, including schema management, lineage, permissions and auditing
  • Collaborating with stakeholders across engineering, research, trading, SRE and security to deliver impactful solutions
  • Driving high-quality engineering practices, including code quality, testing and observability
  • Partnering with global teams to align on shared patterns and platforms

Requirements

  • 7+ years experience as a Software Engineer delivering production platforms
  • Strong experience with distributed systems & reliability fundamentals, including partitioning, replication, consistency/availability tradeoffs
  • Expertise with one or more of Python/C++/Java/Rust
  • Solid API and system design skills
  • Data platform fundamentals, including batch/stream processing, table formats, object storage, SQL
  • Understanding of security and governance, such as encryption, authentication, and access control
  • Practical experience with streaming and batch data (e.g., Kafka + Schema Registry)
  • Experience making architectural decisions on IT solutions, system design & implementation
  • Excellent stakeholder communication

What we offer

  • A performance-based bonus structure
  • Training, mentorship and personal development opportunities
  • Daily breakfast, lunch and an in-house barista
  • Gym membership plus weekly in-house chair massages
  • Regular social events, including a company trip every two years
  • Guided relocation, a competitive relocation package and visa sponsorship where necessary
Optiver - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Principal Engineer (Data)

8 matching positions

Principal Engineer I – Master Data Management Engineer

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Irving
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Must have hands-on experience in Profisee or other MDM tools
  • C# & .NET are Required
  • 8+ years' experience with MDM solutions using Profisee or any other MDM tools
  • 3+ years' experience designing, implementing, and supporting cloud data solutions
  • Azure Data Lake Storage, Azure Data Factory, Azure Data Services, Azure Logic Apps etc
  • Bachelor's degree in related field required
  • Expert level experience with at least one RDBMS and query language such as T-SQL, PL/SQL, Spark SQL
  • Expert level experience in conceptual, logical, and physical data design
  • Advanced to expert knowledge of general Financial Services or Banking is preferred
  • Advanced to expert knowledge of applicable regulatory and legal compliance obligations, rules and regulations, industry standards, and practices
Job Responsibility
Job Responsibility
  • Design, develop, and implement key components of the enterprise data platform
  • Lead the design, development, and maintenance of MDM solutions using Profisee
  • Develop and implement data quality rules and processes within Profisee
  • Design, develop, and implement match and merge functionalities, as well as hierarchical structures, utilizing Profisee
  • Establish and implement survivorship rules, business rules, and workflow automations
  • Diagnose and resolve long-running processes, as well as memory and CPU performance concerns
  • Design and implement Azure DevOps CI/CD pipelines to facilitate the deployment of Profisee deliverables
  • Design and implement data integration processes between Profisee and other enterprise systems
  • Participate in the full MDM implementation lifecycle
  • Collaborate with cross-functional teams and business stakeholders to define master data domains
What we offer
What we offer
  • medical
  • dental
  • vision coverage
  • 401(k) with company match
  • short-term disability
  • life insurance with AD&D
  • Fulltime
Read More
Arrow Right

Principal Engineer I – Master Data Management Engineer

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Irving
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years’ experience with MDM solutions using Profisee or any other MDM tools
  • 3+ years’ experience designing, implementing, and supporting cloud data solutions
  • Azure Data Lake Storage, Azure Data Factory, Azure Data Services, Azure Logic Apps etc.
  • Bachelor’s degree in related field required
  • Expert level experience with at least one RDBMS and query language such as T-SQL, PL/SQL, Spark SQL
  • Expert level experience in conceptual, logical, and physical data design
  • Advanced speaking and writing communication skills
Job Responsibility
Job Responsibility
  • Lead the design, development, and maintenance of MDM solutions using Profisee, including data models, workflows, business rules, and user interfaces
  • Develop and implement data quality rules and processes within Profisee to ensure data accuracy and consistency
  • Design, develop, and implement match and merge functionalities, as well as hierarchical structures, utilizing Profisee
  • Establish and implement survivorship rules, business rules, and workflow automations
  • Diagnose and resolve long-running processes, as well as memory and CPU performance concerns
  • Design and implement Azure DevOps CI/CD pipelines to facilitate the deployment of Profisee deliverables
  • Design and implement data integration processes between Profisee and other enterprise systems using various integration techniques (API, ETL, etc.)
  • Participate in the full MDM implementation lifecycle, including requirements gathering, design, development, testing, deployment, and support
  • Collaborate with cross-functional teams and business stakeholders to define master data domains, hierarchies, and data models aligned with business objectives and data governance principles
  • Design, develop, and deploy enterprise-grade data engineering solutions utilizing Azure Data Factory, Synapse Analytics, Azure DevOps, and related technologies
What we offer
What we offer
  • medical
  • dental
  • vision coverage
  • 401(k) with company match
  • short-term disability
  • life insurance with AD&D
  • Fulltime
Read More
Arrow Right

Principal Engineer I – Master Data Management Engineer

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Phoenix
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years’ experience with MDM solutions using Profisee or any other MDM tools
  • 3+ years’ experience designing, implementing, and supporting cloud data solutions
  • Azure Data Lake Storage, Azure Data Factory, Azure Data Services, Azure Logic Apps etc.
  • Bachelor’s degree in related field required
  • Masters or MBA in related field preferred
  • Previous leadership experience preferred
  • Expert level experience with at least one RDBMS and query language such as T-SQL, PL/SQL, Spark SQL
  • Expert level experience in conceptual, logical, and physical data design
  • Certifications within Azure such as Azure Fundamentals, Azure Data Engineer, Azure Data Scientist, and/or Azure DevOps Engineer preferred
  • Familiarity with multi-cloud data management and transformation platforms or tools such as Databricks, Snowflake etc.
Job Responsibility
Job Responsibility
  • Act as a Subject Matter Expert within the Enterprise Data & Analytics domain and related fields, ensuring that Master Data Management (MDM) and data engineering solutions adhere to standards of safety, security, compliance, and reliability
  • Design, develop, and implement key components of the enterprise data platform in alignment with Western Alliance Bank’s strategic data and analytics goals
  • Assess development and support needs, assume accountability for extensive design initiatives, and play an essential role in delivering successful project outcomes
  • Collaborate directly with business data stewards, analysts, engineers, enterprise architects, and business stakeholders to refine requirements and shape the objectives of enterprise data solutions
  • Lead a high-performing, centralized team of engineers, analysts, and QA engineers, developing solutions that address the complete data lifecycle management needs of our enterprise data platforms
  • Mentor and guide team members, fostering a culture of excellence and continuous learning
  • Lead the design, development, and maintenance of MDM solutions using Profisee, including data models, workflows, business rules, and user interfaces
  • Develop and implement data quality rules and processes within Profisee to ensure data accuracy and consistency
  • Design, develop, and implement match and merge functionalities, as well as hierarchical structures, utilizing Profisee
  • Establish and implement survivorship rules, business rules, and workflow automations
What we offer
What we offer
  • medical, dental, and vision coverage
  • a 401(k) with company match
  • short-term disability
  • life insurance with AD&D
Read More
Arrow Right

Principal Engineer I – Master Data Management Engineer

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Irving
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years' experience with MDM solutions using Profisee or any other MDM tools
  • 3+ years' experience designing, implementing, and supporting cloud data solutions
  • Azure Data Lake Storage, Azure Data Factory, Azure Data Services, Azure Logic Apps etc.
  • Bachelor's degree in related field required
  • Masters or MBA in related field preferred
  • Previous leadership experience preferred
  • Expert level experience with at least one RDBMS and query language such as T-SQL, PL/SQL, Spark SQL
  • Expert level experience in conceptual, logical, and physical data design
  • Certifications within Azure such as Azure Fundamentals, Azure Data Engineer, Azure Data Scientist, and/or Azure DevOps Engineer preferred
  • Familiarity with multi-cloud data management and transformation platforms or tools such as Databricks, Snowflake etc.
Job Responsibility
Job Responsibility
  • Act as a Subject Matter Expert within the Enterprise Data & Analytics domain and related fields, ensuring that Master Data Management (MDM) and data engineering solutions adhere to standards of safety, security, compliance, and reliability
  • Design, develop, and implement key components of the enterprise data platform in alignment with strategic data and analytics goals
  • Assess development and support needs, assume accountability for extensive design initiatives, and play an essential role in delivering successful project outcomes
  • Collaborate directly with business data stewards, analysts, engineers, enterprise architects, and business stakeholders to refine requirements and shape the objectives of enterprise data solutions
  • Lead a high-performing, centralized team of engineers, analysts, and QA engineers, developing solutions that address the complete data lifecycle management needs of our enterprise data platforms
  • Mentor and guide team members, fostering a culture of excellence and continuous learning
  • Lead the design, development, and maintenance of MDM solutions using Profisee, including data models, workflows, business rules, and user interfaces
  • Develop and implement data quality rules and processes within Profisee to ensure data accuracy and consistency
  • Design, develop, and implement match and merge functionalities, as well as hierarchical structures, utilizing Profisee
  • Establish and implement survivorship rules, business rules, and workflow automations
What we offer
What we offer
  • medical, dental, and vision coverage
  • a 401(k) with company match
  • short-term disability
  • life insurance with AD&D
Read More
Arrow Right

Principal Engineer I – Master Data Management Engineer

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Irving, TX
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years’ experience with MDM solutions using Profisee or any other MDM tools
  • 3+ years’ experience designing, implementing, and supporting cloud data solutions
  • Azure Data Lake Storage, Azure Data Factory, Azure Data Services, Azure Logic Apps etc.
  • Bachelor’s degree in related field required
  • Masters or MBA in related field preferred
  • Previous leadership experience preferred
  • Expert level experience with at least one RDBMS and query language such as T-SQL, PL/SQL, Spark SQL
  • Expert level experience in conceptual, logical, and physical data design
  • Certifications within Azure such as Azure Fundamentals, Azure Data Engineer, Azure Data Scientist, and/or Azure DevOps Engineer preferred
  • Familiarity with multi-cloud data management and transformation platforms or tools such as Databricks, Snowflake etc.
Job Responsibility
Job Responsibility
  • Act as a Subject Matter Expert within the Enterprise Data & Analytics domain and related fields, ensuring that Master Data Management (MDM) and data engineering solutions adhere to standards of safety, security, compliance, and reliability
  • Design, develop, and implement key components of the enterprise data platform in alignment with Western Alliance Bank’s strategic data and analytics goals
  • Assess development and support needs, assume accountability for extensive design initiatives, and play an essential role in delivering successful project outcomes
  • Collaborate directly with business data stewards, analysts, engineers, enterprise architects, and business stakeholders to refine requirements and shape the objectives of enterprise data solutions
  • Lead a high-performing, centralized team of engineers, analysts, and QA engineers, developing solutions that address the complete data lifecycle management needs of our enterprise data platforms
  • Mentor and guide team members, fostering a culture of excellence and continuous learning
  • Lead the design, development, and maintenance of MDM solutions using Profisee, including data models, workflows, business rules, and user interfaces
  • Develop and implement data quality rules and processes within Profisee to ensure data accuracy and consistency
  • Design, develop, and implement match and merge functionalities, as well as hierarchical structures, utilizing Profisee
  • Establish and implement survivorship rules, business rules, and workflow automations
What we offer
What we offer
  • medical, dental, and vision coverage
  • a 401(k) with company match
  • short-term disability
  • life insurance with AD&D
Read More
Arrow Right

Principal Engineer I – Master Data Management Engineer

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Irving
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years' experience with MDM solutions using Profisee or any other MDM tools
  • 3+ years' experience designing, implementing, and supporting cloud data solutions
  • Azure Data Lake Storage, Azure Data Factory, Azure Data Services, Azure Logic Apps etc.
  • Bachelor's degree in related field required
  • Masters or MBA in related field preferred
  • Previous leadership experience preferred
  • Expert level experience with at least one RDBMS and query language such as T-SQL, PL/SQL, Spark SQL
  • Expert level experience in conceptual, logical, and physical data design
  • Certifications within Azure such as Azure Fundamentals, Azure Data Engineer, Azure Data Scientist, and/or Azure DevOps Engineer preferred
  • Familiarity with multi-cloud data management and transformation platforms or tools such as Databricks, Snowflake etc.
Job Responsibility
Job Responsibility
  • Act as a Subject Matter Expert within the Enterprise Data & Analytics domain and related fields, ensuring that Master Data Management (MDM) and data engineering solutions adhere to standards of safety, security, compliance, and reliability
  • Design, develop, and implement key components of the enterprise data platform in alignment with strategic data and analytics goals
  • Assess development and support needs, assume accountability for extensive design initiatives, and play an essential role in delivering successful project outcomes
  • Collaborate directly with business data stewards, analysts, engineers, enterprise architects, and business stakeholders to refine requirements and shape the objectives of enterprise data solutions
  • Lead a high-performing, centralized team of engineers, analysts, and QA engineers, developing solutions that address the complete data lifecycle management needs of our enterprise data platforms
  • Mentor and guide team members, fostering a culture of excellence and continuous learning
  • Lead the design, development, and maintenance of MDM solutions using Profisee, including data models, workflows, business rules, and user interfaces
  • Develop and implement data quality rules and processes within Profisee to ensure data accuracy and consistency
  • Design, develop, and implement match and merge functionalities, as well as hierarchical structures, utilizing Profisee
  • Establish and implement survivorship rules, business rules, and workflow automations
What we offer
What we offer
  • medical, dental, and vision coverage
  • a 401(k) with company match
  • short-term disability
  • life insurance with AD&D
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

Our Client is seeking a Principal Data Engineer to roll up their sleeves and bui...
Location
Location
United States , New York
Salary
Salary:
175000.00 - 220000.00 USD / Year
solomonpage.com Logo
Solomon Page
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong foundation in software engineering principles, including SOLID design, modularity, and scalability.
  • Expert proficiency in Databricks, including Delta Lake, Unity Catalog, Delta Live Tables, MLflow, and Databricks Workflows.
  • Deep experience with Snowflake, including data modeling, performance optimization, and integration with upstream/downstream pipeline tooling.
  • Expert proficiency in Python for data pipeline and automation development.
  • Advanced SQL skills with experience optimizing complex queries and data models at scale.
  • Proven experience designing and maintaining cloud-native data pipelines on AWS (e.g., MWAA/Airflow, Lambda, ECS, SQS, Glue, S3, Redshift, etc.).
  • Experience implementing and managing Terraform or similar IaC frameworks.
  • Strong understanding of lakehouse architecture patterns, data ingestion, transformation, and orchestration, including familiarity with ML/AI pipeline integration patterns.
  • Familiarity with CI/CD pipelines, automated testing, and modern DevOps practices.
  • 8+ years of experience in data engineering or backend development, with a focus on scalable data solutions.
Job Responsibility
Job Responsibility
  • Build and maintain end-to-end data pipelines — from raw ingestion through transformation and delivery — across diverse data sources (APIs, web data, internal feeds, etc.).
  • Hands-on development of scalable, production-grade pipelines within Databricks, including Delta Lake table management, Workflows, and cluster optimization.
  • Build and maintain data models, schemas, and transformation logic in Snowflake, optimizing for performance and reliability.
  • Develop and manage Databricks environments including Unity Catalog, Delta Live Tables, and integration patterns that support both internal data consumers and external sharing use cases.
  • Build and manage orchestration workflows using AWS services (MWAA/Airflow, Lambda, ECS, SQS, MSK) and Databricks-native orchestration where appropriate.
  • Implement and maintain infrastructure as code (IaC) using Terraform, ensuring reproducibility and compliance with cloud standards.
  • Establish and enforce best practices in data modeling, schema design, and ETL/ELT processes for high-volume structured and semi-structured data across Snowflake and Databricks.
  • Ensure data quality, lineage, and observability through automated testing, monitoring, and alerting across all pipeline layers.
  • Collaborate closely with technology leadership to align data platform development with business strategy and product goals.
  • Stay at the forefront of industry trends in data engineering, lakehouse architecture, and cloud-native data platforms.
  • Fulltime
Read More
Arrow Right

Principal Data Engineer

Principal Data Engineer. This role has been designed as 'Onsite' with an expecta...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in Data Engineering, with demonstrated architectural ownership
  • Expert-level experience with Snowflake (mandatory), including performance optimization, data modeling, security, and ecosystem components
  • Expert proficiency in SQL and strong Python skills for pipeline development and automation
  • Experience with modern orchestration tools (Airflow, Dagster, Prefect, or equivalent)
  • Strong understanding of ELT/ETL patterns, distributed processing, and data lifecycle management
  • Familiarity with streaming/event technologies (Kafka, Kinesis, Pub/Sub, etc.)
  • Experience implementing data quality, observability, and lineage solutions
  • Solid understanding of cloud infrastructure (AWS, GCP, or Azure)
  • Strong background in DataOps practices: CI/CD, testing, version control, automation
  • Proven leadership in driving architectural direction and mentoring engineering teams
Job Responsibility
Job Responsibility
  • Architecture & Strategy: Own a significant portion of the architecture of our data platform, ensuring scalability, performance, reliability, and security
  • Define standards and best practices for data modeling, transformation, orchestration, governance, and lifecycle management
  • Evaluate and integrate modern data technologies and frameworks that align with our long-term platform strategy
  • Collaborate with engineering and product leadership to shape the technical roadmap
  • Engineering & Delivery: Design, build, and manage scalable, resilient data pipelines for batch, streaming, and event-driven workloads
  • Develop clean, high-quality data models and schemas to support analytics, BI, operational systems, and ML workflows
  • Implement data quality, lineage, observability, and automated testing frameworks
  • Build ingestion patterns for APIs, event streams, files, and third-party data sources
  • Optimize compute, storage, and transformation layers for performance and cost efficiency
  • Leadership & Collaboration: Serve as a senior technical leader and mentor within the data engineering team
What we offer
What we offer
  • Health & Wellbeing: comprehensive suite of benefits supporting physical, financial and emotional wellbeing
  • Personal & Professional Development: programs to help reach career goals
  • Unconditional Inclusion: inclusive work environment
  • Fulltime
Read More
Arrow Right