CrawlJobs Logo

Engineering Lead – Data

Australia, Sydney · Job Posted April 11, 2026
Apply Position
Job Link Share

Job Description

We are looking for an experienced software engineer and team leader to join our APAC Data Engineering Platform team. The team is responsible for developing Optiver’s next-generation data platform for research and real-time trading, covering orchestration, storage, access and governance of data at scale. This team delivers robust, consistent solutions for a range of stakeholders across the business including engineers, researchers and traders. Our Data Engineers develop solutions using a mix of third-party and in-house technologies, owning their changes end-to-end including understanding the business problem, design, implementation, testing, deployment and monitoring. As a leader you will be responsible for working with stakeholders across the company to define the team’s strategy and roadmap, and you will execute that vision with your team. Your leadership will directly enable a faster, simpler, and more scalable innovation cycle for Optiver globally. You will also be responsible for building the team and growing engineers to reach their full potential. This is a hybrid leadership and engineering role with meaningful hands-on time across requirements, architecture, and code.

Job Responsibility

  • Build and lead a high performance data engineering team, including managing performance, career development and delivery
  • Own and deliver the vision, strategy and roadmap for Optiver’s data platform
  • Architect platform components (orchestration, storage/table formats, metadata/governance, streaming) that scale across APAC and integrate globally
  • Partner with stakeholders (research, trading, developers, SRE, security) to prioritise and achieve cross-team changes
  • Standardise ingestion and access workflows, including schema management, lineage, permissions and audit
  • Collaborate with our Europe & US offices to converge patterns and shared services
  • Contribute hands-on to key designs and production code
  • Uphold high engineering standards and best practices

Requirements

  • 7+ years experience as a Software Engineer delivering production platforms
  • 2+ years of experience leading software engineering teams
  • Strong experience with distributed systems & reliability fundamentals, including partitioning, replication, consistency/availability tradeoffs
  • Expertise with one or more of Python/C++/Java/Rust
  • Solid API and system design skills
  • Data platform fundamentals, including batch/stream processing, table formats, object storage, SQL
  • Understanding of security and governance, such as encryption, authentication, and access control
  • Practical experience with streaming and batch data (e.g., Kafka + Schema Registry)
  • Experience making architectural decisions on IT solutions, system design & implementation
  • Excellent stakeholder communication and roadmap ownership

What we offer

  • A performance-based bonus structure
  • Training, mentorship and personal development opportunities
  • Daily breakfast, lunch and an in-house barista
  • Gym membership plus weekly in-house chair massages
  • Regular social events, including a company trip every two years
  • Guided relocation, a competitive relocation package and visa sponsorship where necessary

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Engineering Lead – Data

8 matching positions

Data Engineering Lead (Ab Initio), Vice President

Are you a senior data engineering expert ready to make an impact at a global fin...
Location
Location
United Kingdom , Belfast
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Software development experience in a Banking or Financial Technology environment
  • Extensive, hands-on experience developing complex ETL applications using the Ab Initio tool kit (GDE, Control Centre, Metadata Hub, etc.)
  • Deep expertise in RDBMS (preferably Oracle) and the ability to write advanced SQL to analyse and resolve complex data issues
  • Strong proficiency in UNIX environments and shell scripting
  • Proven experience across the full Software Development Lifecycle (SDLC) and deep familiarity with Agile methodologies and tools like Jira
  • Exceptional analytical and problem-solving skills, with the ability to take ownership, manage risks, and drive tasks to completion
  • Excellent communication and collaboration skills, with the ability to work effectively with business users and cross-functional teams
  • Bachelor's degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Design, develop, and optimize complex, high-performance ETL applications using the full Ab Initio tool kit (GDE, Control Centre, Metadata Hub) to meet business and functional requirements
  • Collaborate closely with business users, financial analysts, and cross-functional technology teams to understand requirements, provide effort estimates, and ensure timely delivery of solutions
  • Drive all phases of the software development lifecycle (SDLC) within an Agile methodology, including preparing handover documents, performing unit/SIT testing, and supporting UAT and production implementations
  • Utilize your SQL and RDBMS knowledge to investigate and analyze complex data issues, ensuring the integrity and reliability of financial data within the Oracle ledger
  • Innovate and improve processes by designing and creating automated utilities that enhance system performance and efficiency
  • Take ownership of technical issues by monitoring processes, resolving testing defects, and managing bug tracking in Jira
What we offer
What we offer
  • Generous holiday allowance starting at 27 days plus bank holidays
  • increasing with tenure
  • A discretional annual performance related bonus
  • Private medical insurance packages to suit your personal circumstances
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources
  • Fulltime
Read More
Arrow Right

Data Engineering Lead (Ab Initio), Vice President

Are you looking for a career move that will put you at the heart of a global fin...
Location
Location
United Kingdom , Belfast
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Software development experience in a Banking or Financial Technology environment
  • Extensive, hands-on experience developing complex ETL applications using the Ab Initio tool kit (GDE, Control Centre, Metadata Hub, etc.)
  • Deep expertise in RDBMS (preferably Oracle) and the ability to write advanced SQL to analyze and resolve complex data issues
  • Strong proficiency in UNIX environments and shell scripting
  • Proven experience across the full Software Development Lifecycle (SDLC) and deep familiarity with Agile methodologies and tools like Jira
  • Exceptional analytical and problem-solving skills, with the ability to take ownership, manage risks, and drive tasks to completion
  • Excellent communication and collaboration skills, with the ability to work effectively with business users and cross-functional teams
  • A Bachelor’s degree/University degree or equivalent practical experience.
Job Responsibility
Job Responsibility
  • Design, develop, and optimize complex, high-performance ETL applications using the full Ab Initio tool kit (GDE, Control Centre, Metadata Hub) to meet business and functional requirements
  • Collaborate closely with business users, financial analysts, and cross-functional technology teams to understand requirements, provide effort estimates, and ensure timely delivery of solutions
  • Drive all phases of the software development lifecycle (SDLC) within an Agile methodology, including preparing handover documents, performing unit/SIT testing, and supporting UAT and production implementations
  • Utilize your SQL and RDBMS knowledge to investigate and analyze complex data issues, ensuring the integrity and reliability of financial data within the Oracle ledger
  • Innovate and improve processes by designing and creating automated utilities that enhance system performance and efficiency
  • Take ownership of technical issues by monitoring processes, resolving testing defects, and managing bug tracking in Jira.
What we offer
What we offer
  • Generous holiday allowance starting at 27 days plus bank holidays
  • increasing with tenure
  • A discretional annual performance related bonus
  • Private medical insurance packages to suit your personal circumstances
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources
  • Fulltime
Read More
Arrow Right

Senior Squad Lead – Data Engineering

As Senior Squad Lead – Data Engineering, key responsibilities will include: Lead...
Location
Location
United Kingdom , Newcastle
Salary
Salary:
130000.00 GBP / Year
t2mresourcing.com Logo
T2M Resourcing
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Leading a pod/squad of 6 Data Engineers building data intensive application and pipelines
  • Broad experience in platform, deployment and hosting (AWS, Azure or GCP), as well as tools like Airflow, Spark and Databricks
  • Curious leader, with high EQ, who enjoys solving stakeholder problems
  • Track record of performant pod or squad leadership
  • Must be eligible for or already hold SC clearance
  • Sole UK nationals only
Job Responsibility
Job Responsibility
  • Leading a pod/squad of 6 Data Engineers building data intensive application and pipelines
  • Working on a range of defence applications from data analytics and Intelligence platforms, defence AI, drones to command-and-control systems
  • Fulltime
Read More
Arrow Right

Data Engineering Lead - Reporting and Analytics - Vice President

Are you looking for a career move that will put you at the heart of a global fin...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience in building high scalable, reliable dashboards covering suite of industry products like Tableau, AG Grid, Oracle and Java
  • Well versed with Front end - React, Angular and Back end Tech stack
  • Experience in all aspects of Data using Python for Reconciliations, Data Comparison, Data Quality and Data Security for SAAS or Cloud based platform
  • Relevant experience in leading development for enterprise scale platforms, products, or frameworks preferably using Unix/SQL/Oracle/NoSql/Netezza/Teradata
  • Strong experience in managing development activities for applications with huge Data and complex rules/transformations
  • Demonstrated experience in collaborating with different teams to ensure proactive Cross application/downstream impact analysis, responsible for creating test plans and strategy across multiple business critical applications in Finance
  • Experience with existing Citi applications, implementations and Configurations
Job Responsibility
Job Responsibility
  • Leading the buildout of Engineering tools, real time dashboards using Tableau, AG Grid, Java, Unix and Oracle
  • Build AI capabilities for all qualified used cases
  • Performing initiatives related to Development, Functional Testing, SIT, User Acceptance Testing (UAT) process and product rollout into production
  • You will be a specialist who works with technology project managers, UAT professionals and users to design and implement appropriate scripts/plans for an application testing strategy/approach
  • Responsibilities may also include software quality assurance testing
  • Analyses trends at an organizational level to improve data related processes
  • follows and analyses AI industry trends and how to fit those in Citi ecosystem
  • Responsible for handling day-to-day staff management issues, including resource management and allocation of work within the team/project
What we offer
What we offer
  • Generous holiday allowance starting at 27 days plus bank holidays
  • increasing with tenure
  • A discretional annual performance related bonus
  • Private medical insurance packages to suit your personal circumstances
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources
  • Fulltime
Read More
Arrow Right

Technical Lead, Data Engineering

The Technical Lead in Data Engineering at NTT DATA is an entry-level management ...
Location
Location
Malaysia , Selangor Darul Ehsan
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Basic proficiency in SQL and Python
  • Familiarity with ETL concepts and cloud platforms (AWS/Azure/GCP)
  • Introductory knowledge of handling unstructured data (JSON, text files, logs)
  • Strong communication and interpersonal skills
  • Ability to coordinate tasks and support team collaboration
  • Problem-solving mindset, adaptability, and willingness to learn
  • Bachelor’s degree in Computer Science, Information Technology, or related field
  • Relevant experience in data engineering or related technical role
  • Exposure to data pipeline development and basic project coordination
  • Some experience in developing and mentoring a team
Job Responsibility
Job Responsibility
  • Guide and support a small team of data engineers in their daily tasks
  • Assist with onboarding and provide basic mentoring under the direction of senior managers
  • Help plan and track progress of data engineering projects
  • Communicate priorities and timelines clearly to the team
  • and escalate risks and issues to senior management when needed
  • Contribute to building and maintaining data pipelines for structured and unstructured data
  • Apply best practices in ETL processes and data quality checks
  • and support troubleshooting and basic performance optimization
  • Work with analysts and business stakeholders to understand data needs
  • Translate simple business requirements into technical tasks with guidance from senior engineers
  • Fulltime
Read More
Arrow Right

Data Engineering Lead

Role: Data Engineering Lead. Location: New York, NY. FTE ONLY. Must Have Technic...
Location
Location
United States , New York
Salary
Salary:
138000.00 USD / Year
realign-llc.com Logo
Realign
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • AWS Data Engineering Services (EMR/Glue, Redshift, Aurora, S3, Lambda)
  • Spark
  • Python
  • Collibra
  • Snowflake/Databricks
  • Tableau
  • 4–15 years of professional data engineering experience
  • Strong Python, SQL, and Spark (PySpark) skills, and/or Kafka
  • Snowflake (Snowpipe, Tasks, Streams) as a complementary warehouse
  • Databricks (Delta formats, workflows, cataloging) or equivalent Spark platforms
Job Responsibility
Job Responsibility
  • Ingest and model data from APIs, files/SFTP, and relational sources
  • implement layered architectures (raw/clean/serving) using PySpark/SQL and dbt, Python
  • Design and operate pipelines with Prefect (or Airflow), including scheduling, retries, parameterization, SLAs, and well documented runbooks
  • Build on cloud data platforms, leveraging S3/ADLS/GCS for storage and a Spark platform (e.g., Databricks or equivalent) for compute
  • manage jobs, secrets, and access
  • Publish governed data services and manage their lifecycle with Azure API Management (APIM) authentication/authorization, policies, versioning, quotas, and monitoring
  • Enforce data quality and governance through data contracts, validations/tests, lineage, observability, and proactive alerting
  • Optimize performance and cost via partitioning, clustering, query tuning, job sizing, and workload management
  • Uphold security and compliance (e.g., PII handling, encryption, masking) in line with firm standards
  • Collaborate with stakeholders (analytics, AI engineering, and business teams) to translate requirements into reliable, production ready datasets
  • Fulltime
Read More
Arrow Right

Data Engineering Lead

We’re looking for a Lead Data Engineer who is ready to both lead and build. This...
Location
Location
Salary
Salary:
Not provided
5CA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Located in Albania, Bosnia and Herzegovina, Bulgaria, Greece, Hungary, Kosovo, North Macedonia, Poland, Romania, Serbia, South Africa, or Tunisia
  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field
  • Minimum 7 years’ experience in data engineering
  • At least 2 years of leadership experience managing or mentoring engineers
  • Proven experience in designing, building, and maintaining scalable ETL/ELT pipelines and data platforms
  • Proficiency in SQL, Python, and data modelling techniques
  • Experience with cloud-based data architectures (e.g., Azure, AWS, or GCP)
  • Strong understanding of data governance, quality frameworks, and security best practices
  • Experience integrating data from multiple sources, including APIs and third-party systems
  • Familiarity with modern data tools and technologies (e.g., Databricks, Snowflake, Apache Spark, Airflow, Microsoft Fabric)
Job Responsibility
Job Responsibility
  • Lead the strategy, design, and implementation of scalable, high-performance data pipelines, platforms, and architectures
  • Manage the Data Engineering team’s backlog, priorities, and capacity to ensure timely delivery of high-quality outputs
  • Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver fit-for-purpose solutions
  • Oversee and contribute to the development of ETL/ELT processes, data models, and data integration frameworks
  • Ensure data quality, consistency, and governance across all systems and datasets
  • Monitor and optimise data infrastructure for performance, scalability, and cost efficiency
  • Define and enforce engineering best practices, coding standards, and security guidelines
  • Evaluate and adopt new technologies, tools, and frameworks that can improve data engineering capabilities
  • Provide technical leadership and mentorship to data engineers, fostering a high-performance culture
  • Troubleshoot complex data issues and drive root cause analysis and remediation
What we offer
What we offer
  • A position at a fast-paced international company with ambitious gaming, e-commerce, and tech clients
  • A diverse and inclusive culture with people from 80+ countries, speaking 25+ languages - where we celebrate everyone's uniqueness
  • Innovative digital tools, and continuous opportunities for learning and development
  • Various learning and career development initiatives throughout the year
  • Fun employee engagement activities and participation in 5CA employee-led communities such as 5CA Connect, Pride, 5CA Gamers, Women of 5CA, to name a few
  • Fulltime
Read More
Arrow Right

Data Engineering Lead

The next step of your career starts here, where you can bring your own unique mi...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
metyis.com Logo
Metyis
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7-10 years of professional hands-on experience
  • A broad practice in multiple software engineering fields
  • Experience on managing & leading data engineering / warehousing projects
  • Must have experience on Python, SQL and Distributed programming languages, preferable Spark
  • Experience working on Cloud, Azure is a plus
  • Experience working setting up data lakes using entire Big Data and DWH ecosystem
  • Experience on data workflows and ETL, Apache Airflow is a plus
  • Comfortable with Unix OS type systems, Bash and Linux
  • Basic knowledge of webservices and APIs
  • Basic knowledge of Containers and Docker as platform
Job Responsibility
Job Responsibility
  • Independently lead & manage execution of data engineering projects
  • Engineer complete technical solutions to solve concrete business challenges in the areas of digital marketing, eCommerce, Business Intelligence and self-service analytics
  • Collect functional and non-functional requirements, consider technical environments, business constraints and enterprise organizations
  • Support our clients in executing their Big Data strategies by designing and building operational data platforms: ETL pipelines, data anonymization pipelines, data lakes, near real-time streaming data hubs, web services, training and scoring machine learning models
  • Troubleshoot and quality check work done by team members
  • Collaborate closely with partners, strategy consultants and data scientists in a flat and agile organization where personal initiative is highly valued
  • Share data engineering knowledge by giving technical trainings
  • Communicate and interact with clients at the executive level
  • Guide and mentor team members
What we offer
What we offer
  • Interact with C-level at our clients on regular basis to drive their business towards impactful change
  • Lead your team in creating new business solutions
  • Seize opportunities at the client and at Metyis in our entrepreneurial environment
  • Become part of a fast-growing international and diverse team
Read More
Arrow Right