CrawlJobs Logo

Tech Lead, Data Pipeline

United Kingdom, London · Job Posted January 10, 2026
Apply Position
Job Link Share

Job Description

As the Technical Lead for Data Pipeline Features within our Model Development Platform, you will play a pivotal role in Wayve's mission to revolutionize autonomous driving. Each day, our teams handle and process multiple petabytes of data collected from our fleet of autonomous vehicles and critical partner integrations. Your technical leadership will directly impact our ability to transform vast amounts of raw data from diverse internal and external sources into structured, actionable insights that power advanced machine learning models and groundbreaking research. By continuously innovating our data ingestion and processing pipelines, you'll help accelerate the development of safer, more efficient autonomous driving technologies, enabling Wayve to maintain its position at the forefront of machine learning innovation.

Job Responsibility

  • Define and execute a strategic technical roadmap for enhancing and scaling data pipeline capabilities
  • Drive innovation in pipeline architecture to support dynamic and evolving use-cases
  • Lead the design and implementation of advanced data pipeline features for efficient data ingestion, transformation, and distribution
  • Normalize and unify multiple disparate data sources - including data ingested from external partners - into a consistent and optimized format tailored specifically for AI training and research needs
  • Collaborate closely with robotics, ML engineering, and research teams to continuously enhance pipeline performance and functionality
  • Develop robust interfaces and systems to reduce bottlenecks and improve reliability and scalability
  • Establish and maintain best practices for pipeline reliability, including comprehensive observability, alerting, and monitoring systems
  • Manage initiatives aimed at minimizing pipeline latency, failure recovery, and ensuring compliance with defined service-level agreements (SLAs)
  • Engage actively with cross-functional teams (robotics, ML, data governance) to ensure alignment of technical efforts with overall business goals
  • Foster a culture of transparency, collaboration, and shared ownership across teams
  • Mentor and develop engineering talent, promoting professional growth and technical excellence within your team
  • Contribute to the hiring and onboarding processes to expand the capabilities of the data pipeline engineering team

Requirements

  • Strong experience (8+ years) in software engineering, specifically focused on developing scalable, complex data pipelines
  • Proven technical leadership experience in a pipeline engineering or related domain
  • Expertise in modern data pipeline architectures, including DAG-based orchestration (e.g., Airflow, Flyte)
  • Solid understanding of data engineering practices, distributed processing frameworks, and pipeline optimization techniques
  • Excellent communication and collaborative skills, capable of working effectively with interdisciplinary teams
  • Track record of mentorship and talent development
  • Bachelor's degree or higher in Computer Science, Engineering, or related technical discipline

Nice to have

  • Experience with robotics or autonomous vehicle sensor data processing pipelines
  • Familiarity with third-party dataset ingestion and transformation
  • Understanding of compliance and data governance frameworks (e.g., GDPR, TSAX)
  • Hands-on experience integrating observability and monitoring solutions

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Tech Lead, Data Pipeline

8 matching positions

Tech Lead - Data Solutions & Data Products

The Tech Lead – Data Solutions & Data Products is a senior technical leader resp...
Location
Location
India , Hyderabad; Bangalore
Salary
Salary:
Not provided
berettaclima.it Logo
Beretta Clima Italia
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6 to 10 years of experience in data engineering, data architecture, or data platform roles
  • Proven experience architecting and delivering data solutions on AWS and GCP
  • Strong hands‑on experience building production‑grade data pipelines and data products
  • Deep understanding of: Data modeling and analytics patterns
  • Batch and streaming architectures
  • Cloud‑native design principles
  • Strong programming skills (e.g., Python, SQL) and experience with modern data frameworks
  • Ability to make sound architectural decisions under real‑world constraints
Job Responsibility
Job Responsibility
  • Data Solution Architecture (AWS & GCP)
  • Data Product Development Leadership
  • Technical Guidance & Team Leadership
  • Platform & Engineering Collaboration
  • Non‑Functional Excellence
  • Stakeholder Engagement & Technical Communication
What we offer
What we offer
  • Retirement savings plan
  • Health insurance
  • Flexible schedules
  • Parental leave
  • Holiday purchase scheme
  • Professional development opportunities
  • Employee Assistance Programme
  • Fulltime
Read More
Arrow Right

Salesforce Tech Lead - Data & AI

We are seeking a visionary and hands-on Senior Engineer / Tech Lead to lead the ...
Location
Location
United States , Boston; Plano; St. Petersburg; Atlanta; Morristown
Salary
Salary:
127000.00 - 169000.00 USD / Year
zelis.com Logo
Zelis
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of experience in software engineering, with a strong focus on Salesforce and data-intensive applications
  • Proven experience integrating Salesforce with enterprise data platforms and building AI/ML-enabled features
  • Hands-on experience with Salesforce Data Cloud and agent frameworks like Agentforce
  • Proficiency in Apex, Lightning Components, Salesforce APIs, and data engineering tools
  • Experience with AI/ML platforms (e.g., Azure ML, AWS SageMaker, Einstein AI) and model deployment strategies
  • Strong understanding of data architecture, governance, and security
  • Excellent communication and stakeholder management skills
Job Responsibility
Job Responsibility
  • Design and develop intelligent solutions using Salesforce Data Cloud to unify, activate, and analyze customer data across channels
  • Lead the deployment and orchestration of autonomous agents, such as Agentforce, to automate workflows, enhance user experiences, and drive operational efficiency
  • Architect AI/ML-enabled features within Salesforce, including predictive analytics, intelligent routing, and generative AI applications
  • Build scalable data pipelines and APIs to support real-time and batch data flows between Salesforce and other enterprise systems
  • Collaborate with data scientists and analysts to operationalize models within Salesforce workflows and ensure business impact
  • Own the full development lifecycle: from design and prototyping to implementation, testing, deployment, and support
  • Ensure data governance, security, and compliance across all integrations and AI applications
  • Establish and enforce engineering best practices, including CI/CD, testing, documentation, and performance optimization
  • Mentor junior engineers and foster a culture of innovation and technical excellence
  • Stay current with Salesforce platform updates, AI/ML trends, and data technologies to inform roadmap and strategy
What we offer
What we offer
  • 401k plan with employer match
  • flexible paid time off
  • holidays
  • parental leaves
  • life and disability insurance
  • health benefits including medical, dental, vision, and prescription drug coverage
  • Fulltime
Read More
Arrow Right

Data Engineer Tech Lead

We are seeking a Data Engineer Tech Lead to join our capital markets data engine...
Location
Location
Israel , Tel Aviv
Salary
Salary:
Not provided
jll.com Logo
JLL
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Engineering experience with strong expertise in Apache Spark and distributed computing
  • Strong programming skills in Python including data handling libraries (pandas, numpy, etc.), and data modeling
  • Proficient in Databricks platform and Delta Lake for data lake architecture
  • Proficient in writing complex SQL queries
  • Experience with cloud-based databases (Google BigQuery \ Snowflake)
  • Advanced SQL skills and experience with both relational and NoSQL databases
  • Bachelor's degree in Computer Science, Engineering or related field
  • Experience with integration of multiple data sources and working with various databases technologies
Job Responsibility
Job Responsibility
  • Design and implement robust, scalable data pipelines using Databricks, Apache Spark, and Delta Lake as well as BigQuery
  • Use SQL and Python to develop, scale, and optimize advanced data pipelines
  • Build and optimize ETL/ELT processes for capital markets data
  • Develop real-time and batch processing solutions to support trading and risk management operations
  • Implement data quality monitoring, validation, and alerting systems
  • Configure and optimize Databricks workspaces, clusters, and job scheduling
  • Work in a Multi-cloud environment including Azure, GCP and AWS
  • Implement security best practices including access controls, encryption, and audit logging
  • Build integrations with market data vendors, trading systems, and risk management platforms
  • Establish monitoring and performance tuning for data pipeline health and efficiency
  • Fulltime
Read More
Arrow Right

Data Engineering Tech Lead

Location
Location
Poland
Salary
Salary:
Not provided
lingarogroup.com Logo
Lingaro
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 5 years of experience in data engineering or a related field
  • Strong technical skills in data engineering, including proficiency in programming languages such as Python, SQL and Scala
  • Familiarity with Azure cloud platform and services, and experience in implementing data solutions in a cloud environment
  • Expertise in working with various data tools and technologies, such as ETL frameworks, data pipelines, and data warehousing solutions
  • Proven experience in leading and managing a team of data engineers, providing guidance, mentorship, and technical support
  • In-depth knowledge of data management principles and best practices, including data governance, data quality, and data integration
  • Strong project management skills, with the ability to prioritize tasks, manage timelines, and deliver high-quality results within designated deadlines
  • Excellent problem-solving and analytical skills, with the ability to identify and resolve complex data engineering issues
  • Knowledge of data security and privacy regulations, and the ability to ensure compliance within data engineering projects
  • Excellent communication and interpersonal skills, with the ability to effectively collaborate with cross-functional teams, stakeholders, and senior management
Job Responsibility
Job Responsibility
  • Provide leadership and guidance to the data engineering team, including mentoring, coaching, and fostering a collaborative work environment
  • Set clear goals, assign tasks, and manage resources to ensure successful project delivery
  • Work closely with developers to support them and improve data engineering processes
  • Support team members with troubleshooting and resolving complex technical issues and challenges
  • Provide technical expertise and direction in data engineering, guiding the team in selecting appropriate tools, technologies, and methodologies
  • Stay updated with the latest advancements in data engineering and ensure the team follows best practices and industry standards
  • Collaborate with stakeholders to understand project requirements, define scope, and create project plans
  • Support project managers to ensure that projects are executed effectively, meeting timelines, budgets, and quality standards
  • Monitor progress, identify risks, and implement mitigation strategies
  • Act as a trusted advisor for the customer
What we offer
What we offer
  • Stable employment
  • “Office as an option” model
  • Workation
  • Great Place to Work® certified employer
  • Flexibility regarding working hours and your preferred form of contract
  • Comprehensive online onboarding program with a “Buddy” from day 1
  • Cooperation with top-tier engineers and experts
  • Unlimited access to the Udemy learning platform from day 1
  • Certificate training programs
  • Upskilling support
  • Fulltime
Read More
Arrow Right

Data Engineering Tech Lead-Azure

Location
Location
India
Salary
Salary:
Not provided
lingarogroup.com Logo
Lingaro
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A bachelor's or master’s degree in computer science, Information Systems, or a related field is typically required
  • Minimum of 8-10 years of experience in data engineering or a related field
  • Strong technical skills in data engineering, including proficiency in programming languages such as Python, SQL, R or Scala
  • Practical experience with Microsoft Azure cloud and Databricks platform
  • Expertise in working with various data tools and technologies, such as ETL frameworks, data pipelines, and data warehousing solutions
  • Proven experience in leading and managing a team of data engineers
  • In-depth knowledge of data management principles and best practices, including data governance, data quality, and data integration
  • Strong project management skills
  • Excellent problem-solving and analytical skills
  • Knowledge of data security and privacy regulations
Job Responsibility
Job Responsibility
  • Efficient and effective project delivery is the primary responsibility of the tech lead
  • Provide leadership and guidance to the data engineering team, including mentoring, coaching, and fostering a collaborative work environment
  • Set clear goals, assign tasks, and manage resources to ensure successful project delivery
  • Work closely with developers to support them and improve data engineering processes
  • Support team members with troubleshooting and resolving complex technical issues and challenges
  • Utilize and promote Generative AI tools to accelerate project delivery
  • Provide technical expertise and direction in data engineering
  • Collaborate with stakeholders to understand project requirements, define scope, and create project plans
  • Support project managers to ensure that projects are executed effectively, meeting timelines, budgets, and quality standards
  • Act as a trusted advisor for the customer
What we offer
What we offer
  • Stable employment
  • 100% remote
  • Flexibility regarding working hours
  • Comprehensive online onboarding program with a “Buddy” from day 1
  • Cooperation with top-tier engineers and experts
  • Unlimited access to the Udemy learning platform from day 1
  • Certificate training programs
  • Upskilling support
  • Grow as we grow as a company
  • A diverse, inclusive, and values-driven community
  • Fulltime
Read More
Arrow Right
New

Tech lead - Java/AI Tech Lead/Sr Developer

At Schwab, you're empowered to make an impact on your career.Here, innovative th...
Location
Location
United States , Austin
Salary
Salary:
150000.00 - 175000.00 USD / Year
schwab.com Logo
Charles Schwab
Expiration Date
July 10, 2026
Flip Icon
Requirements
Requirements
  • Demonstrated exposure to, or hands-on experience with, GenAI coding assistants used across day-to-day engineering workflows in the SDLC—implementation, refactoring, unit testing, regression support, code reviews, scripting/automation, troubleshooting, and documentation—while applying engineering judgment and validation. Practical familiarity with tools such as GitHub Copilot or Claude Code is expected.
  • Uses AI assistance for common dev tasks (code suggestions, test generation, review support, automation scripts, troubleshooting) with clear ownership for correctness and quality.
  • Applies GenAI in ways that reduce context switching and accelerate delivery across typical workflows (not limited to writing new code).
  • Demonstrates practical familiarity with GitHub Copilot and/or Claude Code within IDE/CLI-style workflows.
  • Working knowledge of agentic workflows, spec-driven development (translating low-level design artifacts such as class structures, API contracts, and data models into structured specs that guide AI-assisted implementation), and custom instructions and prompt engineering, with the ability to establish team-level practices for effective AI-assisted development.
  • Proven track record with 2-3+ years of hands-on experience in data modeling.
  • 3+ years of experience in building ETL (Extract, Transform, Load) data pipelines both on-premises and in the cloud. GoldenGate, Debezium, Airflow.
  • 2+ years of experience working on Google Data flow for batch, Google Composer for batch orchestration on Google Cloud Platform.
  • 7+ years of experience in actively developing large scale software using: Java or Python and Spring/Spring Boot framework.
  • 3+ years of experience working with containers and microservices in the cloud.
Job Responsibility
Job Responsibility
  • Provide technical guidance to a team that will design, develop, and implement enterprise data integration solutions.
  • Partner with business stakeholders and development teams to design and develop segmentation solutions for various stakeholders to use for analytics and reporting.
What we offer
What we offer
  • 401(k) with company match and Employee stock purchase plan
  • Paid time for vacation, volunteering, and 28-day sabbatical after every 5 years of service for eligible positions
  • Paid parental leave and family building benefits
  • Tuition reimbursement
  • Health, dental, and vision insurance
  • Medical, dental and vision benefits
  • 401(k) and employee stock purchase plans
  • Tuition reimbursement to keep developing your career
  • Paid parental leave and adoption/family building benefits
  • Sabbatical leave available after five years of employment
  • Fulltime
!
Read More
Arrow Right
New

Lead Data Engineer - Data Transformation (Modeling and Architecture)

Lead Data Engineer - Data Transformation (Modeling and Architecture). Do you lov...
Location
Location
United States , Richmond, Virginia; McLean, Virginia; Cambridge, Massachusetts
Salary
Salary:
179400.00 - 225100.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree
  • At least 4 years of experience in application development
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Job Responsibility
Job Responsibility
  • Build and maintain comprehensive data models—spanning conceptual, logical, and physical layers—to ensure scalable architecture and high data integrity across enterprise systems
  • Lead design of the org data landscape by applying Consumer Driven design principles, ensuring that data structures reflect business realities and evolving organizational needs
  • Architect and implement robust data ecosystem solutions, including Data Lake and Data Warehouse patterns, to support diverse analytical and operational requirements
  • Support high-performance data pipelines and complex transformations that utilize SQL, Spark, and Python to process large-scale datasets efficiently
  • Define and Enforce rigorous data governance standards while managing metadata frameworks to ensure data compliance and discoverability
  • Translate complex technical concepts into actionable business insights, working independently to lead initiatives and collaborate with stakeholders to meet organizational goals
  • Contribute to the evolution of the data ecosystem by designing AI-ready architectures
  • Collaborate with and across Agile teams to design, develop, implement, and support technical solutions
  • Work with a team of developers with deep experience in machine learning, AI, distributed microservices, and full stack systems
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
What we offer
What we offer
  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • Comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right

Tech Lead / Full Stack Lead

We are looking for an experienced Tech Lead / Full Stack Lead to design and lead...
Location
Location
Armenia
Salary
Salary:
Not provided
itransition.com Logo
Itransition
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of software development experience
  • 2+ years in a Tech Lead / Senior Full Stack role
  • Strong hands-on experience with Node.js (NestJS / Express or similar) and React (Next.js preferred)
  • Strong understanding of: E-commerce architecture (catalog, cart, checkout, orders)
  • marketplace mechanics (multi-role systems)
  • API-first architecture
  • System Design: Proven experience with microservices architecture and building event-driven systems
  • API Design: Strong proficiency in designing and implementing REST and/or GraphQL APIs
  • Data Management: Hands-on experience with PostgreSQL or other relational databases, including schema design and optimization
  • Caching & Performance: Practical knowledge of Redis and the implementation of effective caching strategies
Job Responsibility
Job Responsibility
  • Lead the architecture and development of the platform based on MedusaJS
  • Customize and extend MedusaJS to support complex business flows
  • Design scalable backend services and APIs
  • Participate in frontend architecture (React-based applications)
  • Decompose the system into clear domains and services
  • Ensure scalability, performance, and reliability of the platform
  • Define and enforce coding standards and best practices
  • Lead and mentor the development team
  • Collaborate closely with Product Owner / Business Analyst
  • Review code and make key technical decisions
What we offer
What we offer
  • Projects for such clients as PayPal, Wargaming, Xerox, Philips, adidas and Toyota
  • Competitive compensation that depends on your qualification and skills
  • Career development system with clear skill qualifications
  • Flexible working hours aligned to your schedule
  • Options to work remotely
  • Corporate medical insurance covering services of private and public medical centers
  • English courses online
  • Corporate parties and events for employees and their children
  • Gym membership compensation, corporate sport competitions (cybersport included)
  • 5 days of paid sick leave per year with no obligation to submit a sick-leave certificate
  • Fulltime
Read More
Arrow Right