Data Engineering Lead Job at Orbis Consultants (London)

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...

Location

United Arab Emirates , Dubai

Salary:

Not provided

NorthBay

Expiration Date

Until further notice

Requirements

8+ years of experience in data engineering and data platform development
Strong hands-on experience with: AWS Glue
Amazon EMR (Spark)
AWS Lambda
Apache Airflow (MWAA)
Amazon EC2
Amazon CloudWatch
Amazon Redshift
Amazon DynamoDB
AWS DataZone

Job Responsibility

Design, develop, and optimize scalable data pipelines using AWS native services
Lead the implementation of batch and near-real-time data processing solutions
Architect and manage data ingestion, transformation, and storage layers
Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
Orchestrate complex data workflows using Apache Airflow (MWAA)
Develop and manage serverless data processing using AWS Lambda
Design and optimize data warehouses using Amazon Redshift
Implement and manage NoSQL data models using Amazon DynamoDB
Utilize AWS DataZone for data governance, cataloging, and access management
Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch

Fulltime

Lead Data Engineer

Location

Uzbekistan , Tashkent

Salary:

Not provided

Vention

Expiration Date

Until further notice

Requirements

4+ years in data engineering, including leading teams
Strong experience with Python and SQL
Solid knowledge of Apache Airflow, Kafka, Big Query, and AWS/Azure
Strong experience with ETL processes, data warehousing, and stream processing
Leadership skills with proven ability to mentor and grow engineering teams
Experience working in an Agile environment (Scrum, Kanban, etc.)
B2+ English, with experience communicating with English-speaking customers

Job Responsibility

Guide a team of data engineers in building and optimizing data pipelines
Oversee architecture for data ingestion, transformation, and storage with Big Query and SQL, ensuring high performance and reliability
Collaborate with product managers and clients to define data strategies and resolve complex technical challenges
Stay up-to-date with the latest cloud data technologies and industry best practices, bringing innovation to our data ecosystem

What we offer

EDU corporate community (300+ members): tech communities, interest clubs, events, a small R&D lab, a knowledge base, and a dedicated AI track
Licenses for AI tools: GitHub Copilot, Cursor, and others
Expanded medical support for employees in Tashkent
19 working days of vacation per year, 21 after two years in the company
Corporate getaway & teambuilding activities
Support for the significant events in your life
Referral bonuses for bringing in new talent

Fulltime

Lead Data Engineer

Embark on an exciting journey into the realm of software product development wit...

Location

India , Noida

Salary:

Not provided

3Pillar Global

Expiration Date

Until further notice

Requirements

Demonstrated expertise with a minimum of 8 years of relevant experience in data engineering with experience of leading and managing a technical support or data engineering team
Ability to function as a player-coach — leading while also contributing hands-on
Proficiency in Python for data engineering, ETL/ELT workflows, and automation tasks
Solid understanding of data governance, security, and performance optimization practices
Experience designing, creating and maintaining data pipelines in AWS environments, with strong exposure to AWS Data services like S3, Glue, Lambda, and Step Functions, and other related services
Data Architecture: Expert-level understanding of ETL, data warehouse design, and pipeline optimization
Database Expertise: Deep experience with Teradata, Change Data Capture (CDC) processes, and data synchronization
Integration: Experience with Lambda-based integrations, such as triggering SAS or RMJ events
Strong communication skills and ability to translate complex technical concepts for business stakeholders
Resilience: Ability to make critical, autonomous decisions and lead a team in a high-pressure environment

Job Responsibility

Team Management: Manage the on-site team of 3-4 Data Engineers
Roster & Scheduling: Create, manage, and maintain team rosters and schedules to ensure full 8-hour coverage, 7 days a week, including rotational weekend and holiday coverage
Availability: Must be able to work a day shift and manage a team that provides 7-day-a-week coverage, including weekends and holidays on rotation
Coverage Assurance: Handle all team leave requests (planned and unplanned). You are responsible for ensuring coverage is always maintained, especially during medical or other unexpected absences
Performance & SLAs: Act as the primary operational point of contact, ensuring the team executes all tasks efficiently, makes sound decisions, and adheres to all agreed-upon SLAs
Final Escalation Point: Serve as the final technical escalation point for the team
personally handle the most critical job failures and system issues
Expert RCA: Lead all high-priority Root Cause Analysis (RCA) efforts, documenting solutions and implementing permanent fixes to prevent recurrence
Strategic Improvement: Drive the team's efforts in performance tuning and AWS Glue cost optimization
Automation Strategy: Define and lead the development of new automation jobs (using Glue, Lambda, and Step Functions) to reduce manual support tasks

Fulltime