This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
Embark on an exciting journey into the realm of software development with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.
Job Responsibility:
Manage the team of 3-4 Data Engineers
Create, manage, and maintain team rosters and schedules to ensure full 8-hour coverage, 7 days a week, including rotational weekend and holiday coverage
Must be able to work a day shift and manage a team that provides 7-day-a-week coverage, including weekends and holidays on rotation
Handle all team leave requests (planned and unplanned)
You are responsible for ensuring coverage is always maintained, especially during medical or other unexpected absences
Act as the primary operational point of contact, ensuring the team executes all tasks efficiently, makes sound decisions, and adheres to all agreed-upon SLAs
Serve as the final technical escalation point for the team
personally handle the most critical job failures and system issues
Lead all high-priority Root Cause Analysis (RCA) efforts, documenting solutions and implementing permanent fixes to prevent recurrence
Drive the team's efforts in performance tuning and AWS Glue cost optimization
Define and lead the development of new automation jobs (using Glue, Lambda, and Step Functions) to reduce manual support tasks
Oversee all data integrity and synchronization tasks, including CDC reloads and write-backs to external databases like Teradata
Own the knowledge transfer process to internal resources and ensure the team maintains excellent documentation and runbooks
Requirements:
In-depth, expert-level knowledge and extensive hands-on experience with AWS S3, Glue, Lambda, Step Functions and other related services
Demonstrated expertise with a minimum of 5-8 years of relevant experience
Deep understanding of Data engineering, ETL principles and data warehouse design
Demonstrable experience in system performance tuning and cloud cost optimization, specifically with AWS Glue
Proven ability to design and develop automation solutions from the ground up
Expert-level troubleshooting and RCA skills
Ability to work independently, make critical decisions, and provide technical guidance
Must be able to work a day shift that includes coverage for weekends and holidays on a rotational basis
Strong experience with Change Data Capture (CDC), managing database synchronization, and working with Teradata