This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
The Data Engineer will be responsible for end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse. This role requires a minimum of 3-5 years of hands-on experience in data engineering, with proficiency in Python, Kafka, and Snowflake. The candidate will engage with stakeholders to ensure data integrity and quality during the migration process.
Job Responsibility:
Perform end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse
Pipeline migration including refactoring and migrating extraction logic and job scheduling
Data Transfer executing physical migration of underlying datasets while ensuring data integrity
Stakeholder Engagement acting as a technical liaison to internal clients
Code Conversion translating and optimizing legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg
Usage analysis to understand usage patterns to deliver required data products
Data Reconciliation and Quality with a rigorous approach to data validation
Work with other internal data management platform
Requirements:
Bachelor or Master degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
Ability to troubleshoot (SQL) and basic scripting experience
Professional proficiency in Python or Java
Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices and K8s deployment experience
Demonstrated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, Architectural Theory
Experience with Kafka, ANSI SQL, FTP, Apache Spark
Experience with JSON, Avro, Parquet
Experience with Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ