This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
Design, build, and maintain scalable data platforms, pipelines, and data lakes to enable large-scale data processing and analysis. Collaborate with data scientists and stakeholders to ensure proper data collection, storage, and security, troubleshoot infrastructure issues, and optimize systems for scalability and efficiency.
Job Responsibility:
Design, build, and maintain scalable data platforms, pipelines, and data lakes to enable large-scale data processing and analysis
Collaborate with data scientists and stakeholders to ensure proper data collection, storage, and security, troubleshoot infrastructure issues, and optimize systems for scalability and efficiency
Build and maintain scalable data pipelines and robust data models for AI/ML from structured and unstructured sources
Develop Big Data pipelines using orchestration tools (Airflow/Oozie) and implement access management, monitoring, and self-service ETL/analytics solutions
Write data pipelines using Spark, Python, and Scala
develop frameworks/utilities in Python
and follow DevOps best practices
Advanced SQL skills with ability to query and transform large structured/unstructured datasets using Spark/PySpark, Spark SQL/Hive, Hive/NoSQL
hands-on experience with On-Prem Big Data platforms, distributed frameworks like YARN, and proficiency in building data pipelines using Spark, Python, and Scala
Diagnose software issues, optimize performance, and support BI tools (Tableau, Power BI, MicroStrategy) for Big Data
Lead and guide team members, adapt to new technologies, and work independently or collaboratively to achieve goals
Requirements:
Bachelor’s degree in computer science or related field (or equivalent)
10+ years in IT application support
7+ years as a Big Data Engineer
Big Data Expertise: Hadoop, Spark, Spark Streaming, Trino, Flink, Hive, Pig, Kafka, NoSQL (MongoDB, Cassandra, HBase)
Programming & Modeling: Proficiency in Java, Python, Scala, SQL
strong data modeling and ETL experience
Cloud & Security: Familiarity with AWS/Azure/GCP
knowledge of data security principles and implementation
Performance & Analytics: Experience with performance tuning, data warehousing, and ML/analytics integration in Big Data systems