This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
The Business Consulting-Technical Analyst role focuses on ETL and GCP using PySpark, requiring strong programming skills and experience in data pipeline development. The candidate will collaborate with teams to optimize data solutions and ensure data quality.
Job Responsibility:
Designing, implementing, and optimizing data pipelines on GCP using PySpark for efficient and scalable data processing
Building and maintaining ETL workflows for extracting, transforming, and loading data into various GCP services
Leveraging GCP services like BigQuery, Cloud Storage, Dataflow, and Dataproc for data storage, processing, and analysis
Utilizing PySpark for data manipulation, cleansing, enrichment, and validation
Ensuring the performance and scalability of data processing jobs on GCP
Working with data scientists, analysts, and other stakeholders to understand data requirements and translate them into technical solutions
Implementing and maintaining data quality standards, security measures, and compliance with data governance policies on GCP
Diagnosing and resolving issues related to data pipelines and infrastructure
Keeping abreast of the latest GCP services, PySpark features, and best practices in data engineering
Requirements:
GCP Professional Data Engineering certification
Strong understanding of GCP services like BigQuery, Cloud Storage, Dataflow, and Dataproc
Demonstrated experience in using PySpark for data processing, transformation, and analysis
Solid Python programming skills for data manipulation and scripting
Experience with data modeling, ETL processes, and data warehousing concepts
Proficiency in SQL for querying and manipulating data in relational databases
Understanding of big data principles and distributed computing concepts
Ability to effectively communicate technical solutions and collaborate with cross-functional teams
Systems design experience with the ability to architect and explain complex systems interactions, data flows, common interfaces and APIs
Deep understanding of and experience with software development and programming languages such as Java/Kotlin, and Shell scripting
Hands-on experience with the following technologies as a senior software developer: Java/Kotlin, Spring, Spring Boot, Wiremock, Docker, Terraform, GCP services (Kubernetes, CloudSQL, PubSub, Storage, Logging, Dashboards), Oracle & Postgres, SQL, PgWeb, Git, Github & Github Actions