This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
PySpark Developer with 5-10 years experience in data engineering practice. Responsible for designing, developing and maintaining scalable data pipelines, optimizing data workflows and ensuring the integrity and availability of data for business intelligence.
Job Responsibility:
Responsible for designing, developing and maintaining scalable data pipelines, optimizing data workflows and ensuring the integrity and availability of data for business intelligence
Collaborate with the stakeholders and analysts to understand data requirement and deliver robust, creative and innovative solutions
Requirements:
PySpark Developer with 5-10 years experience in data engineering practice
Strong experience in Apache Spark framework including good understanding of core concepts, performance optimization and industry best practices
Proficient in PySpark with hands-on coding experience and ability to implement complex business level transformations
Familiarity with unit testing, object-oriented programming (OOPS) concepts and interpreting test results
Proficient to write complex and efficient SQL queries to extract the business critical insights from large-scale data
Experience with scheduling of the transformation jobs as per business requirement
Perform root-cause analysis and troubleshoot errors on data pipelines, evaluating data quality issues, and implementing corrective fixes