Python PySpark Developer

Key Responsibilities: • Designing and developing robust PySpark applications for large-scale data processing. • Building and optimizing data ingestion, transformation, and storage processes. • Implementing efficient algorithms and data structures for distributed computing. • Collaborating with cross-functional teams to integrate data-driven solutions into business processes. • Troubleshooting performance bottlenecks and ensuring high availability and reliability of data pipelines. • Writing and optimizing SQL queries for data extraction and manipulation. Required Skills and Qualifications: • Bachelor’s/Master’s degree in Computer Science, Engineering, Proven experience (3-10 years) in Python development with a focus on PySpark. • Strong understanding of distributed computing principles and experience with Apache Spark. • Proficiency in SQL and experience with relational databases (MySQL, PostgreSQL, etc.). • Experience with data serialization formats such as JSON, Parquet, Avro. • Familiarity with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes) is a plus. • Excellent problem-solving skills and ability to work independently or as part of a team. • Good communication skills with the ability to effectively collaborate with stakeholders. Overture Rede is an Equal Opportunity Employer and does not discriminate on the basis of race or ethnicity, religion, sex, national origin, age, veteran disability or genetic information or any other reason prohibited by law in employment.