Sr Python Data Engineer

• 4+ Years of experience in creation of end-to-end data engineering pipelines for data extraction, transformation and storage using Python
• Good at Python Core concepts (Datatypes, Data structure, Multiprocessing, Object oriented programming, programming concepts)
• Proficiency in Integrating large-scale data from a variety of sources (ftp, api, databases, web scraping) using Python
• Working experience with large datasets through python is a must
• Experience in implementing, maintaining and developing the functionality of Python packages for ETL processes
• Strong hands-on experience in SQL, Normalization, Performance Optimization techniques
• Good working knowledge of Google services : BigQuery, Workflows, DataProc, Google cloud Storage
• Good working knowledge of containerization, CI/CD pipelines and DevOps practices
• Knowledge of version control systems, particularly Github
• Troubleshooting applications and processes for data consistency and integrity
• Excellent software engineering skills, including unit testing, integration testing, and ability to write clean, efficient and scalable code
• Strong understanding of design patterns and data structures
• Excellent problem-solving skills and attention to detail
• Experience with Agile/Scrum methodologies
• Strong communication skills and the ability to work collaboratively in a team environment
• Overall Data Engineering experience of 8+ Years

Knowledge of PySpark with the ability to apply it to write Spark applications as well as analyse data in a distributed environment is good to have