Big Data Developer

• Participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team
• Contribute to applications systems analysis and programming activities
• Design, develop, and optimize large-scale data processing systems
• Work closely with cross-functional teams to build efficient data pipelines, perform data analysis, and support business-critical financial solutions

• Minimum of 5 years of overall IT experience, with at least 3 years of hands-on experience in Big Data technologies
• Proven experience working with large-scale, high-volume datasets in distributed environments
• Strong proficiency in Hadoop ecosystem tools, including: HDFS (Hadoop Distributed File System) for data storage
• Hive for data querying and warehousing
• Sqoop for data ingestion from relational databases
• Advanced knowledge of Apache Spark, including: Spark Core, Spark SQL, and Spark Streaming (preferred)
• Performance tuning and optimization techniques (e.g., partitioning, caching, memory management)
• Solid programming skills in Python and PySpark for data processing and pipeline development
• Strong command of SQL for complex queries, data transformations, and performance tuning
• Hands-on experience in data sourcing, ingestion, and extraction from multiple structured and unstructured data sources
• Strong understanding of ETL (Extract, Transform, Load) concepts, data modeling, and workflow orchestration
• Experience building and maintaining robust data pipelines for batch and/or real-time processing
• Prior exposure to financial services projects or products is mandatory
• Strong analytical thinking with the ability to troubleshoot and resolve complex data issues
• Expertise in data analysis, root cause investigation, and data validation
• Excellent communication skills with the ability to collaborate effectively with stakeholders, business users, and technical teams
• Ability to coordinate with end users to understand requirements and deliver data solutions accordingly
• Strong attention to detail and commitment to data quality and accuracy
• Bachelor’s degree/University degree or equivalent experience
• Familiar with HIVE, HDFS, SPARK, PYTHON, PYSPARK, SQL and SQOOP

Knowledge and experience of DataBricks is a plus