Big Data and SQL Developer

• Lead the design and implementation of highly scalable, robust, and performant Big Data solutions, including data warehouses, streaming platforms, and analytical systems
• Write high-quality, production-ready code for complex data pipelines, processing frameworks, and analytical tools
• Drive performance tuning, troubleshooting, and optimization of Big Data applications and infrastructure
• Act as a go-to expert for resolving complex technical challenges across the Big Data ecosystem
• Research, evaluate, and integrate new Big Data technologies and tools
• Champion and enforce engineering best practices, including code quality, testing, CI/CD, and documentation
• Provide technical leadership and guidance to multiple engineering teams
• Mentor senior and junior engineers
• Lead comprehensive code reviews
• Contribute significantly to the Big Data strategy and roadmap
• Drive initiatives for knowledge sharing, technical documentation, and community building
• Participate in interviewing and evaluating senior technical candidates
• Act as a key technical liaison between the Big Data engineering team and internal business units, product managers, and clients
• Proactively engage with stakeholders to understand business requirements
• Provide expert technical consultation and guidance to clients and business partners
• Assist clients and internal stakeholders with complex data queries, troubleshooting data discrepancies, and providing insights derived from Big Data platforms
• Effectively communicate complex technical concepts, project progress, risks, and solutions to non-technical audiences
• Collaborate seamlessly with other teams (e.g., architecture, data sourcing) to ensure end-to-end delivery and support of data-driven initiatives

• 6+ years of progressive experience in software engineering, specifically focused on designing, building, and optimizing large-scale Big Data systems
• Proven track record as a lead or principal engineer on significant data initiatives
• Significant experience in client-facing or business partner-facing roles
• Experience in the financial services domain – Liquidity Management, Liquidity Stress Modelling, should be well verse with Markets terminologies
• Experience with large data volumes, real-time processing, and stringent data quality/security requirements
• Expert-level proficiency with Apache Spark (Spark SQL, Spark Streaming, PySpark/Scala Spark), Hadoop ecosystem (HDFS, YARN)
• Expert in Scala, Python, or Java
• Deep understanding and practical experience with data warehouse models, and relevant technologies (e.g., Delta Lake, Iceberg, Parquet)
• Strong proficiency in SQL
• Experience with NoSQL databases (e.g., Cassandra, MongoDB) is a plus
• Experience with CI/CD pipelines (e.g., Jenkins, GitLab CI), infrastructure as code (e.g., Terraform), containerization (Docker, Kubernetes/OpenShift), and monitoring tools
• Familiarity with MLOps principles for data science workflows
• Solid understanding of data governance, data quality, data lineage, and data security principles in a Big Data environment
• Bachelor’s degree/University degree or equivalent experience
• Master’s degree preferred

Experience with NoSQL databases (e.g., Cassandra, MongoDB) is a plus