Scala & Spark Developer

• Design and develop reusable Frameworks for Data ingestion, extraction, report submission etc.
• Translate high-level & functional requirements, Data Modelling (dimensional, semi structured, transactional use cases) to technical design
• Develop batch & real time data ingestion Pipelines involving wide range of technologies like Messaging middleware, Kafka, SFTP, Spark, Hive etc.
• Develop programs to migrate Historical Data from legacy platforms to the BigData platform
• Develop programs for real time and EOD reconciliations
• Provide SME support for development of automated QA scripts
• Participate in UAT/ SIT Test cycles, Release cycles, triage and resolve issues
• Setup monitoring and management for services
• Partner with Project Manager, BA and Business stakeholders and prioritize the Book of Work
• Perform Code reviews, test case reviews and ensure Functional & Non-Functional requirements
• Analyse Platform & Software version upgrades, evaluate new tools and technologies for Big Data handling
• Ensure adherence to and develop best practices supporting Citi’s Project Management Standards
• Ensure SDLC standards are followed with artefacts to support Internal & External Audits
• Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
• Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
• Has the ability to operate with a limited level of direct supervision
• Can exercise independence of judgement and autonomy
• Acts as SME to senior stakeholders and /or other team members
• Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency

• 5+ years of software development experience building large scale distributed data processing systems or large-scale Web applications
• Experience of at least 3 years in designing & developing Big Data solutions with at least one end to end implementation
• Strong Hands-on experience in following technical skills: Apache Spark
• Java/ Scala
• XML/ JSON/ Parquet/ Avro/ protobuf
• SQL
• Spring Boot/ Microservices
• Linux
• Hadoop Ecosystem (HDFS, Spark, Impala, HIVE, HBASE etc.)
• Kafka
• Performance analysis, troubleshooting and issue resolution
• Experience working with Software vendor teams on open issues and resolutions
• Strong experience with SQL, building queries, analysing, troubleshooting and improving queries
• A history of delivering against agreed objectives
• Ability to multi-task and work under pressure
• Enthusiastic and proactive approach with willingness to learn, ability to pick up new concepts and applying the knowledge
• Demonstrated problem solving skills
• Excellent analytical and process-based skills, ability to produce process flow diagrams, business modelling, and functional design
• The candidate is expected to be dynamic, flexible with a high energy level as this is a demanding and rapidly changing environment
• Intermediate level experience in Applications Development role
• Consistently demonstrates clear and concise written and verbal communication
• Demonstrated problem-solving and decision-making skills
• Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements

Exposure to Cloudera offerings like Ozone, Iceberg etc. is good to have