Senior Hadoop developer Job at Realign (Newark)

Senior Developer - Java & Spark – Vice President

We are seeking a highly skilled and experienced Senior Software Engineer special...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Java Core: Strong proficiency in Java fundamentals, including Equals/hashCode, Collections Framework, Generics, Lambdas, and new features.
Concurrency: Deep understanding and practical experience with Java Concurrency APIs and patterns.
Spring Ecosystem: Extensive experience with Spring Framework (Core, Boot, Data, Security, Batch, Integration, JDBC).
JVM Expertise: Solid understanding of JVM internals, class loading, memory model, garbage collection mechanisms, and performance tuning.
Apache Spark: Proven expertise with Apache Spark (RDD, Spark SQL, DataFrames, DataSets) for large-scale data processing.
Big Data Ecosystem: Experience with other Big Data technologies such as Hadoop, Hive, Impala, or similar.
Containerization: Hands-on experience with Docker or similar containerization technologies.
Orchestration: Experience with Kubernetes, OpenShift, or similar container orchestration platforms for deploying and managing distributed applications and Spark workloads.
Data Structures: In-depth knowledge of common data structures and their appropriate use, including time/space complexity analysis.
Algorithms: Awareness and application of searching and sorting algorithms, especially in a distributed context.

Job Responsibility

Design, develop, and maintain high-quality, scalable, and efficient Java-based applications, with a significant emphasis on data processing pipelines using Apache Spark.
Contribute to architectural discussions and decisions, ensuring solutions are scalable, maintainable, performant, and aligned with enterprise standards for big data and distributed systems.
Implement and enforce best practices in object-oriented programming, design patterns, and SOLID principles.
Champion Test-Driven Development (TDD) and Domain-Driven Design (DDD) methodologies.
Optimize application performance, considering JVM internals, memory management, garbage collection, and Spark job tuning.
Work with various database technologies, including relational and NoSQL, ensuring data integrity and optimal performance for both operational and analytical workloads.
Leverage cloud-native services and container orchestration platforms (e.g., Kubernetes, OpenShift) for deploying and managing applications and Spark clusters.
Participate in code reviews, providing constructive feedback and ensuring code quality, security, and adherence to coding standards.
Contribute to the continuous improvement of CI/CD pipelines and development tooling for both Java and Spark applications.
Actively engage in documentation of designs, processes, and systems to foster knowledge sharing.

Fulltime

Pyspark Big Data Senior Developer - Vice President

We are building an A-team of highly skilled and autonomous engineers, and we are...

Location

Canada , Mississauga

Salary:

120800.00 - 170800.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

6+ years of extensive, hands-on experience as a Senior Big Data Developer, with a strong emphasis on PySpark and the Apache Spark ecosystem, operating as a player/coach
Expert proficiency in Python, with a proven track record of developing robust, scalable, and high-performance PySpark applications for large-scale data processing
Deep understanding and extensive hands-on experience with Apache Spark (Spark Core, Spark SQL, Spark Streaming) and its ecosystem
Experience with distributed computing frameworks such as Hadoop (HDFS, YARN)
Expert proficiency in SQL and extensive experience with data warehousing concepts and technologies (e.g., Hive, Snowflake, Redshift, Databricks SQL)
Proven experience with various data storage formats (e.g., Parquet, ORC, Avro) and data lake solutions (e.g., Delta Lake, Iceberg)
Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
Strong experience with Apache Kafka for building real-time data pipelines and event-driven architectures
Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is a mandatory requirement

Job Responsibility

Operate end-to-end in the design, development, and implementation of robust big data solutions, ensuring optimal performance, scalability, data quality, and security
Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for big data processing and analytical solutions
Act as a player/coach within the team, mentoring junior members and leading by example in the development of efficient and innovative big data architectures
Design, develop, and optimize large-scale data pipelines using PySpark for data ingestion, transformation, and aggregation, always with an eye towards efficiency and domain relevance
Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka
Design and implement sophisticated data warehousing solutions and dimensional models for efficient data storage and retrieval, ensuring alignment with business needs
Work with various distributed data storage technologies, including distributed file systems (e.g., HDFS, S3) and NoSQL databases (e.g., MongoDB, Cassandra), selecting the right tool for the right problem
Implement efficient data processing and storage strategies to optimize the performance and scalability of big data applications, with a strong focus on the 'why' behind the technology choices
Champion best practices in software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines
Demonstrate high autonomy and agency in driving projects forward, making informed decisions, and proactively identifying areas for improvement

Fulltime

New

Senior ETL Developer

Senior ETL Developer will be responsible for designing, implementing, and optimi...

Location

United States , Tampa

Salary:

96960.00 - 145440.00 USD / Year

Citi

Expiration Date

July 16, 2026

Requirements

5-8 years of relevant experience in Software Development
Experience in systems analysis and programming of software applications
Experience in managing and implementing successful projects
Working knowledge of consulting/project management techniques/methods
Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Bachelor’s degree/University degree or equivalent experience

Job Responsibility

Design and Implement of Spark applications to process and transform large datasets in HDFS
Develop ETL Pipelines in Spark using Python for data Ingestion, cleaning, aggregation, and transformations
Optimize Spark jobs for efficiency, reducing run time and resource usage
Finetune memory management, caching, and partitioning strategies for Optimal performance
Load data from different sources into HDFS, ensuring data accuracy and integrity
Integrate Spark Applications with Hadoop frameworks like Hive, Sqoop etc.
Troubleshoot and debug Spark Job failures, monitor job logs, and Spark UI to Identify Issues

What we offer

Medical, dental & vision coverage
401(k)
Life, accident, and disability insurance
Wellness programs
Paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays

Fulltime

Senior Lead Developer (Java, Spark, HDFS, Hive) - Vice President

The Applications Development Technology Lead Analyst is a senior level position ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

10+ years of relevant experience in Apps Development or systems analysis role
Extensive experience system analysis and in programming of software applications
Experience in managing and implementing successful projects
Subject Matter Expert (SME) in at least one area of Applications Development
Ability to adjust priorities quickly as circumstances dictate
Demonstrated leadership and project management skills
Consistently demonstrates clear and concise written and verbal communication
Highly experienced and skilled Java technical lead with 10+years of experience with software building and platform engineering
Extensive development expertise in building the high scaled and performant software platforms for data computation and processing
Expert level knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns

Job Responsibility

Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency

Fulltime

Senior Algorithm Developer

We are seeking a Senior Algorithm Developer to join the Altamira team. Our ideal...

Location

United States , Warrenton, VA or Augusta, GA

Salary:

Not provided

Altamira Technologies

Expiration Date

Until further notice

Requirements

Bachelor’s (or equivalent experience) in Electrical/Computer/Systems Engineering, Applied mathematics, Computer Science, Physics or a related field
Knowledge and experience in COMINT, ELINT, and/or FISINT collection and processing
Knowledge and experience in one or more of the following: signals applications, signal data processing, prototype development and supporting transition of prototype Ops to limited and baseline Ops
Experience in one or more of the following: X0Midas, C, C++, FORTRAN, Java, MongoDB, Oracle, Red Hat Linux, Apache, Python, HTML, Dynamic HTML, JavaScript, MySQL, Perl, Extensible Markup Language, Hadoop, Java Message Service, Rails, Esper
Must hold TS/SCI clearance w/polygraph (U.S. Citizenship required for clearance)
Self-motivated and eager to work intently to satisfy mission requirements
Adaptable and has the desire to maintain our company culture
Ability to effectively communicate in verbal and written communications
Ability to multitask and adjust priorities as needed

Job Responsibility

Development of new software, algorithms, analytics, and other operator-generated mission requirements for understanding and articulating I&W
Integration of data, tools, and capabilities from across laboratory environments into tools and environments readily accessible by operators
Design/implement wrapper routines as needed around Astrograph executable in R&D and ops environments
Implementation of advanced processing techniques into relevant tools and software for expanding mission awareness

Fulltime

Senior Java Developer – Assistant Vice President

Location

India , Chennai, Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

8+ years of strong experience in Full Stack software engineering developing enterprise-scale applications
Strong experience in Java/J2EE, Spring, Hibernate with expertise in design, development, performance tuning, troubleshooting and deployment
Good understanding of ECS, Kubernetes, and Open Shift
Thorough knowledge and hands-on experience in following technologies Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig , Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
The candidate should be commendable in Data Structures and Algorithms
Experience in complex project execution in Big Data Spark eco system, where processing volumes of data thorough understanding of distributed processing and integrated applications
Expertise in building web applications using Java, Angular/React, and Oracle/PostgreSQL technology stack
Expertise in enterprise integrations through RESTful APIs, Kafka messaging etc.
Expertise in designing and optimizing the software solutions for performance and stability
Expertise in troubleshooting and problem solving

Job Responsibility

Work in an agile environment following through the best practices of agile Scrum
Analyze the requirements, seek clarifications, contribute to good acceptance criteria, estimate, and be committed
Take pride in designing solutions, developing the code free from defects and vulnerabilities, meeting functional and non-functional requirements by following modern engineering practices, reducing rework, continuously addressing technical debt
Contribute to overall team performance by helping others, peer reviewing the code diligently
Bring agility to application development through DevOps practices - automated builds, unit/functional tests, static/dynamic scans, regression tests etc.
Lookout for providing best possible customer support by troubleshooting, resolving production incidents and by eliminating the problems from the root level
Bring innovative solutions to reduce the operational risks by automating mundane repetitive tasks across SDLC
Learn to become full stack developer to address end-to-end delivery of user stories

Fulltime

Senior Data Developer (Azure)

We are looking for a highly skilled and experienced Senior Azure Data Developer ...

Location

Egypt , Cairo

Salary:

Not provided

Coca-Cola HBC

Expiration Date

Until further notice

Requirements

Bachelor's or Master’s degree in computer science, information technology, data science, or a related field (or an equivalent of 7+ years of practical experience in a tech-domain)
Hands-on experience with MS Power BI, MS PowerApps, MS Azure Data Factory, MS Azure Data Lake Store, SQL Database (T-SQL), DAX & Power Query M (3+ years)
Proven track record of implementing proof-of-concept (POC) backend solutions
Expert knowledge with cloud platforms (Azure, Google Cloud Storage, Amazon), preferably Microsoft Azure – with a focus on data warehousing & backend development
Strong programming skills in languages such as C#, .NET, Python, Scala or Java
Experience with big data architectures and large data volumes technologies (partitioned tables with billions of records, files with hundreds of GBs)
Familiarity with big data technologies such as Hadoop, Spark, and Hive as well as strong knowledge with data manipulation frameworks such a PySpark
Strong analytical & problem-solving abilities, as well as good communication skills
Strong willingness to learn and adapt to new technologies and tools
Good proficiency in English as a day-to-day business language is a must.

Job Responsibility

Assist in the design, development, and maintenance of ETL (Extract, Transform, Load) processes using Azure Data Factory
Implement data ingestion processes from various data sources into Azure data storage solutions
Work with Azure SQL Database, Azure Data Lake Storage, and Hive & Databricks to store and manage large datasets (Terabytes of data)
Ensure data integrity and availability by implementing appropriate storage solutions
Integrate data from various sources, ensuring accuracy, completeness, and consistency
Collaborate with data scientists and analysts to understand data requirements and provide necessary support
Contribute to the strategy, design and development of our Enterprise and Operational Reporting, providing insights and supporting decision making for end users
Integrate & transform data from various data systems into structures that are suitable for building analytical solutions
Design, develop and implement Azure-based solutions connecting certified and validated data sources to Power BI front-end visualizations and creating a single source of truth for reporting

What we offer

Development opportunities
Equal opportunity employer
IT Equipment
Learning programs
Work with iconic brands
Supportive team
Volunteering Opportunities
Wellbeing program

Senior Java Developer

As a Full Stack Java Developer at Hawk, your mission is to contribute to the dev...

Location

Germany , Munich

Salary:

Not provided

Hawk

Expiration Date

Until further notice

Requirements

BSc or MSc degree in Computer Science or a related technical field
4+ years of experience in delivering always-on code
Rock-solid development experience in Java and front-end technologies such as React
Experience with databases (Elasticsearch, PostgreSQL) and/or big-data stacks (Hadoop, Spark, Kafka)
Good interpersonal and communication skills
You live agile and lightweight processes and take pride in your craftsmanship as a programmer
Experience with Agile and Lean methodologies such as Scrum and Kanban
Fluent English & German communication skills

Job Responsibility

Take ownership of the entire feature development process, ensuring seamless integration and performance across all layers of the stack
Use your expertise to streamline the development process, ensuring timely and consistent release of new features and updates
Promote and implement best practices in coding, testing, and deployment to foster a culture of excellence and ongoing improvement within the team
Develop code that is reliable, efficient, and maintainable, with a strong focus on quality and scalability to meet the needs of our users and stakeholders
Develop modular and reusable components that can be leveraged across different projects and features to improve development efficiency and consistency
Work closely with operations, sales, and other development team members to ensure that our product meets the needs of all stakeholders and delivers a seamless user experience

Fulltime

Select Country

Senior Hadoop developer

Job Description

Job Responsibility

Requirements

Nice to have

Looking for more opportunities?