CrawlJobs Logo
Realign Logo Realign · IT - Software Development

Hadoop Engineer

United States, Bellevue 180000.00 USD / Year · Job Posted March 19, 2026
Apply Position
Job Link Share

Job Description

Role - Hadoop Engineer; Experience - 6+

Job Responsibility

  • Hadoop administration, Cloud Era, AWS, Hortonworks
  • Supporting HDInsight product team in Hadoop Admin support
  • Helping customers in providing resolutions to end customers by resolving the ICM
  • Responsibilities would include – Run prototype to migrate Big-data workloads Performance tuning for Spark, Hive, and Hadoop jobs
  • Troubleshoot production issues, identify the root cause, and provide mitigation
  • Build tools and services to improve debuggability/supportability
  • Monitoring cluster health for top customers and providing support

Requirements

  • Must Have Technical/Functional Skills: Digital: Big Data and Hadoop Ecosystems, Digital: Kafka, Digital : HBase
  • Required Skills: DEVOPS ENGINEER SENIOR EMAIL SECURITY ENGINEER
Realign - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Hadoop Engineer

8 matching positions

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

The Data Engineer role at NTT DATA requires a Bachelor’s or Master’s degree in C...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • 5–7 years of professional hands-on coding experience in collaborative, team-based environments
  • strong troubleshooting skills in SQL and scripting
  • proficiency in Python or Java
  • deep familiarity with SDLC, CI/CD best practices, and Kubernetes deployment
  • expertise in temporal data modeling (e.g., SCD Type 2)
  • schema management with a focus on schema evolution (Iceberg Apache)
  • performance optimization through data partitioning and clustering
  • architectural theory involving normalization/denormalization and natural vs. surrogate keys
  • experience with Python
Job Responsibility
Job Responsibility
  • Designing and implementing data solutions
  • optimizing performance
  • collaborating within a team
  • Fulltime
Read More
Arrow Right

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

The Data Engineer will play a crucial role in migrating data from on-prem DataLa...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to troubleshoot SQL and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices
  • K8s deployment experience
  • Sophisticated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, and Architectural Theory
  • Experience with Kafka, ANSI SQL, FTP, Apache Spark
  • Experience with JSON, Avro, Parquet
  • Experience with Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ
Job Responsibility
Job Responsibility
  • Perform end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse
  • Pipeline Migration - Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Data Transfer - Executing the physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement - Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners to ensure migrated assets meet business requirements
  • Consumption Pattern Migration - Translating and optimizing legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg
  • Usage analysis to understand usage patterns and deliver required data products
  • Data Reconciliation and Quality - Work with reconciliation frameworks to build confidence that migrated data is functionally equivalent to that already used within production flows
  • Fulltime
Read More
Arrow Right

Data Engineer

We are looking for a skilled Data Engineer to join our team on a long-term contr...
Location
Location
United States , New York
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience in implementing and managing Snowflake and Palantir platforms
  • Strong proficiency in programming languages such as Python for data manipulation and analysis
  • Expertise in working with Apache Spark, Hadoop, and Kafka for large-scale data processing
  • In-depth knowledge of ETL processes and tools to handle complex data workflows
  • Familiarity with building and optimizing data pipelines and architectures
  • Ability to troubleshoot and resolve data system issues in a timely manner
  • Excellent problem-solving skills and attention to detail
  • Strong communication skills to collaborate effectively with cross-functional teams
Job Responsibility
Job Responsibility
  • Design, develop, and maintain data pipelines to support efficient data integration and processing
  • Implement and manage Snowflake-based data repositories to centralize and secure organizational data
  • Utilize Palantir to standardize and analyze data across multiple divisions, ensuring consistency and accuracy
  • Develop ETL processes to extract, transform, and load data from various sources into centralized systems
  • Collaborate with stakeholders to define data requirements and optimize system performance
  • Leverage tools like Apache Spark, Hadoop, and Kafka to build scalable data solutions
  • Monitor and troubleshoot data systems to ensure reliability and address any issues promptly
  • Deploy advanced analytics and AI models to drive operational efficiencies
  • Document processes and provide technical support to ensure smooth implementation and usage of data systems
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • company 401(k) plan
Read More
Arrow Right
New

Junior Algorithm Developer

Altamira Technologies has a long and successful history providing innovative sol...
Location
Location
United States , Warrenton, VA or Augusta, GA
Salary
Salary:
Not provided
altamiracorp.com Logo
Altamira Technologies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s (or equivalent experience) in Electrical/Computer/Systems Engineering, Applied mathematics, Computer Science, Physics or a related field
  • Knowledge and experience in COMINT, ELINT, and/or FISINT collection and processing
  • Knowledge and experience in one or more of the following: signals applications, signal data processing, prototype development and supporting transition of prototype Ops to limited and baseline Ops
  • Experience in one or more of the following: X0Midas, C, C++, FORTRAN, Java, MongoDB, Oracle, Red Hat Linux, Apache, Python, HTML, Dynamic HTML, JavaScript, MySQL, Perl, Extensible Markup Language, Hadoop, Java Message Service, Rails, Esper
  • Must hold TS/SCI clearance w/polygraph (U.S. Citizenship required for clearance)
  • Self-motivated and eager to work intently to satisfy mission requirements
  • Adaptable and has the desire to maintain our company culture
  • Ability to effectively communicate in verbal and written communications
  • Ability to multitask and adjust priorities as needed
Job Responsibility
Job Responsibility
  • Development of new software, algorithms, analytics, and other operator-generated mission requirements for understanding and articulating I&W
  • Integration of data, tools, and capabilities from across laboratory environments into tools and environments readily accessible by operators
  • Design/implement wrapper routines as needed around Astrograph executable in R&D and ops environments
  • Implementation of advanced processing techniques into relevant tools and software for expanding mission awareness
  • Fulltime
Read More
Arrow Right
New

L2 Application Support Analyst – Financial Systems

Join Citi's Production and Application Support team, where you will keep mission...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6 or more years of experience in application or production support, with at least 2 years operating at an advanced level in a financial services environment
  • Hands-on capability with Unix/Linux, including scripting, file manipulation, and job control, combined with advanced SQL skills across platforms such as MSSQL or Oracle, including stored procedures, SSIS, and SSRS
  • Practical experience working with Big Data technologies — Hadoop, Hive, and Spark — and job scheduling tools such as Autosys, including log analysis and pipeline troubleshooting
  • Solid knowledge of financial instruments and trade lifecycle operations, covering Equities, Fixed Income, Secured Financing, Derivatives, and Liquidity Management
  • Familiarity with front office and back office workflows, including reporting operations and the systems that support them
  • Demonstrated ability to analyse complex, time-sensitive issues systematically and communicate findings clearly to both technical and business audiences
  • Working knowledge of ITIL practices, including incident management, problem management, and change management processes
  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience
Job Responsibility
Job Responsibility
  • Deliver L2 production support for mission-critical liquidity reporting and financial applications, maintaining high availability and consistent performance across the trading lifecycle
  • Monitor, triage, and resolve production incidents related to trade capture, batch processing failures, position keeping, market data, pricing, and risk and liquidity reporting
  • Manage scheduled jobs, alerts, and system logs using job scheduling tools and Unix-based monitoring platforms to prevent and contain service disruptions
  • Execute advanced SQL queries and scripts to analyse data, validate outputs, and resolve data integrity issues across multiple application layers
  • Troubleshoot data pipeline failures across applications built on stored procedures, reporting services, and Big Data ecosystems including Hadoop, Hive, and Spark
  • Lead root cause analysis for major incidents, coordinate post-incident reviews, and implement permanent fixes to prevent recurrence
  • Coordinate change management and release activities, including deployment validations and post-release checks, to maintain production stability
  • Maintain and improve operational knowledge bases, standard operating procedures, and runbooks to support team effectiveness and knowledge continuity
What we offer
What we offer
  • Hybrid working model — 3 days in the office and 2 days working remotely, giving you flexibility alongside meaningful in-person collaboration
  • Exposure to complex, high-stakes financial systems spanning multiple asset classes and trading workflows, deepening both your technical and domain expertise
  • Access to learning and development opportunities that support your growth across production support, financial technology, and service management disciplines
  • The opportunity to work across global teams, collaborating with infrastructure, development, and business stakeholders on initiatives that improve system resilience and service quality
  • A supportive environment that values work-life balance, wellbeing, and long-term career development within one of the world's leading financial institutions
  • Fulltime
Read More
Arrow Right
New

Data Engineer

We are looking for a Data Engineer to join a real estate and property organizati...
Location
Location
United States , Chicago
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on experience with Microsoft Fabric and/or Azure Data Factory, including building and optimizing pipelines
  • Proven background working with Azure data services and cloud-based data architecture
  • Advanced experience with data modeling, complex querying, and performance tuning in enterprise data platforms
  • Solid programming skills in Python, Java, Scala, or a comparable language used in data engineering
  • Experience with ETL development, relational databases, and data warehousing concepts
  • Familiarity with technologies such as Apache Spark, Hadoop, or Kafka in support of large-scale data processing
  • Ability to work effectively as an individual contributor in a lean team environment with high accountability
  • Exposure to Power BI or Tableau, along with the ability to align technical solutions with business needs
Job Responsibility
Job Responsibility
  • Design, build, and enhance end-to-end data pipelines using Microsoft Fabric and/or Azure Data Factory for production use
  • Manage and improve the data environment with a strong ownership mindset, ensuring performance, reliability, and maintainability
  • Integrate data from varied sources such as APIs, databases, and flat files into structured, usable datasets
  • Develop data models and schema designs that support reporting, analytics, and downstream business needs
  • Monitor data quality and implement validation checks, troubleshooting issues through root cause analysis and continuous optimization
  • Create scalable cloud-based data solutions within Azure that support long-term operational and analytical goals
  • Partner with cross-functional stakeholders to translate business questions into practical data engineering solutions
  • Contribute to orchestration, automation, and ongoing support of data workflows using modern engineering tools and programming languages
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
Read More
Arrow Right
New

Applications Development Group Manager

Location
Location
United States , Tampa
Salary
Salary:
185300.00 - 219600.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
August 04, 2026
Flip Icon
Requirements
Requirements
  • Requires a Bachelor’s degree (3 or 4 year) or foreign equivalent in Business Management, Computer Science, Information Technology or related field and 10 years of experience as an Applications Development Senior Manager, System Analyst, Senior Specialist Data Engineer, Consultant, Project Manager or related position using Ab Initio, UNIX and Oracle. 2 years of experience must include: Designing and developing financial and banking applications for institutional clients
  • Delivering enterprise-scale applications
  • ETL technologies and data warehousing
  • Tableau, Autosys, and Solaris
  • Working with offshore technology consultants
  • Building large scale applications by using big data tools (Big Data Hadoop using HDFS, Sqoop, Hive, Python)
  • Gathering and analyzing business and technical requirements, software design, software development lifecycle, and application implementation methodology
  • Conducting feasibility study of new requirements
  • Design preparation
  • and Performing unit testing, code migration, and quality assurance.
Job Responsibility
Job Responsibility
  • Lead IT project teams and provide technical oversight across systems and applications
  • Provide tech solutions across treasury function in Citi, leading liquidity/resolution and recovery planning applications
  • Review and analyze proposed technical solutions for projects and contribute to strategy formulation for application development
  • Understand the functional and non-functional requirements and program objectives
  • Participate in release planning events, understand the program priorities, and resolve cross team dependencies
  • Participate in architectural workshops to address current architectural bottlenecks and constraints to improve the system
  • Create and update high-level design and detailed design templates
  • Design and develop reusable frameworks and tools that can reduce development time and cost or quality and consistency of services
  • Explore new and upcoming technology trends including Big Data, Google Cloud, and Snowflake
  • Ensure delivery excellence through estimation, planning, execution, governance, reviews with Client Stakeholders, and mitigating the risks and issues
What we offer
What we offer
  • medical, dental & vision coverage
  • 401(k)
  • life, accident, and disability insurance
  • wellness programs
  • paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays
  • Fulltime
Read More
Arrow Right

GCP Lead Engineer

We are seeking a Lead GCP Engineer to provide technical leadership for enterpris...
Location
Location
United States , Phoenix
Salary
Salary:
Not provided
technologent.com Logo
Technologent
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in data engineering, platform engineering, or big data environments
  • 8+ years of experience leading teams supporting enterprise-scale batch and real-time data platforms
  • Strong hands-on experience with: Hadoop, HBase, Kafka, Redis, Elasticsearch, MapRDB or similar distributed data platforms
  • 3+ years of experience with Google Cloud Platform (GCP), including: Bigtable, Dataproc, Pub/Sub, Composer
  • Strong Python development experience
  • Experience designing and supporting large-scale distributed systems and real-time data architectures
  • Proven ability to lead technical initiatives and mentor engineering teams
  • Strong analytical, troubleshooting, and communication skills
Job Responsibility
Job Responsibility
  • Lead the design, implementation, and optimization of enterprise real-time and batch data platforms
  • Provide technical leadership and mentorship to data engineering teams supporting large-scale data ecosystems
  • Drive modernization efforts leveraging Google Cloud Platform (GCP) technologies
  • Design and support scalable data pipelines for ingestion, processing, storage, and analytics
  • Collaborate with architecture, infrastructure, security, and analytics teams to deliver enterprise data solutions
  • Support and optimize distributed data platforms including streaming, search, caching, and data processing technologies
  • Develop automation, monitoring, and operational processes to improve platform reliability and performance
  • Partner with stakeholders to define platform roadmaps and future-state architecture strategies
  • Evaluate emerging technologies and support AI, machine learning, and GenAI-related initiatives
  • Establish engineering best practices for scalability, performance, security, and operational excellence
  • Fulltime
Read More
Arrow Right