Hadoop PySpark, Python, Apache Kafka Job at Realign (Charlotte, NC / New York, NY / Dallas, TX / Jersey City, NJ)

New

Hadoop PySpark, Python, Apache Kafka

Location

United States , Charlotte; New York; Dallas; Jersey City

Salary:

160000.00 USD / Year

Realign

Expiration Date

Until further notice

Requirements

Primary Skill: Hadoop ecosystem (HDFS, Hive, Spark), PySpark, Python, Apache Kafka
Secondary: UI – Angular
Experience: Minimum 9 years
Technical Expertise: Strong experience with Hadoop ecosystem (HDFS, Hive, Spark)
Proficiency in PySpark for distributed data processing
Advanced programming skills in Python
Hands-on experience with Apache Kafka for real-time streaming
Frontend development using Angular (TypeScript, HTML, CSS)
Architectural Skills: Expertise in designing scalable, secure, and high-performance systems
Familiarity with microservices, API design, and cloud-native architectures

Job Responsibility

Architectural Leadership: Define end-to-end architecture for data platforms, streaming systems, and web applications
Ensure alignment with enterprise standards, security, and compliance requirements
Evaluate emerging technologies and recommend adoption strategies
Data Engineering: Design and implement data ingestion, transformation, and processing pipelines using Hadoop, PySpark, and related tools
Optimize ETL workflows for large-scale datasets and real-time streaming
Integrate Apache Kafka for event-driven architectures and messaging
Application Development: Build and maintain backend services using Python and microservices architecture
Develop responsive, dynamic front-end applications using Angular
Implement RESTful APIs and ensure seamless integration between components
Collaboration & Leadership: Work closely with product owners, business analysts, and DevOps teams

Fulltime

Senior Big Data Engineer

The Big Data Engineer is a senior level position responsible for establishing an...

Location

Canada , Mississauga

Salary:

94300.00 - 141500.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

5+ Years of Experience in Big Data Engineering (PySpark)
Data Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from multiple sources
Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
Proficiency in programming languages like Python, or Scala
Strong expertise in data processing frameworks such as Apache Spark, Hadoop
Expertise in Data Lakehouse technologies (Apache Iceberg, Apache Hudi, Trino)
Experience with cloud data platforms like AWS (Glue, EMR, Redshift), Azure (Synapse), or GCP (BigQuery)
Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)
Experience with data orchestration tools like Apache Airflow or Prefect
Familiarity with containerization (Docker, Kubernetes) is a plus

Job Responsibility

Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
Appropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients and assets

Fulltime

Senior Big Data Engineer

The Big Data Engineer is a senior level position responsible for establishing an...

Location

Canada , Mississauga

Salary:

94300.00 - 141500.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

5+ Years of Experience in Big Data Engineering (PySpark)
Data Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from multiple sources
Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
Proficiency in programming languages like Python, or Scala
Strong expertise in data processing frameworks such as Apache Spark, Hadoop
Expertise in Data Lakehouse technologies (Apache Iceberg, Apache Hudi, Trino)
Experience with cloud data platforms like AWS (Glue, EMR, Redshift), Azure (Synapse), or GCP (BigQuery)
Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)
Experience with data orchestration tools like Apache Airflow or Prefect
Familiarity with containerization (Docker, Kubernetes) is a plus

Job Responsibility

Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
Appropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency

What we offer

Well-being support
Growth opportunities
Work-life balance support

Fulltime

New

Hadoop PySpark, Python, Apache Kafka

Architectural Leadership, Data Engineering, Application Development, Collaborati...

Location

United States , Charlotte, NC / New York, NY/ Dallas, TX / Jersey City, NJ

Salary:

160000.00 USD / Year

Realign

Expiration Date

Until further notice

Requirements

Minimum 9 years experience
Strong experience with Hadoop ecosystem (HDFS, Hive, Spark)
Proficiency in PySpark for distributed data processing
Advanced programming skills in Python
Hands-on experience with Apache Kafka for real-time streaming
Frontend development using Angular (TypeScript, HTML, CSS)
Expertise in designing scalable, secure, and high-performance systems
Familiarity with microservices, API design, and cloud-native architectures
Knowledge of CI/CD pipelines, containerization (Docker/Kubernetes)
Exposure to cloud platforms (AWS, Azure, GCP)

Job Responsibility

Define end-to-end architecture for data platforms, streaming systems, and web applications
Ensure alignment with enterprise standards, security, and compliance requirements
Evaluate emerging technologies and recommend adoption strategies
Design and implement data ingestion, transformation, and processing pipelines using Hadoop, PySpark, and related tools
Optimize ETL workflows for large-scale datasets and real-time streaming
Integrate Apache Kafka for event-driven architectures and messaging
Build and maintain backend services using Python and microservices architecture
Develop responsive, dynamic front-end applications using Angular
Implement RESTful APIs and ensure seamless integration between components
Work closely with product owners, business analysts, and DevOps teams

Fulltime

New

Hadoop PySpark, Python, Apache Kafka

Architectural Leadership: Define end-to-end architecture for data platforms, str...

Location

United States , Charlotte, NC / New York, NY/ Dallas, TX / Jersey City, NJ

Salary:

160000.00 USD / Year

Realign

Expiration Date

Until further notice

Requirements

Minimum 9 years experience
Strong experience with Hadoop ecosystem (HDFS, Hive, Spark)
Proficiency in PySpark for distributed data processing
Advanced programming skills in Python
Hands-on experience with Apache Kafka for real-time streaming
Frontend development using Angular (TypeScript, HTML, CSS)
Expertise in designing scalable, secure, and high-performance systems
Familiarity with microservices, API design, and cloud-native architectures
Knowledge of CI/CD pipelines, containerization (Docker/Kubernetes)
Exposure to cloud platforms (AWS, Azure, GCP)

Job Responsibility

Define end-to-end architecture for data platforms, streaming systems, and web applications
Ensure alignment with enterprise standards, security, and compliance requirements
Evaluate emerging technologies and recommend adoption strategies
Design and implement data ingestion, transformation, and processing pipelines using Hadoop, PySpark, and related tools
Optimize ETL workflows for large-scale datasets and real-time streaming
Integrate Apache Kafka for event-driven architectures and messaging
Build and maintain backend services using Python and microservices architecture
Develop responsive, dynamic front-end applications using Angular
Implement RESTful APIs and ensure seamless integration between components
Work closely with product owners, business analysts, and DevOps teams

Fulltime

Big Data Engineering Lead

The Senior Big Data engineering lead will play a pivotal role in designing, impl...

Location

India , Chennai

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Bachelor's or Master’s degree in Computer Science, Information Technology, or related field
Atleast 10 -12 years overall software development experience on majorly working with handling application with large scale data volumes from ingestion, persistence and retrieval
Deep understanding of big data technologies, including Hadoop, Spark, Kafka, Flink, NoSQL databases, etc.
Experience with Bigdata technologies Developer Hadoop, Apache Spark, Python, PySpark
Strong programming skills in languages such as Java, Scala, or Python
Excellent problem-solving skills with a knack for innovative solutions
Strong communication and leadership abilities
Proven ability to manage multiple projects simultaneously and deliver results

Job Responsibility

Lead the design and development of a robust and scalable big data architecture handling exponential data growth while maintaining high availability and resilience
Design complex data transformation processes using Spark and other big data technologies using Java, Pyspark or Scala
Design and implement data pipelines that ensure data quality, integrity, and availability
Collaborate with cross-functional teams to understand business needs and translate them into technical requirements
Evaluate and select technologies that improve data efficiency, scalability, and performance
Oversee the deployment and management of big data tools and frameworks such as Hadoop, Spark, Kafka, and others
Provide technical guidance and mentorship to the development team and junior architects
Continuously assess and integrate emerging technologies and methodologies to enhance data processing capabilities
Optimize big data frameworks, such as Hadoop, Spark, for performance improvements and reduced processing time across distributed systems
Implement data governance frameworks to ensure data accuracy, consistency, and privacy across the organization, leveraging metadata management and data lineage tracking

Fulltime

Spark Engineer

Bright Vision Technologies is looking for a Spark Engineer to join their team to...

Location

United States , Bridgewater

Salary:

Not provided

Bright Vision Technologies

Expiration Date

Until further notice

Requirements

Apache Spark
PySpark
Spark SQL
Scala / Python
Hadoop ecosystem (HDFS, YARN)
Kafka
DataFrames & Datasets
ETL pipelines
SQL
NoSQL databases

What we offer

H-1B sponsorship for the 2026 quota
H-1B filing with level 4 prevailing wage

Fulltime

Spark Engineer

Bright Vision Technologies is looking for a skilled Spark Engineer to join our d...

Location

United States , Bridgewater

Salary:

Not provided

Bright Vision Technologies

Expiration Date

Until further notice

Requirements

Apache Spark
PySpark
Spark SQL
Scala / Python
Hadoop ecosystem (HDFS, YARN)
Kafka
DataFrames & Datasets
ETL pipelines
SQL
NoSQL databases

Job Responsibility

Contribute to the mission of transforming business processes through technology
Build scalable, high-performance analytics platforms

What we offer

H-1B sponsorship
Career growth potential

Fulltime

Hadoop PySpark, Python, Apache Kafka

Realign

Location:
United States , Charlotte, NC / New York, NY / Dallas, TX / Jersey City, NJ

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

Job Description:

Job Responsibility:

Requirements:

Nice to have:

Additional Information:

Job Posted:
March 21, 2026

Looking for more opportunities? Search for other job offers that match your skills and interests.

Similar Jobs for Hadoop PySpark, Python, Apache Kafka

Hadoop PySpark, Python, Apache Kafka

Senior Big Data Engineer

Senior Big Data Engineer

Hadoop PySpark, Python, Apache Kafka

Hadoop PySpark, Python, Apache Kafka

Big Data Engineering Lead

Spark Engineer

Spark Engineer

Hadoop PySpark, Python, Apache Kafka

Realign

Location:United States , Charlotte, NC / New York, NY / Dallas, TX / Jersey City, NJ

Category:IT - Software Development

Contract Type:Not provided

Salary:

Job Description:

Job Responsibility:

Requirements:

Nice to have:

Additional Information:

Job Posted:March 21, 2026

Looking for more opportunities? Search for other job offers that match your skills and interests.

Similar Jobs for Hadoop PySpark, Python, Apache Kafka

Hadoop PySpark, Python, Apache Kafka

Senior Big Data Engineer

Senior Big Data Engineer

Hadoop PySpark, Python, Apache Kafka

Hadoop PySpark, Python, Apache Kafka

Big Data Engineering Lead

Spark Engineer

Spark Engineer

Location:
United States , Charlotte, NC / New York, NY / Dallas, TX / Jersey City, NJ

Category:
IT - Software Development

Contract Type:
Not provided

Job Posted:
March 21, 2026