CrawlJobs Logo

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

India, Bangalore Employment contract · Job Posted May 04, 2026
Apply Position
Job Link Share

Job Description

The Data Engineer role at NTT DATA requires a Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related field, along with 5-7 years of hands-on experience in data engineering. Candidates should be proficient in Python, Kafka, and Hadoop, with strong skills in SQL and data modeling. Familiarity with CI/CD practices and Kubernetes is essential. The position involves designing and implementing data solutions, optimizing performance, and collaborating within a team. NTT DATA offers a dynamic work environment with opportunities for growth and innovation.

Job Responsibility

  • Designing and implementing data solutions
  • optimizing performance
  • collaborating within a team

Requirements

  • Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • 5–7 years of professional hands-on coding experience in collaborative, team-based environments
  • strong troubleshooting skills in SQL and scripting
  • proficiency in Python or Java
  • deep familiarity with SDLC, CI/CD best practices, and Kubernetes deployment
  • expertise in temporal data modeling (e.g., SCD Type 2)
  • schema management with a focus on schema evolution (Iceberg Apache)
  • performance optimization through data partitioning and clustering
  • architectural theory involving normalization/denormalization and natural vs. surrogate keys
  • experience with Python
  • experience with Kafka
  • experience with ANSI SQL
  • experience with FTP
  • experience with Apache Spark
  • experience with JSON
  • experience with Avro
  • experience with Parquet
  • experience with Hadoop (HDFS/Hive)
  • experience with Snowflake
  • experience with Apache Iceberg
  • experience with Sybase IQ

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

8 matching positions

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

The Data Engineer will play a crucial role in migrating data from on-prem DataLa...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to troubleshoot SQL and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices
  • K8s deployment experience
  • Sophisticated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, and Architectural Theory
  • Experience with Kafka, ANSI SQL, FTP, Apache Spark
  • Experience with JSON, Avro, Parquet
  • Experience with Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ
Job Responsibility
Job Responsibility
  • Perform end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse
  • Pipeline Migration - Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Data Transfer - Executing the physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement - Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners to ensure migrated assets meet business requirements
  • Consumption Pattern Migration - Translating and optimizing legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg
  • Usage analysis to understand usage patterns and deliver required data products
  • Data Reconciliation and Quality - Work with reconciliation frameworks to build confidence that migrated data is functionally equivalent to that already used within production flows
  • Fulltime
Read More
Arrow Right

Pyspark Big Data Senior Developer - Vice President

We are building an A-team of highly skilled and autonomous engineers, and we are...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of extensive, hands-on experience as a Senior Big Data Developer, with a strong emphasis on PySpark and the Apache Spark ecosystem, operating as a player/coach
  • Expert proficiency in Python, with a proven track record of developing robust, scalable, and high-performance PySpark applications for large-scale data processing
  • Deep understanding and extensive hands-on experience with Apache Spark (Spark Core, Spark SQL, Spark Streaming) and its ecosystem
  • Experience with distributed computing frameworks such as Hadoop (HDFS, YARN)
  • Expert proficiency in SQL and extensive experience with data warehousing concepts and technologies (e.g., Hive, Snowflake, Redshift, Databricks SQL)
  • Proven experience with various data storage formats (e.g., Parquet, ORC, Avro) and data lake solutions (e.g., Delta Lake, Iceberg)
  • Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
  • Strong experience with Apache Kafka for building real-time data pipelines and event-driven architectures
  • Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
  • Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is a mandatory requirement
Job Responsibility
Job Responsibility
  • Operate end-to-end in the design, development, and implementation of robust big data solutions, ensuring optimal performance, scalability, data quality, and security
  • Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for big data processing and analytical solutions
  • Act as a player/coach within the team, mentoring junior members and leading by example in the development of efficient and innovative big data architectures
  • Design, develop, and optimize large-scale data pipelines using PySpark for data ingestion, transformation, and aggregation, always with an eye towards efficiency and domain relevance
  • Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka
  • Design and implement sophisticated data warehousing solutions and dimensional models for efficient data storage and retrieval, ensuring alignment with business needs
  • Work with various distributed data storage technologies, including distributed file systems (e.g., HDFS, S3) and NoSQL databases (e.g., MongoDB, Cassandra), selecting the right tool for the right problem
  • Implement efficient data processing and storage strategies to optimize the performance and scalability of big data applications, with a strong focus on the 'why' behind the technology choices
  • Champion best practices in software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines
  • Demonstrate high autonomy and agency in driving projects forward, making informed decisions, and proactively identifying areas for improvement
  • Fulltime
Read More
Arrow Right
New

Secondary SEND Teacher

We are seeking Qualified Secondary Teachers (QTS/QTLS) or experienced Primary pr...
Location
Location
United Kingdom , Patchway
Salary
Salary:
140.00 - 233.00 GBP / Day
https://www.randstad.com Logo
Randstad
Expiration Date
June 30, 2026
Flip Icon
Requirements
Requirements
  • Hold QTS/QTLS
  • You have Qualified Teacher Status and experience within a Secondary or specialist setting
  • Are a SEND Advocate: You have a genuine interest in or experience with Autism (ASC), PMLD, SLD, or SEMH, and a passion for helping young people reach their full potential
  • Are Adaptable: You can manage complex behaviours with a trauma-informed approach and pivot lessons to meet the emotional energy of the classroom
  • Are Committed: You hold an enhanced DBS on the update service (or are willing to apply) and are 100% dedicated to Safeguarding
Job Responsibility
Job Responsibility
  • Deliver engaging lessons that translate academic concepts into real-world skills-budgeting, travel training, and workplace communication
  • Use total communication strategies (Makaton, PECS, or AAC) tailored to teenage learners to ensure their voices are heard during their transition years
  • Collaborate with Occupational Therapists and Speech and Language specialists to integrate therapeutic targets into a secondary-appropriate timetable
  • Manage a dedicated team of Learning Support Assistants, fostering a 'low-arousal' and emotionally regulated environment for students with complex needs
What we offer
What we offer
  • Competitive daily rates of £140 - £233, paid every Friday via Randstad
  • Referral Bonus
  • training and education
  • Safeguarding & Prevent
Read More
Arrow Right
New

Automation Test Engineer

Join us as an Automation Test Engineer at Barclays where you will spearhead the ...
Location
Location
India , Pune
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Core Java
  • Selenium
  • API Testing/UI Testing
  • Payment's domain
Job Responsibility
Job Responsibility
  • Design, develop, and execute testing strategies to validate functionality, performance, and user experience
  • Collaborate with cross-functional teams to identify and resolve defects
  • Continuously improve testing processes and methodologies
  • Development and implementation of comprehensive test plans and strategies
  • Creation and execution of automated test scripts
  • Collaboration with cross-functional teams to analyze requirements and contribute to development of acceptance criteria
  • Root cause analysis for identified defects
  • Collaboration with peers and participation in code reviews
  • Stay informed of industry technology trends and innovations
What we offer
What we offer
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
  • Fulltime
Read More
Arrow Right
New

Secondary Cover Teacher

Secondary Cover Teacher | Qualified Lead | Local Secondary Schools Position: Fu...
Location
Location
United Kingdom , Bradley Stoke
Salary
Salary:
140.00 - 233.00 GBP / Day
https://www.randstad.com Logo
Randstad
Expiration Date
June 30, 2026
Flip Icon
Requirements
Requirements
  • Hold QTS (Qualified Teacher Status) or equivalent recognised teaching qualification (ECTs are highly welcome)
  • Are Versatile & Confident
  • Are a Strong Communicator
  • Are Committed
  • Enhanced DBS on the update service (or willing to apply)
  • 100% dedicated to Safeguarding
Job Responsibility
Job Responsibility
  • Expert Lesson Facilitation: Deliver pre-planned work across various subjects, using your teaching expertise to explain concepts, answer questions, and keep students on track
  • Advanced Behaviour Management: Utilise your professional training to establish instant authority and maintain a calm, respectful, and productive environment
  • Adaptive Teaching: Use your QTS skills to differentiate tasks on the fly, supporting students with SEND and stretching high-attainers even in unfamiliar subject areas
  • Professional Handovers: Provide the school with high-quality feedback on student progress and classroom dynamics, ensuring seamless continuity for the regular teacher
What we offer
What we offer
  • Weekly Pay
  • Referral Bonus
  • PAYE
Read More
Arrow Right
New

Relationship Director - Real Estate

Location
Location
United Kingdom , Bristol; Cardiff
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Management of client relationships to identify the clients financial goals, challenges, and risk tolerance to support the analysis of data obtained from various sources, including the investment portfolio and cash flow, to identify trends, insights, areas for improvement and additional services to support client needs
  • Research and understanding of the client's industry trends, regulatory landscape, and competitive environment to inform strategic recommendations
  • Design of customised solutions that address the client's specific needs and objectives, incorporating a range of products and services from the bank's portfolio
  • Communication of the value proposition of proposed solutions, justification of recommendations, and negotiation of terms that are beneficial for both the client and the bank
  • Provision of guidance to clients to support their financial decisions, risk management and wealth management strategies and updates on market trends to ensure a positive and continuous relationship
  • Assessment of financial, legal, and operational risks associated with client relationships, and implementation of measures to minimise potential losses
  • Documentation of all client interactions, transactions, and agreements to ensure transparency and auditability, and communicate findings effectively to support product development, service offerings, and the overall bank strategy
  • Monitoring of client satisfaction, revenue generated, and other relevant metrics to evaluate the effectiveness of relationship management efforts
  • Execution of market research and analysis to support the identification of potential new opportunities, market trends, and customer needs, to develop strategies for business growth
Job Responsibility
Job Responsibility
  • Targeting new to bank clients and lending opportunities in the region alongside growing relationships with existing clients
  • Collaborate seamlessly with a diverse group of real estate specialist bankers together with teams in business development, real estate specialist lending, Credit Risk, Portfolio and Product
  • Be an ambassador for Barclays, facing off to key real estate sector professionals, investors, advisers and brokers, fostering strong relationships to generate new opportunities and business
  • Support a portfolio of real estate clients, driving growth, ensuring client satisfaction, and maintaining control over that portfolio
  • Play a crucial role in developing your colleagues by providing coaching, sharing industry insights, to ensure the highest standards of consistently excellent delivery
What we offer
What we offer
  • Wellness rooms
  • Gyms
  • Mental health support
  • Welcoming and inclusive culture
  • Modern workspaces
  • Collaborative areas
  • State-of-the-art meeting rooms
  • On-site cafeterias
  • Fitness centers
  • Tech-equipped workstations
  • Fulltime
Read More
Arrow Right
New

Secondary Cover Teacher

We are seeking Qualified Teachers (QTS/ECTs) who are versatile, resilient, and r...
Location
Location
United Kingdom , Filton
Salary
Salary:
140.00 - 233.00 GBP / Day
https://www.randstad.com Logo
Randstad
Expiration Date
June 30, 2026
Flip Icon
Requirements
Requirements
  • Hold QTS
  • You have Qualified Teacher Status (QTS) or an equivalent recognised teaching qualification (ECTs are highly welcome)
  • Are Versatile & Confident
  • You are happy to cover any subject on the timetable and can walk into a new classroom with a positive, authoritative presence
  • Are a Strong Communicator
  • You can build rapport with teenagers quickly and manage diverse classroom personalities with ease
  • Are Committed
  • You hold an enhanced DBS on the update service (or are willing to apply) and are 100% dedicated to Safeguarding
Job Responsibility
Job Responsibility
  • Expert Lesson Facilitation: Deliver pre-planned work across various subjects, using your teaching expertise to explain concepts, answer questions, and keep students on track
  • Advanced Behaviour Management: Utilise your professional training to establish instant authority and maintain a calm, respectful, and productive environment
  • Adaptive Teaching: Use your QTS skills to differentiate tasks on the fly, supporting students with SEND and stretching high-attainers even in unfamiliar subject areas
  • Professional Handovers: Provide the school with high-quality feedback on student progress and classroom dynamics, ensuring seamless continuity for the regular teacher
What we offer
What we offer
  • PAYE
  • Referral Bonus
  • Zero Admin Stress
  • Weekly Pay
Read More
Arrow Right
New

Mobile Expert, Bilingual

Mobile Experts (ME) work as part of a Retail Team of Experts to bring the T-Mobi...
Location
Location
United States , Las Vegas
Salary
Salary:
19.50 USD / Hour
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High School Diploma/GED (Required)
  • Less than 2 years 30 days T-Mobile Mobile Associate experience (Preferred)
  • Less than 2 years 6 months of customer service and/or sales experience, Retail environment (Preferred)
  • Change Agility (Required)
  • Communication (Required)
  • Customer Service (Required)
  • Engaging People (Required)
  • Sales (Required)
  • Teamwork Orientation (Required)
  • At least 18 years of age
Job Responsibility
Job Responsibility
  • Serving and selling to our customers, while providing a best in class customer experience and building loyalty
  • Consistently leverages digital tools in interactions and onboarding
  • Completes training on T-Mobile in-store experience, new skills and processes, knowledge of systems and reference resources
  • Makes the most of their time on shift, consistently seeking out information between customers, learning about innovations in wireless and technology
  • Partners with T-Mobile employees across channels, including business and customer service
What we offer
What we offer
  • Annual Stock Grant
  • Employee stock purchase plan
  • 401(k)
  • Free year-round money coaches
  • Medical insurance
  • Dental insurance
  • Vision insurance
  • Flexible spending account
  • Paid time off
  • Up to 12 paid holidays
  • Parttime
Read More
Arrow Right