CrawlJobs Logo

Sr Big Data Engineer - Oozie and Pig (GCP)

rackspace.com Logo

Rackspace

Location Icon

Location:
United States

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

116100.00 - 198440.00 USD / Year
Save Job
Save Icon
Apply Position

Job Description:

We are seeking a Senior Big Data Engineer with deep expertise in distributed systems, batch data processing, and large-scale data pipelines. The ideal candidate has strong hands-on experience with Oozie, Pig, the Apache Hadoop ecosystem, and programming proficiency in Java (preferred) or Python. This role requires a deep understanding of data structures and algorithms, along with a proven track record of writing production-grade code and building robust data workflows. This is a fully remote position and requires an independent, self-driven engineer who thrives in complex technical environments and communicates effectively across teams.

Job Responsibility:

  • Design and develop scalable batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, MapReduce, and HBase, with hands-on coding in Java or Python (Java is a must)
  • Must be able to lead Jira Epics
  • Write clean, efficient, and production-ready code with a strong focus on data structures and algorithmic problem-solving applied to real-world data engineering tasks
  • Develop, manage, and optimize complex data workflows within the Apache Hadoop ecosystem, with a strong focus on Oozie orchestration and job scheduling
  • Leverage Google Cloud Platform (GCP) tools such as Dataproc, GCS, and Composer to build scalable and cloud-native big data solutions
  • Implement DevOps and automation best practices, including CI/CD pipelines, infrastructure as code (IaC), and performance tuning across distributed systems
  • Collaborate with cross-functional teams to ensure data pipeline reliability, code quality, and operational excellence in a remote-first environment

Requirements:

  • Bachelor's degree in Computer Science, software engineering or related field of study
  • Experience with managed cloud services and understanding of cloud-based batch processing systems
  • Proficiency in Oozie, Airflow, Map Reduce, Java
  • Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
  • Expertise in public cloud services, particularly in GCP
  • Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
  • Familiarity with BigTable and Redis
  • Experienced in Infrastructure and Applied DevOps principles in daily work
  • Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform
  • Proven experience in engineering batch processing systems at scale
  • 5+ years of experience in customer-facing software/technology or consulting
  • 5+ years of experience with “on-premises to cloud” migrations or IT transformations
  • 5+ years of experience building, and operating solutions built on GCP
  • Proficiency in Oozie and Pig
  • Must be able to lead Jira Epics
  • Proficiency in Java or Python

Additional Information:

Job Posted:
January 05, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Icon
Rackspace - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon
PREMIUM
More languages and countries
+ Unlock 31698 hidden job offers
Languages
English Čeština Deutsch Ελληνικά Español Français +15
Countries
United States United Kingdom India Canada Australia +
See plans
Plans from $2.99 / month

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Sr Big Data Engineer - Oozie and Pig (GCP)

Sr Big Data Engineer - Oozie and Pig (GCP)

Location
Location
United States
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strongest programming language (Java or Python)
  • Experience with data structures and algorithms
  • Experience with frameworks in that language
  • Recent coding experience in that language
  • Years of coding experience in that language
  • Percentage of a week spent coding in that language
  • Specific time frame of experience with Oozie
  • Number of DAGs created in Oozie
  • Number of DAGs refactored in Oozie
  • Specific time frame of experience with Pig
  • Fulltime
Read More
Arrow Right

Student Success Coach

We are looking for a motivated and resilient Student Success Coach to join a spe...
Location
Location
United Kingdom , Bristol
Salary
Salary:
Not provided
affinityworkforce.com Logo
AFFINITY WORKFORCE
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Has experience working with young people with SEMH, SEND or challenging behaviour
  • Is calm, patient and confident in managing highly challenging situations
  • Has excellent communication and relationship-building skills
  • Is proactive, adaptable and able to think on their feet
  • Is passionate about making a real difference to young people's lives
  • A full UK driving licence is essential
Job Responsibility
Job Responsibility
  • Provide 1:1 and small group coaching to students with SEMH needs
  • Build strong, trusting relationships to support engagement and emotional wellbeing
  • Support students to develop self-regulation, resilience and social skills
  • Act as a positive role model, promoting confidence and independence
  • Help students access learning by working alongside teaching staff
  • Support the implementation of individual behaviour and support plans
  • Use de-escalation techniques to manage challenging behaviour calmly and effectively
  • Deliver targeted interventions around emotional wellbeing, behaviour and life skills
  • Monitor and track student progress, contributing to reviews and reports
  • Deliver outreach sessions in alternative settings or within the community
What we offer
What we offer
  • Supportive and experienced team
  • Ongoing training and professional development
  • Work in a setting that prioritises student wellbeing and long-term success
Read More
Arrow Right

Cybersecurity Engineering Associate – .NET

Wells Fargo is seeking an Engineering Associate within the Cybersecurity organiz...
Location
Location
United States , Iselin; Charlotte; Irving
Salary
Salary:
28.85 - 52.40 USD / Hour
https://www.wellsfargo.com/ Logo
Wells Fargo
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ months of Technology Engineering and Solutions experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
  • 6+ months of hands-on experience in backend development, including Microsoft .NET Core, C#, or VB.NET, with exposure to software design patterns such as MVC, Factory, Builder, DAO, Entity Framework, or Singleton
  • 6+ months of hands-on experience developing and integrating web services, including RESTful and SOAP APIs using ASP.NET Web API or ODATA, with exposure to authentication methods such as OAuth and JWT
  • 6+ months of hands-on experience building modern web applications, including front-end development with HTML, CSS, Bootstrap, JavaScript, jQuery, and React, with an emphasis on responsive and user-friendly interfaces
  • 6+ months of hands-on experience with data and integration technologies, including working with data formats such as XML, JSON, and YAML, and databases such as SQL Server, MongoDB, or other relational/non-relational systems
  • Demonstrated ability to develop, test, and debug full-stack applications, with attention to secure coding, performance, and reliability
  • Experience working with modern development and cloud technologies, including container-based and microservices environments (such as Docker, Kubernetes, or OpenShift)
  • Familiarity with DevOps and continuous delivery practices, using tools like Jenkins, GitHub, Bitbucket, Harness, YAML, or Artifactory to support build and deployment processes
  • Exposure to Agile ways of working, including Scrum or Kanban, and tools like Jira for tracking work and collaborating with teams
  • Experience supporting application quality and testing, including writing or running unit and integration tests (e.g., XUnit, MSTest)
Job Responsibility
Job Responsibility
  • Maintain secure web applications and services using Microsoft .NET Core, C#, and VB.NET, applying standard design patterns (e.g., MVC, Factory, Singleton) in alignment with secure development practices
  • Support front-end applications using HTML, CSS, Bootstrap, JavaScript, jQuery, and React to deliver secure, responsive user interfaces for cybersecurity tools and platforms
  • Support RESTful and SOAP web services using ASP.NET Web API and ODATA, incorporating secure authentication and authorization protocols such as OAuth and JWT
  • Validate structured data formats including XML, JSON, and YAML to support secure integration and data exchange across cybersecurity systems
  • Develop and maintain database solutions using SQL Server and NoSQL databases such as MongoDB, including writing optimized queries, managing schemas, and supporting secure data access controls
  • Apply secure coding standards to identify and remediate vulnerabilities, ensuring applications align with enterprise cybersecurity policies and regulatory requirements
  • Support identity and access management integrations, including authentication, authorization, role-based access controls, and secure session management
  • Participate in code reviews, testing, and validation processes to ensure application functionality, performance, and security requirements are met
  • Monitor, troubleshoot, and resolve application and service issues, contributing to incident response, vulnerability management, and root-cause analysis efforts
  • Collaborate with cybersecurity engineers, architects, and risk teams to implement security controls that protect enterprise applications, networks, and data
What we offer
What we offer
  • Health benefits
  • 401(k) Plan
  • Paid time off
  • Disability benefits
  • Life insurance, critical illness insurance, and accident insurance
  • Parental leave
  • Critical caregiving leave
  • Discounts and savings
  • Commuter benefits
  • Tuition reimbursement
  • Fulltime
Read More
Arrow Right

IT Project Manager - ONSITE

We are currently seeking a IT Project Manager - ONSITE to join our team in San D...
Location
Location
United States , San Diego
Salary
Salary:
78948.00 - 164476.00 USD / Year
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree or equivalent years of directly related experience
  • 5+ years of experience working in an Agile environment as a Scrum Master
  • 5+ years experience working with 3+ teams simultaneously
  • 5+ years experience with managing 3 to 4 Implementation projects
  • 5+ years experience in all facets of the software development life cycle (requirements gathering, system design, development, testing, release cycle, support)
  • 5+ years experience with Agile execute is preferred
  • 5+ years experience working with Product Owners
  • 5+ years expertise with running story workshops that create product backlogs and story maps
  • 5+ years in sprint and program increment planning that produce consistent metrics and frequent delivery of working product
  • 5+ years experience using Azure DevOps or similar applications for Project tracking
Job Responsibility
Job Responsibility
  • Focus is on delivering Information Technology solutions to internal business groups involving new or improved practices, procedures, techniques or systems
  • Implementation is accomplished through multiple small - large cross-functional teams which are not direct reports and are shared resources
  • Teams will be comprised of individuals from multiple departments, multiple business units and geographical locations
  • Efforts may typically include coordinating, integrating and working with teams in a Waterfall or Iterative manner
  • Excellent influencing skills are leveraged, combined with the ability to execute
  • May require occasional travel to overcome geographic, technical and cultural barriers
  • The PM will be self driven and manage overall project with minimal direction or supervision
  • Facilitate workshops with SME's and Product Owner(s) that contrubte valuable work items to a product backlog in order to deliver prioritized features/capabilities
  • Ensure team is incorporating feedback from customers, end users, and key stakeholders to continually increase value for our customers
  • Develop estimates schedules, effort and cost using established estimating models, best practices and past experience and provide continual improvement guidance
  • Fulltime
Read More
Arrow Right

Sports Mentor

We are looking for an enthusiastic and resilient Sports Mentor to support young ...
Location
Location
United Kingdom , Bristol
Salary
Salary:
Not provided
affinityworkforce.com Logo
AFFINITY WORKFORCE
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Has experience working with young people with SEMH, SEND or challenging behaviour
  • Has a strong interest or background in sport, coaching or physical activity
  • Is confident managing challenging situations in a calm and professional manner
  • Builds positive relationships quickly and effectively
  • Is energetic, proactive and passionate about making a difference
  • A full UK driving licence is essential
Job Responsibility
Job Responsibility
  • Plan and deliver sports sessions, physical activities and enrichment opportunities
  • Use sport as a tool to promote teamwork, resilience, confidence and self-discipline
  • Encourage participation and help students develop a positive connection with physical activity
  • Adapt sessions to meet a range of abilities and behavioural needs
  • Provide 1:1 and small group mentoring through both classroom and activity-based settings
  • Build strong relationships to improve engagement, behaviour and wellbeing
  • Act as a positive role model, promoting healthy lifestyles and positive choices
  • Support students in managing emotions and behaviour in a safe and constructive way
  • Support the implementation of individual behaviour and support plans
  • Manage and de-escalate challenging behaviour using calm, consistent approaches
What we offer
What we offer
  • Use sport to make a real, lasting impact on young people’s lives
  • Work in a dynamic role where no two days are the same
  • Combine mentoring, coaching and outreach work
  • Be part of a supportive and experienced team
  • Access ongoing training and development opportunities
Read More
Arrow Right

Pharmacy Technician

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , East Providence
Salary
Salary:
18.00 - 28.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
July 06, 2026
Flip Icon
Requirements
Requirements
  • Must comply with any state board of pharmacy requirements or laws governing the practice of pharmacy, which includes but is not limited to, age, education, and licensure/certification
  • If the state board of pharmacy does not address or mandate a minimum age requirement, must be at least 16 years of age
  • If the state board of pharmacy does not address or mandate a minimum educational requirement, must have a high school diploma or equivalent, or be actively enrolled in high school or high school equivalency program
  • Regular and predictable attendance, including nights and weekends
  • Ability to complete required training within designated timeframe
  • Attention and Focus: Ability to concentrate on a task over a period of time
  • Ability to pivot quickly from one task to another to meet patient and business needs
  • Ability to confirm prescription information and label accuracy, ensuring patient safety
  • Customer Service and Team Orientation: Actively look for ways to help people, and do so in a friendly manner
  • Notice and understand patients’ reactions, and respond appropriately
Job Responsibility
Job Responsibility
  • Living our purpose by following all company SOPs at each workstation to help our Pharmacists manage and improve patient health
  • Following pharmacy workflow procedures at each pharmacy workstation (i.e., production, pick-up, drive-thru, and drop-off) for safe and accurate prescription fulfillment
  • Contributing to positive patient experiences by showing empathy and genuine care: creating heartfelt and personalized moments while serving patients at pick-up, drive-thru, and over the phone
  • keeping patients healthy by offering immunizations and other services at the register and over the phone
  • and demonstrating compassionate care by solving or escalating patient problems
  • Completing basic inventory activities, as permitted by law, and as directed by the pharmacy leadership team, such as accurately putting away medication deliveries and completing cycle counts, returns-to-stocks, waiting bin inventories, etc.
  • Contributing to a high-performing team, embracing a growth mindset, and being receptive to feedback
  • actively seeking opportunities to expand clinical and technical knowledge needed to better assist patients
  • Remaining flexible for both scheduling and business needs, while contributing to a safe, inclusive, and engaging team dynamic
  • voluntarily traveling to stores in the market to work shifts as needed by the business
What we offer
What we offer
  • dental
  • vision
  • wellness resources
  • employee discounts
  • access to certain voluntary benefits
  • other programs
  • Parttime
Read More
Arrow Right

Kitchen Manager

Reporting to the General Manager, our Kitchen Managers play a pivotal part in th...
Location
Location
United Kingdom , Maidstone
Salary
Salary:
39000.00 - 42250.00 GBP / Year
beefeater.co.uk Logo
BEEFEATER
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Support the General Manager in the recruitment, induction and on boarding of any new Kitchen team members
  • Demonstrate your leadership with a hands-on approach and effective communication, engaging and motivating your team through pre-shift briefs
  • Manage stock control and food margins
  • Manage NSF, EHO and Health and Safety audits to a high standard
  • Deliver to spec and brand standards
  • Drive individual high performance through development and regular one to ones
  • Have an eye for detail and thrive in a high volume fast paced environment
  • Show passion for the brand, visible when ensuring your team are knowledgeable and confident
Job Responsibility
Job Responsibility
  • Support the General Manager in the recruitment, induction and on boarding of any new Kitchen team members
  • Demonstrate your leadership with a hands-on approach and effective communication, engaging and motivating your team through pre-shift briefs
  • Manage stock control and food margins
  • Manage NSF, EHO and Health and Safety audits to a high standard
  • Deliver to spec and brand standards
  • Drive individual high performance through development and regular one to ones
  • Have an eye for detail and thrive in a high volume fast paced environment
  • Show passion for the brand, visible when ensuring your team are knowledgeable and confident
What we offer
What we offer
  • Full induction and training
  • Seasonal menu changes with training and launch at one of our dedicated Kitchen Academy’s
  • Working to a labour model that delivers correct management cover
  • Excellent access to training and development programmes
  • A big say in how you run your kitchen
  • Tools and systems to manage your KPI’s
  • 25% off in Whitbread Restaurants and up to 50% off Premier Inn hotels
  • High street and retail discounts, including phone packages, cinema tickets and Gym membership
  • Fulltime
Read More
Arrow Right

Behaviour & Wellbeing Mentor

We are recruiting for a resilient and compassionate Behaviour & Wellbeing Mentor...
Location
Location
United Kingdom , Weston-Super-Mare
Salary
Salary:
Not provided
affinityworkforce.com Logo
AFFINITY WORKFORCE
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience working with young people with SEMH, SEND or challenging behaviour
  • Calm, patient and confident when managing complex or high-pressure situations
  • Can build strong, professional relationships quickly
  • Proactive, adaptable and able to work independently
  • Passionate about supporting young people to achieve positive outcomes
  • A full UK driving licence is essential
Job Responsibility
Job Responsibility
  • Provide 1:1 and small group mentoring to students with behavioural and emotional needs
  • Build strong, trusting relationships to support engagement, wellbeing and personal development
  • Act as a positive role model, promoting confidence, resilience and independence
  • Support students in accessing education both in and outside of the classroom
  • Support the implementation of individual behaviour and wellbeing plans
  • Use calm and consistent strategies to manage and de-escalate challenging behaviour
  • Deliver targeted interventions to develop emotional regulation, coping strategies and social skills
  • Monitor and report on student progress and wellbeing
  • Deliver outreach support in the local community, alternative provisions or mainstream settings
  • Travel regularly to provide off-site mentoring and intervention sessions
What we offer
What we offer
  • Make a meaningful difference to the lives of young people
  • Work across both in-school and outreach environments
  • Be part of a supportive and experienced team
  • Gain valuable experience within a specialist SEMH setting
  • Access ongoing training and development opportunities
Read More
Arrow Right