CrawlJobs Logo
Rackspace Logo Rackspace · IT - Software Development

Sr Big Data Engineer - Oozie and Pig (GCP)

United States 116100.00 - 198440.00 USD / Year · Job Posted January 05, 2026
Apply Position
Job Link Share

Job Description

We are seeking a Senior Big Data Engineer with deep expertise in distributed systems, batch data processing, and large-scale data pipelines. The ideal candidate has strong hands-on experience with Oozie, Pig, the Apache Hadoop ecosystem, and programming proficiency in Java (preferred) or Python. This role requires a deep understanding of data structures and algorithms, along with a proven track record of writing production-grade code and building robust data workflows. This is a fully remote position and requires an independent, self-driven engineer who thrives in complex technical environments and communicates effectively across teams.

Job Responsibility

  • Design and develop scalable batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, MapReduce, and HBase, with hands-on coding in Java or Python (Java is a must)
  • Must be able to lead Jira Epics
  • Write clean, efficient, and production-ready code with a strong focus on data structures and algorithmic problem-solving applied to real-world data engineering tasks
  • Develop, manage, and optimize complex data workflows within the Apache Hadoop ecosystem, with a strong focus on Oozie orchestration and job scheduling
  • Leverage Google Cloud Platform (GCP) tools such as Dataproc, GCS, and Composer to build scalable and cloud-native big data solutions
  • Implement DevOps and automation best practices, including CI/CD pipelines, infrastructure as code (IaC), and performance tuning across distributed systems
  • Collaborate with cross-functional teams to ensure data pipeline reliability, code quality, and operational excellence in a remote-first environment

Requirements

  • Bachelor's degree in Computer Science, software engineering or related field of study
  • Experience with managed cloud services and understanding of cloud-based batch processing systems
  • Proficiency in Oozie, Airflow, Map Reduce, Java
  • Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
  • Expertise in public cloud services, particularly in GCP
  • Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
  • Familiarity with BigTable and Redis
  • Experienced in Infrastructure and Applied DevOps principles in daily work
  • Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform
  • Proven experience in engineering batch processing systems at scale
  • 5+ years of experience in customer-facing software/technology or consulting
  • 5+ years of experience with “on-premises to cloud” migrations or IT transformations
  • 5+ years of experience building, and operating solutions built on GCP
  • Proficiency in Oozie and Pig
  • Must be able to lead Jira Epics
  • Proficiency in Java or Python
Rackspace - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Sr Big Data Engineer - Oozie and Pig (GCP)

8 matching positions

Sr Big Data Engineer - Oozie and Pig (GCP)

Location
Location
United States
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strongest programming language (Java or Python)
  • Experience with data structures and algorithms
  • Experience with frameworks in that language
  • Recent coding experience in that language
  • Years of coding experience in that language
  • Percentage of a week spent coding in that language
  • Specific time frame of experience with Oozie
  • Number of DAGs created in Oozie
  • Number of DAGs refactored in Oozie
  • Specific time frame of experience with Pig
  • Fulltime
Read More
Arrow Right
New

Administrative Assistant

Location
Location
United States , Saint Louis
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous experience in an administrative or office support role preferred
  • Strong written and verbal communication skills
  • Excellent time management and organizational abilities
  • Proficiency with Microsoft Office Suite and office technology
  • Ability to handle confidential information with integrity
Job Responsibility
Job Responsibility
  • Manage schedules, appointments, and meetings
  • Prepare and edit correspondence, reports, and presentations
  • Answer and direct phone calls
  • respond to inquiries promptly and professionally
  • Maintain and organize files, both physical and digital
  • Assist with data entry and database management
  • Support event planning and coordination tasks
  • Order office supplies and maintain inventory
  • Collaborate with team members on special projects as needed
Read More
Arrow Right
New

Early Years And Oshc Educators

Are you looking for a work life balance in early years education? Newly qualifie...
Location
Location
Australia , Brisbane
Salary
Salary:
37.24 - 60.10 AUD / Hour
https://www.randstad.com Logo
Randstad
Expiration Date
June 17, 2026
Flip Icon
Requirements
Requirements
  • Qualified or working toward your Certificate III in Early Childhood Education and Care, Diploma or Bachelor
  • Unqualified work in OSHC but must have experience working with children
  • Transport or access to Public tranaport and a willingness to travel to different locations around your area
  • Motivation to work with children to implement quality, engaging, and activity-based programs
  • An understanding of the Early Learning Framework
  • Australian work rights and a willingness to gain your Working With Children Card
  • Cert 3
  • Diploma
  • Bachelor or unqualified for OSHC but must have experience working with children
What we offer
What we offer
  • flexible hours
  • access to your pay when you need it
  • choose when and how you work via our app
  • ECEC retention payment for shifts at approved centres
  • discounts on petrol, groceries, food delivery, entertainment
  • Fulltime
Read More
Arrow Right
New

Filling Operator - Day shift (T line)

Location
Location
Canada , Alliston
Salary
Salary:
26.55 USD / Hour
https://www.baxter.com/ Logo
Baxter
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Successful completion of High School diploma or GED required (Proof will be requested)
  • Must pass ergonomic assessment to ensure ability to perform bonafide essential job duties
  • Experience doing repetitive tasks
  • Manufacturing/Quality experience in a GMP environment
  • Experience in food or manufacturing industry
  • Previous experience working in a fast-paced environment (experience working in an assembly line setting is preferred)
  • Flexibility to weekends
  • Positive attitude, and an ability to work in a team setting with a focus on quality assurance and attention to detail.
Job Responsibility
Job Responsibility
  • Visual Inspection of IV bag assembly
  • Filling IV solution in various bag sizes
  • placing bags into protective over-wrap
  • loading bags onto sterilizer trays
  • Using scales, weigh filled bags at regular intervals to ensure proper fill volume
  • Visually inspect each filled bag assembly for proper fill volume, air volume, code number, expiration date, lot number, and other quality characteristics
  • Pick up filled bag assembly from conveyor, position bag in over wrapper spec
  • Load over-wrapped bag into sealing machine
  • Load sealed over-wrapped assemblies onto sterilizer trucks ensuring proper number of samples and proper positioning
  • Perform necessary inspections including quality checks throughout each step-in process
What we offer
What we offer
  • Employee Stock Purchasing Plan
  • Defined contribution pension plan
  • medical/dental benefits on your first day
  • Fulltime
Read More
Arrow Right
New

Administrative Assistant

We are looking for an Administrative Assistant to support daily office operation...
Location
Location
United States , Baton Rouge
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous experience in an administrative assistant, receptionist, or general office support role
  • Proficiency with Microsoft Office Suite, including Word, Excel, and Outlook
  • Strong verbal and written communication skills with a detail-oriented approach
  • Ability to manage inbound calls, multitask effectively, and stay organized throughout the day
  • Accurate data entry skills and strong attention to detail
  • Availability to work on-site Monday through Friday in Baton Rouge, Louisiana
Job Responsibility
Job Responsibility
  • Manage day-to-day administrative tasks to keep office operations organized and running efficiently
  • Welcome visitors, oversee front desk activities, and create a positive first impression for the office
  • Answer incoming calls, direct inquiries appropriately, and relay messages in a timely manner
  • Prepare, update, and maintain documents, spreadsheets, and records using Microsoft Office applications
  • Enter and verify data with accuracy while supporting reporting and general recordkeeping needs
  • Coordinate schedules, assist with correspondence, and provide administrative support to internal staff as needed
  • Handle routine office duties such as filing, organizing materials, and maintaining an orderly workspace
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • 401(k) plan
  • Free online training
  • Fulltime
Read More
Arrow Right
New

Hr Waste Management Driver

We are seeking experienced and highly skilled HR Waste Management Drivers to joi...
Location
Location
Australia , Rocklea
Salary
Salary:
46.88 AUD / Hour
https://www.randstad.com Logo
Randstad
Expiration Date
June 18, 2026
Flip Icon
Requirements
Requirements
  • A current and valid HR (Heavy Rigid) licence or above is mandatory
  • Proven previous experience operating Front Lift and Rear Lift waste management trucks is essential
  • A solid understanding of driving in and around Brisbane and the greater surrounds
  • Comfortable with early morning starts and the physical nature of the role, which involves bin maneuvering
  • Must be able to pass a pre-employment Drug and Alcohol (D&A) test
  • Valid HR Licence
  • Valid Work Rights
Job Responsibility
Job Responsibility
  • Operating Front Lift and Rear Lift trucks (often referred to as 'Garbo' or 'Dustbin' trucks)
  • Collecting commercial and residential waste bins across assigned routes in the Brisbane region
  • Operating the hydraulic lift mechanism and performing manual intervention (pushing/pulling of wheelie bins) to position them for loading
  • Maintaining a high degree of safety awareness and good judgment when identifying safe bin pickup locations
  • Fulltime
Read More
Arrow Right
New

Corporate Operations Leader

We are seeking a highly driven and strategic Corporate Operations Leader to mana...
Location
Location
Salary
Salary:
Not provided
aramex.com Logo
Aramex
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong leadership and ownership mindset with the ability to drive end-to-end initiatives
  • Excellent analytical and problem-solving skills with a data-driven approach
  • to have a good knowledge of Power BI, data visualization, and performance tracking
  • Familiarity with AI tools, automation, and digital transformation concepts
  • Strong understanding of logistics, last-mile operations, and fleet management
  • Product-oriented mindset with the ability to translate operational needs into system improvements
  • Effective stakeholder management and cross-functional collaboration skills
  • Strong communication and presentation abilities
  • Ability to manage multiple priorities in a fast-paced, dynamic environment
  • High attention to detail with strong follow-up and execution capabilities
Job Responsibility
Job Responsibility
  • Act as the business owner for Fleet Rollout and support in Champion application Rollout ensuring alignment with operational needs
  • Define business requirements, enhancements, and support product roadmap development
  • Monitor application performance, usability, and adoption across markets
  • Coordinate with product and Digital teams to resolve issues and implement improvements
  • Drive user adoption through structured rollout plans, training, and continuous feedback loops
  • Develop and manage crowdsourcing strategies to ensure scalable capacity
  • Identify opportunities to expand fleet supply while maintaining cost efficiency
  • Define, track, and monitor key operational KPIs such as SLA, utilization, cost per delivery, and courier performance
  • Develop performance dashboards and reporting tools using Power BI
  • Analyze performance trends and identify improvement opportunities
Read More
Arrow Right
New

Sub-Assembly Operator

This is where your work makes a difference. At Baxter, we believe every person—r...
Location
Location
Canada , Alliston
Salary
Salary:
28.45 USD / Hour
https://www.baxter.com/ Logo
Baxter
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma/GED
  • Flexibility to work overtime as required
  • Experience in Trouble shooting mechanical problems
  • Demonstrated problem solving skills
  • Experience in working effectively both independently, and as part of a team
  • Proven ability to be proactive and self-motivated on the job
  • Must have the ability to comprehend Baxter Specifications / SOP’s and determine appropriate corrective actions.
Job Responsibility
Job Responsibility
  • Operate and clean membrane/injection site, blue-cap closure machines, Recon machine and CAPD machine to meet or exceed daily accountability/ productivity requirements
  • Load hopper of each machine with parts and packing finished product into totes
  • Ensure equipment is operating within parameters
  • Observe operation to ensure machine are functioning according to specifications and make all necessary adjustments
  • Perform and document ongoing QA checks to achieve quality product using JDE
  • Perform all start up cleaning and checks – Oven test and pressure test for membrane machine
  • Material Handling: Obtain raw materials from staging areas to prepare for shift production
  • Order materials when needed
  • Constant visual check to assure that all machines are performing properly and product is being supplied
  • Perform minor mechanical adjustments as required
What we offer
What we offer
  • Employee Stock Purchasing Plan
  • Defined contribution pension plan
  • medical/dental benefits on your first day
  • Parttime
Read More
Arrow Right