CrawlJobs Logo

Sr Big Data Engineer - Oozie and Pig (GCP)

rackspace.com Logo

Rackspace

Location Icon

Location:
United States

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

116100.00 - 198440.00 USD / Year
Save Job
Save Icon
Apply Position

Job Description:

We are seeking a Senior Big Data Engineer with deep expertise in distributed systems, batch data processing, and large-scale data pipelines. The ideal candidate has strong hands-on experience with Oozie, Pig, the Apache Hadoop ecosystem, and programming proficiency in Java (preferred) or Python. This role requires a deep understanding of data structures and algorithms, along with a proven track record of writing production-grade code and building robust data workflows. This is a fully remote position and requires an independent, self-driven engineer who thrives in complex technical environments and communicates effectively across teams.

Job Responsibility:

  • Design and develop scalable batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, MapReduce, and HBase, with hands-on coding in Java or Python (Java is a must)
  • Must be able to lead Jira Epics
  • Write clean, efficient, and production-ready code with a strong focus on data structures and algorithmic problem-solving applied to real-world data engineering tasks
  • Develop, manage, and optimize complex data workflows within the Apache Hadoop ecosystem, with a strong focus on Oozie orchestration and job scheduling
  • Leverage Google Cloud Platform (GCP) tools such as Dataproc, GCS, and Composer to build scalable and cloud-native big data solutions
  • Implement DevOps and automation best practices, including CI/CD pipelines, infrastructure as code (IaC), and performance tuning across distributed systems
  • Collaborate with cross-functional teams to ensure data pipeline reliability, code quality, and operational excellence in a remote-first environment

Requirements:

  • Bachelor's degree in Computer Science, software engineering or related field of study
  • Experience with managed cloud services and understanding of cloud-based batch processing systems
  • Proficiency in Oozie, Airflow, Map Reduce, Java
  • Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
  • Expertise in public cloud services, particularly in GCP
  • Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
  • Familiarity with BigTable and Redis
  • Experienced in Infrastructure and Applied DevOps principles in daily work
  • Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform
  • Proven experience in engineering batch processing systems at scale
  • 5+ years of experience in customer-facing software/technology or consulting
  • 5+ years of experience with “on-premises to cloud” migrations or IT transformations
  • 5+ years of experience building, and operating solutions built on GCP
  • Proficiency in Oozie and Pig
  • Must be able to lead Jira Epics
  • Proficiency in Java or Python

Additional Information:

Job Posted:
January 05, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Icon
Rackspace - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon
PREMIUM
More languages and countries
+ Unlock 2204 hidden job offers
Languages
English Čeština Deutsch Ελληνικά Español Français +15
Countries
United States United Kingdom India Canada Australia +
See plans
Plans from $2.99 / month

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Sr Big Data Engineer - Oozie and Pig (GCP)

Sr Big Data Engineer - Oozie and Pig (GCP)

Location
Location
United States
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strongest programming language (Java or Python)
  • Experience with data structures and algorithms
  • Experience with frameworks in that language
  • Recent coding experience in that language
  • Years of coding experience in that language
  • Percentage of a week spent coding in that language
  • Specific time frame of experience with Oozie
  • Number of DAGs created in Oozie
  • Number of DAGs refactored in Oozie
  • Specific time frame of experience with Pig
  • Fulltime
Read More
Arrow Right

Lagger

This is a commercial project and therefore, as a Lagger, you will need to have e...
Location
Location
United Kingdom , Lincoln
Salary
Salary:
25.50 GBP / Hour
https://www.wallacehind.com Logo
Wallace Hind
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • TICA Card
  • Tools and PPE
  • Experience of working with Metal, Ductwork, Pipework and Cladding
  • Can communicate well with others already on site and follow instructions given
  • Able to work the full 42.5 hrs per week
  • Can provide a reference if asked
Job Responsibility
Job Responsibility
  • Experience in ductwork, pipework, metal and cladding
  • Communicate well with others on site and follow instructions
  • Work full 42.5 hrs per week
  • Fulltime
Read More
Arrow Right

Team Member - Part Time

You will live for fashion
Location
Location
United Kingdom , Colchester
Salary
Salary:
Not provided
lovisa.com.au Logo
lovisa
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong desire to help and guide customers
  • A drive to deliver great results
  • An eye for fashion and are aware on all the latest trends
  • Strong performance in a face-to-face customer environment
Job Responsibility
Job Responsibility
  • Sell with great customer experience
  • Conduct our Piercing service
  • Replenish sold stock on the move
  • Manage the register
  • Clean up the store at open and close
  • Build great relationships with colleagues and customers
  • Be yourself! Enjoy yourself!
What we offer
What we offer
  • Top-notch training to become a great Team Member (and piercing training!)
  • First-rate Store and Regional Manager to draw support
  • Team Referral Rewards Program
  • Product for you to wear in-store from our top-fashion ranges
  • Funky, on-trend customers
  • Bright, warm store
  • Easy to explain product
  • Oh, and a generous discount on our jewellery!
  • Parttime
Read More
Arrow Right

Asset Protection Coordinator

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , New York
Salary
Salary:
19.00 - 28.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
July 01, 2026
Flip Icon
Requirements
Requirements
  • Physical abilities to support the essential functions of the role as listed above, such as able to stand and/or move throughout the store for the majority of work time in order to provide excellent customer service and conduct effective theft prevention. Able to stoop, knell or crouch, and able to reach or grasp objects, including objects on lower shelves. Able to perform duties requiring manual dexterity, including but not limited to: applying security tags, and using keys to open and close locks
  • Excellent observational skills and able to recall and describe detailed facts. Able to see and hear in order to perform observational duties, including but not limited to: reading small print on labels, tags or containers
  • Able to engage and communicate well with customers and provide excellent customer service
  • Able to communicate effectively, both verbally and in writing. Able to prepare and submit external theft reports as needed. Able to communicate effectively with both Asset Protection and Store Leadership Teams
  • Work a flexible schedule that will include nights and weekends
  • 1 to 3 years experience
  • High School Diploma or General Equivalent Development (GED)
Job Responsibility
Job Responsibility
  • Provides attentive, friendly customer service. Greets all customers, provides answers to general customer questions, and assists with opening and securing locked shelves and showcases per customer requests
  • Moves throughout the store to monitor assigned store(s) for activity that causes the loss of CVS Pharmacy assets/products. Communicates with Store Management about observations and asks Store Management for guidance/assistance if theft is suspected. Does not apprehend suspected shoplifters
  • Inspects high-risk product categories regularly to detect and deter theft activity
  • Ensures use of approved asset/product protection equipment and the security labeling program
  • Prepares External Theft reports in Auror system as incidents occur and monitors feeds to gather and share intelligence on known repeat offenders with key stakeholders
  • Provide customer service to prevnt theft and reduce friction caused by product protection by unlocking and securing devices as needed
  • External Theft Reporting and Review potential links of habitual offenders in Auror System. Proactive Intelligence Sharing with DAPL. ORC Team and Store Manager
  • Monitor store compliance with AP related programs that prevent external theft. Take corrective action within this job description to remedy any vulnerabilities. Report findings to DAPL and Store Leadership
What we offer
What we offer
  • medical, dental, and vision coverage
  • paid time off
  • retirement savings options
  • wellness programs
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data

As an Engineer on the Earner Data Intelligence team, you will be dealing with la...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
uber.com Logo
Uber
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of extensive Data engineering experience working with large data volumes and different sources of data
  • Strong data modeling skills, domain knowledge and domain mapping experience
  • Strong experience of using SQL language and writing complex queries
  • Experience with using other programming languages like Java, Scala, Python
  • Good problem solving and analytical skills
  • Good communication, mentoring and collaboration skills
Job Responsibility
Job Responsibility
  • Responsible for defining the Source of Truth (SOT), Dataset design for multiple Uber teams
  • Identify unified data models collaborating with Data Science teams
  • Streamline data processing of the original event sources and consolidate them in source of truth event logs
  • Build and maintain real-time/batch data pipelines that can consolidate and clean up usage analytics
  • Build systems that monitor data losses from the different sources and improve the data quality
  • Own the data quality and reliability of the Tier-1 & Tier-2 datasets including maintaining their SLAs, TTL and consumption
  • Devise strategies to consolidate and compensate the data losses by correlating different sources
  • Solve challenging data problems with cutting edge design and algorithms
  • Fulltime
Read More
Arrow Right

Shift Supervisor

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , Lafayette
Salary
Salary:
16.00 - 23.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
June 15, 2026
Flip Icon
Requirements
Requirements
  • Deductive reasoning ability, analytical skills and computer skills
  • Advanced communication skills and supervision skills
  • Ability to work a flexible schedule, including some early morning, overnight and weekend shifts, to work overtime as needed, and to respond to urgent issues at the store when they arise
Job Responsibility
Job Responsibility
  • Work effectively with store management and store crews
  • Supervise the store’s crew through assigning, directing and following up of all activities
  • Effectively communicate information both to and from store management and crews
  • Assist customers with their questions, problems and complaints
  • Promote CVS customer service culture (Greet, offer help, and thank)
  • Handle all customer relations issues in accordance with company policy and promote a positive shopping experience for all CVS customers
  • Maintain customer/patient confidentiality
  • Price merchandise
  • Stock shelves
  • Execute the displays, sign and inventory of weekly, promotional, and seasonal merchandise
What we offer
What we offer
  • dental
  • vision
  • wellness resources
  • employee discounts
  • access to certain voluntary benefits
  • other programs
  • Parttime
Read More
Arrow Right

Senior Garment Technologist

We are looking for an experienced Senior Garment Technologist to join our team a...
Location
Location
United Arab Emirates; United Kingdom , Dubai; Manchester
Salary
Salary:
Not provided
clubllondon.com Logo
Club L London
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous experience as senior technologist is preferred but not essential
  • Preferably a degree or similar within technical textiles
  • Ability to fit accurately and at speed to ensure Club L London are first to market with the latest trends
  • Strong understanding of pattern cutting and garment construction, a pattern cutting background is preferred but not essential
  • Attention to detail is key to maintain the brands quality standards
  • An understanding of testing and compliance is essential
  • Passionate about the Club L London product, customer focused attitude and commercial eye
  • Must be organised and able to lead and work efficiently as part of a team in a fast-paced environment
  • Desire to progress and grow within a team, taking on extra responsibilities where relevant
Job Responsibility
Job Responsibility
  • Working closely with the design, production, and merchandising team throughout the product development process
  • Own and lead live fit sessions, working collaboratively with the design team to achieve the perfect Club L London product
  • Creation of comprehensive, logical, and clearly written fit comments complete with supporting imagery and diagrams
  • Communicating daily with a global supply base to ensure all products are manufactured to the required fit and quality standards
  • Ability to prioritise workload to ensure critical business deadlines are achieved
  • Approval of care labels and test reports
  • Ownership of departmental product returns, investigating high returning styles and bringing solutions to achieve company returns targets
  • Develop junior members of the team to support progression within the department
  • Support and educate suppliers to achieve consistent quality and fit across a global supply base
  • Implement departmental projects to standardise and improve ways of working
What we offer
What we offer
  • Flexible Working Hours and Early Finish Fridays
  • 40% Staff Discount across Club L and Lavish Alice products
  • Bi-Annual Bonus Scheme
  • Fulltime
Read More
Arrow Right

Entry-Level Crewmember

At Sierra Pacific Industries, we understand our greatest strength is the people ...
Location
Location
United States , Red Bluff
Salary
Salary:
21.31 USD / Hour
spi-ind.com Logo
Sierra Pacific Industries
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Recent, related work history and verifiable references
  • At least age 18
Job Responsibility
Job Responsibility
  • Load, package, and stack windows, doors, and window parts
  • Work with a 2–3-person crew to package and load shipping trucks
  • Work in a fast-paced, at times demanding environment
  • Repetitive lifting
  • Work in both hot and cold work environments
  • Work various shifts, with some overtime and weekend work during busy production times
What we offer
What we offer
  • Low-cost health benefits
  • Retirement plan with employer-paid contributions
  • Paid vacation and 10 holidays
  • Full Health Benefits Plan (including medical, dental, and vision) with low-cost premiums
  • 401(k) with Company Match
  • Additional Retirement Contributions
  • Company-paid Life Insurance
  • Fulltime
Read More
Arrow Right