CrawlJobs Logo
Rackspace Logo Rackspace · IT - Software Development

Sr Big Data Engineer - Oozie and Pig (GCP)

United States 116100.00 - 198440.00 USD / Year · Job Posted January 05, 2026
Apply Position
Job Link Share

Job Description

We are seeking a Senior Big Data Engineer with deep expertise in distributed systems, batch data processing, and large-scale data pipelines. The ideal candidate has strong hands-on experience with Oozie, Pig, the Apache Hadoop ecosystem, and programming proficiency in Java (preferred) or Python. This role requires a deep understanding of data structures and algorithms, along with a proven track record of writing production-grade code and building robust data workflows. This is a fully remote position and requires an independent, self-driven engineer who thrives in complex technical environments and communicates effectively across teams.

Job Responsibility

  • Design and develop scalable batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, MapReduce, and HBase, with hands-on coding in Java or Python (Java is a must)
  • Must be able to lead Jira Epics
  • Write clean, efficient, and production-ready code with a strong focus on data structures and algorithmic problem-solving applied to real-world data engineering tasks
  • Develop, manage, and optimize complex data workflows within the Apache Hadoop ecosystem, with a strong focus on Oozie orchestration and job scheduling
  • Leverage Google Cloud Platform (GCP) tools such as Dataproc, GCS, and Composer to build scalable and cloud-native big data solutions
  • Implement DevOps and automation best practices, including CI/CD pipelines, infrastructure as code (IaC), and performance tuning across distributed systems
  • Collaborate with cross-functional teams to ensure data pipeline reliability, code quality, and operational excellence in a remote-first environment

Requirements

  • Bachelor's degree in Computer Science, software engineering or related field of study
  • Experience with managed cloud services and understanding of cloud-based batch processing systems
  • Proficiency in Oozie, Airflow, Map Reduce, Java
  • Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
  • Expertise in public cloud services, particularly in GCP
  • Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
  • Familiarity with BigTable and Redis
  • Experienced in Infrastructure and Applied DevOps principles in daily work
  • Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform
  • Proven experience in engineering batch processing systems at scale
  • 5+ years of experience in customer-facing software/technology or consulting
  • 5+ years of experience with “on-premises to cloud” migrations or IT transformations
  • 5+ years of experience building, and operating solutions built on GCP
  • Proficiency in Oozie and Pig
  • Must be able to lead Jira Epics
  • Proficiency in Java or Python
Rackspace - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Sr Big Data Engineer - Oozie and Pig (GCP)

8 matching positions

Sr Big Data Engineer - Oozie and Pig (GCP)

Location
Location
United States
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strongest programming language (Java or Python)
  • Experience with data structures and algorithms
  • Experience with frameworks in that language
  • Recent coding experience in that language
  • Years of coding experience in that language
  • Percentage of a week spent coding in that language
  • Specific time frame of experience with Oozie
  • Number of DAGs created in Oozie
  • Number of DAGs refactored in Oozie
  • Specific time frame of experience with Pig
  • Fulltime
Read More
Arrow Right
New

Mobile Associate - Retail Sales

This role supports retail operations by engaging customers and facilitating thei...
Location
Location
United States , Carlyle
Salary
Salary:
18.00 - 23.00 USD / Hour
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High School Diploma/GED (Required)
  • 6 months of customer service and/or sales experience, Retail environment (Preferred)
  • Passionate customer advocate with the desire to be yourself when connecting and having fun doing it! (Required)
  • Competitive drive and proven ability to succeed in a fast-paced sales environment (Required)
  • Willingness to work alongside peers and store leaders, learning and sharing ideas, while serving customers and providing resolutions to issues (Required)
  • Effective at balancing customer needs and performance goals (Required)
  • At least 18 years of age
  • Legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Develop proficiency in customer service and sales to deliver personalized technology and service solutions that meet individual needs
  • Utilize digital tools to demonstrate network coverage, service plans, and product features to enhance customer understanding and engagement
  • Complete required training to build knowledge of retail processes, systems, and wireless technology innovations
  • Collaborate with colleagues across channels to support a seamless customer experience and contribute to team initiatives
  • Also responsible for other duties/projects as assigned by business management as needed
What we offer
What we offer
  • Medical insurance
  • Dental insurance
  • Vision insurance
  • Flexible spending account
  • 401(k)
  • Employee stock grants
  • Employee stock purchase plan
  • Paid time off
  • Up to 12 paid holidays
  • Paid parental and family leave
  • Parttime
Read More
Arrow Right
New

Guest Experience Expert

Our jobs aren’t just about giving guests a smooth check-in and check-out. Instea...
Location
Location
Dominican Republic , Cap Cana
Salary
Salary:
Not provided
https://www.marriott.com Logo
Marriott Bonvoy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or G.E.D. equivalent
  • No related work experience
  • No supervisory experience
  • None
Job Responsibility
Job Responsibility
  • Giving guests a smooth check-in and check-out
  • Building a memorable and unique experience
  • Delivering a wide range of services that guide guests through their entire stay
  • Processing operational needs
  • Addressing guest requests
  • Completing reports
  • Sharing the highlights of the local area
  • Creating a safe work place
  • Following company policies and procedures
  • Maintaining confidentiality
  • Fulltime
Read More
Arrow Right
New

Director, Human Resources Business Partner

As the Director, Human Resources Business Partner, you'll partner with business ...
Location
Location
United States , Needham
Salary
Salary:
163500.00 - 257500.00 USD / Year
sharkninja.com Logo
SharkNinja
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 8-10 years of HR experience with a blend of business partnering and talent management/development focus, including organizational design and effectiveness, team building, performance management, career coaching, and leadership development
  • Bachelor's degree required
  • Master's degree preferred
  • Global experience desired
  • consumer products industry highly preferred but not required
  • Strong organization, planning, and project management skills
  • ability to prioritize tasks for both self and team to meet business requirements and deadlines
  • Effective working across functions and levels to deliver business results
  • Strong written and verbal communication
  • able to communicate with conviction, listen actively, and lead difficult conversations at every level
Job Responsibility
Job Responsibility
  • Act as a trusted advisor to the operations leadership, driving P&C strategies that support business objectives, foster innovation, and enhance organizational effectiveness. Build deep, sustained relationships with senior leaders through consistent presence and proactive engagement
  • Align the P&C agenda to the corporate strategy and the key business drivers for the business unit supported
  • Shape talent strategy for your client groups — build the leadership pipeline, run succession planning, and drive end-to-end strategic workforce planning in partnership with the business
  • Lead change. Drive P&C initiatives and process improvements end-to-end, and champion SharkNinja's culture through how we hire, develop, and retain talent
  • Coach and mentor associates at every level — as an integrated member of the business unit, not an outside function
  • Design the organization. Partner with leaders to build structures, processes, and cultures that drive agility and high performance. Facilitate in-person workshops, leadership offsites, and team effectiveness sessions
  • Use data to drive change. Analyze people trends, surface complex issues, identify capability gaps, and recommend the programs and structural changes that close them
  • Shape the talent development agenda. Diagnose capability gaps and partner with the Talent Development team to design, broker, and deliver the programs that close them
  • Communicate P&C policy changes clearly across your business units and champion cross-functional collaboration
  • Oversee local compliance with employment law and regulations across teams and geographies
What we offer
What we offer
  • medical insurance
  • dental insurance
  • vision insurance
  • flexible spending accounts
  • health savings accounts (HSA) with company contribution
  • 401(k) retirement plan with matching
  • employee stock purchase program
  • life insurance
  • AD&D
  • short-term disability insurance
  • Fulltime
Read More
Arrow Right
New

Neurophys Fellow - Junior

This position involves managing and testing patients with neuromuscular, neuroim...
Location
Location
Australia , Melbourne
Salary
Salary:
119876.00 - 158878.00 AUD / Year
austin.org.au Logo
Austin Health
Expiration Date
June 14, 2026
Flip Icon
Requirements
Requirements
  • Registration as a Medical Practitioner with the Australian Health Practitioner Regulation Agency (AHPRA)
  • Appropriate training and experience applicable to the field of Neurology
  • Commitment to high quality patient care
Job Responsibility
Job Responsibility
  • Care of outpatients, inpatients and Ambulatory Care Centre patients with neuromuscular diseases, including genetic, degenerative, toxic and autoimmune peripheral nerve and muscle diseases
  • Supervised testing of outpatients and inpatients referred for nerve conduction studies (NCS) and electromyography (EMG)
  • Regular attendance at outpatient clinics each week, including General Neurology and Neuromuscular Clinic
What we offer
What we offer
  • Salary packaging for living expenses, meals, and holiday accommodation
  • Discounted fitness memberships
  • Special offers on health insurance
  • Comprehensive wellbeing program
  • Onsite childcare at Austin Hospital
  • Easy access via public transport and car parking options
  • Mentoring and career development opportunities
  • Fulltime
Read More
Arrow Right
New

Endocrinologist - Gender Clinic

As the Endocrinologist – Gender Clinic, you will provide specialist endocrine le...
Location
Location
Australia , Heidelberg
Salary
Salary:
Not provided
austin.org.au Logo
Austin Health
Expiration Date
June 08, 2026
Flip Icon
Requirements
Requirements
  • A strong clinical background in Endocrinology with a specific interest or experience in gender-affirming hormone therapy (GAHT) and the healthcare needs of transgender and gender-diverse populations
  • Specialist registration with AHPRA and Fellowship of the Royal Australasian College of Physicians (FRACP) or equivalent
  • A demonstrated commitment to providing inclusive, culturally safe, and non-judgmental care that respects the autonomy of gender-diverse patients
  • Excellent interpersonal and communication skills, with a proven ability to work effectively within a multidisciplinary team and across statewide health networks
  • A drive for continuous service improvement, clinical research, or education to help evolve the Gender Clinic's specialist offerings
Job Responsibility
Job Responsibility
  • Provide expert clinical leadership and endocrine consultation for transgender and gender-diverse (TGD) individuals, specifically focused on initiating and managing gender-affirming hormone therapy (GAHT)
  • Drive clinical throughput in specialized Gender Clinic sessions, ensuring patients have timely access to specialist care and reducing waitlist pressures
  • Work closely with the broader Endocrinology department and statewide services to provide holistic, patient-centred care for complex endocrine presentations
  • Support the development of junior medical staff and registrars, sharing specialised knowledge in TGD health to build future capacity within the service
  • Participate in clinical review and service development initiatives to maintain Austin Health's position as a leader in statewide specialty services
What we offer
What we offer
  • Greater take-home pay through generous salary packaging, for living expenses, meals, and holiday accommodation
  • A suite of wellness initiatives designed to support you, including discounted fitness memberships and special offers on health insurance, and a comprehensive wellbeing program
  • Onsite childcare at Austin Hospital, easy access via public transport and car parking options
  • Mentoring and career development opportunities
  • Parttime
Read More
Arrow Right
New

Conflicts Analyst

We are looking for a detail-oriented Conflicts Analyst to support legal operatio...
Location
Location
United States , Louisville
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience supporting legal or paralegal functions in a litigation-focused environment
  • Working knowledge of discovery processes and civil litigation procedures
  • Familiarity with trial preparation tasks, including file and exhibit organization
  • Proficiency with case management software and maintaining accurate electronic records
  • Strong analytical skills with the ability to review information carefully and identify inconsistencies
  • Excellent organizational and communication skills with a high level of attention to detail
Job Responsibility
Job Responsibility
  • Review new matters and parties to identify potential conflicts of interest and escalate issues for further analysis when needed
  • Maintain accurate case and client information within case management software to support organized legal operations
  • Assist attorneys and legal staff with discovery coordination, including tracking documents and deadlines
  • Support civil litigation workflows by preparing matter summaries, updating records, and monitoring case activity
  • Contribute to trial preparation by organizing files, exhibits, and supporting materials for attorney review
  • Conduct database searches and compile relevant background information related to parties, matters, and prior engagements
  • Help ensure documentation is complete, consistent, and accessible for ongoing litigation and compliance needs
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • 401(k) plan
  • Free online training
  • Access to top jobs
  • Competitive compensation
Read More
Arrow Right
New

Neurologist

As the Neurology Consultant you will be responsible for: Medical oversight of th...
Location
Location
Australia , Heidelberg
Salary
Salary:
Not provided
austin.org.au Logo
Austin Health
Expiration Date
June 07, 2026
Flip Icon
Requirements
Requirements
  • Be a registered Medical Practitioner with the Australian Health Practitioner Regulation Agency (AHPRA)
  • Hold a Fellowship of the [relevant Australian Learned College (FRACP), or equivalent
  • Have appropriate training and experience applicable to the field of Stroke
  • Demonstrate a commitment to high quality patient care
  • Demonstrate a commitment to working as a team member with peer medical, junior medical, nursing and other clinical staff to provide efficient, safe and quality care across the continuum, and
  • Demonstrate a commitment to teaching, training and research
  • Demonstrated independent expertise in neurosonology, including: carotid and vertebral duplex ultrasound
  • transcranial Doppler (TCD), including emboli detection and vasomotor reactivity studies
  • and agitated saline contrast studies for right-to-left shunt / PFO detection
  • Recognised neurosonology credentialing or equivalent demonstrated competency (e.g., ASN Neurosonology certification, RVT, ESNCH/ANIM accreditation), or willingness and capacity to obtain within an agreed timeframe
Job Responsibility
Job Responsibility
  • Medical oversight of the Neurodiagnostics Laboratory, encompassing carotid duplex ultrasound, transcranial Doppler (TCD), and right-to-left shunt / patent foramen ovale (PFO) detection studies
  • Day-to-day operations of the laboratory are managed by the Neurodiagnostics Lab Manager
  • this role provides medical and clinical governance input only
  • Clinical and academic leadership of a dedicated Vascular Neurology subspecialty clinic, with development of a focused clinical and translational research program in an area of neurovascular medicine of the appointee's choosing (for example, but not limited to, cerebral amyloid angiopathy, vascular cognitive impairment, intracerebral haemorrhage, neurocardiology, or cerebral vasculopathies)
What we offer
What we offer
  • Greater take-home pay through generous salary packaging, for living expenses, meals, and holiday accommodation
  • A suite of wellness initiatives designed to support you, including discounted fitness memberships and special offers on health insurance, and a comprehensive wellbeing program
  • Onsite childcare at Austin Hospital, easy access via public transport and car parking options
  • Mentoring and career development opportunities
  • Parttime
Read More
Arrow Right