CrawlJobs Logo
Rackspace Logo Rackspace · IT - Software Development

Sr Big Data Engineer - Oozie and Pig (GCP)

United States 116100.00 - 198440.00 USD / Year · Job Posted January 05, 2026
Apply Position
Job Link Share

Job Description

We are seeking a Senior Big Data Engineer with deep expertise in distributed systems, batch data processing, and large-scale data pipelines. The ideal candidate has strong hands-on experience with Oozie, Pig, the Apache Hadoop ecosystem, and programming proficiency in Java (preferred) or Python. This role requires a deep understanding of data structures and algorithms, along with a proven track record of writing production-grade code and building robust data workflows. This is a fully remote position and requires an independent, self-driven engineer who thrives in complex technical environments and communicates effectively across teams.

Job Responsibility

  • Design and develop scalable batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, MapReduce, and HBase, with hands-on coding in Java or Python (Java is a must)
  • Must be able to lead Jira Epics
  • Write clean, efficient, and production-ready code with a strong focus on data structures and algorithmic problem-solving applied to real-world data engineering tasks
  • Develop, manage, and optimize complex data workflows within the Apache Hadoop ecosystem, with a strong focus on Oozie orchestration and job scheduling
  • Leverage Google Cloud Platform (GCP) tools such as Dataproc, GCS, and Composer to build scalable and cloud-native big data solutions
  • Implement DevOps and automation best practices, including CI/CD pipelines, infrastructure as code (IaC), and performance tuning across distributed systems
  • Collaborate with cross-functional teams to ensure data pipeline reliability, code quality, and operational excellence in a remote-first environment

Requirements

  • Bachelor's degree in Computer Science, software engineering or related field of study
  • Experience with managed cloud services and understanding of cloud-based batch processing systems
  • Proficiency in Oozie, Airflow, Map Reduce, Java
  • Strong programming skills with Java (specifically Spark), Python, Pig, and SQL
  • Expertise in public cloud services, particularly in GCP
  • Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce
  • Familiarity with BigTable and Redis
  • Experienced in Infrastructure and Applied DevOps principles in daily work
  • Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform
  • Proven experience in engineering batch processing systems at scale
  • 5+ years of experience in customer-facing software/technology or consulting
  • 5+ years of experience with “on-premises to cloud” migrations or IT transformations
  • 5+ years of experience building, and operating solutions built on GCP
  • Proficiency in Oozie and Pig
  • Must be able to lead Jira Epics
  • Proficiency in Java or Python
Rackspace - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Sr Big Data Engineer - Oozie and Pig (GCP)

8 matching positions

Sr Big Data Engineer - Oozie and Pig (GCP)

Location
Location
United States
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strongest programming language (Java or Python)
  • Experience with data structures and algorithms
  • Experience with frameworks in that language
  • Recent coding experience in that language
  • Years of coding experience in that language
  • Percentage of a week spent coding in that language
  • Specific time frame of experience with Oozie
  • Number of DAGs created in Oozie
  • Number of DAGs refactored in Oozie
  • Specific time frame of experience with Pig
  • Fulltime
Read More
Arrow Right
New

Client Engagement Manager

As a Client Engagement Manager, you'll be responsible for overseeing procurement...
Location
Location
United Kingdom , City of London
Salary
Salary:
Not provided
socialvalueportal.com Logo
Social Value Portal Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Solid foundation in procurement delivery
  • Strong understanding of sourcing processes
  • Supplier management skills
  • Stakeholder engagement skills
  • Ability to navigate complex client environments
  • Ability to align procurement activities with strategic goals
  • Ability to resolve delivery challenges with a hands-on, solution-oriented approach
  • Strong communication skills
  • Analytical thinking
  • Ability to work with performance data
Job Responsibility
Job Responsibility
  • Managing client relationships and ensuring procurement services are delivered effectively
  • Supporting strategic sourcing plans and supplier engagement initiatives
  • Leading governance activities and reporting on procurement performance
  • Identifying and resolving delivery challenges across procurement workstreams
  • Collaborating with internal teams to ensure alignment with client objectives
  • Driving procurement value initiatives and continuous improvement
What we offer
What we offer
  • Competitive salary and benefits package
  • Opportunities to influence procurement strategy
  • Opportunities to contribute to high-impact client initiatives
  • Collaborative and structured working culture
  • Strong focus on professional development and continuous improvement
  • Fulltime
Read More
Arrow Right
New

Account Executive, Business Sales

The Account Executive, Business Sales role at T-Mobile is designed for ambitious...
Location
Location
United States , Downers Grove
Salary
Salary:
71700.00 - 129500.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High School Diploma/GED (Required)
  • 1+ years verifiable new customer acquisition sales experience, preferably within a commissioned environment (Preferred)
  • Outside B2B sales experience (Preferred)
  • Task Management Ability to work well in a dynamic, fast changing environment that requires a high degree of multi-tasking (Required)
  • Customer Service Demonstrated experience delivering superior customer service and attention to detail (Required)
  • Communication Excellent interpersonal, written, and oral communication skills (Required)
  • Negotiation Effective negotiating and closing skills, including communication, emotional intelligence, and problem-solving (Required)
  • At least 18 years of age
  • Legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Generate and work leads through prospecting, cold calling, and networking under sales manager supervision
  • Identify customer needs and use solution-based selling to demonstrate T-Mobile's value. Recommend wireless solutions, including price plans, data services, handsets, and accessories
  • Negotiate and close deals
  • Develop skills in prospecting, call execution, and relationship management with leadership. Participate in product training and sales meetings
  • Create effective sales approaches, solutions, and proposals
  • Utilize sales force automation, manage sales funnel, and report on sales activities and forecasts
  • Maintain and grow the customer base within a territory model
What we offer
What we offer
  • Medical, dental and vision insurance
  • Flexible spending account
  • 401(k)
  • Employee stock grants
  • Employee stock purchase plan
  • Paid time off and up to 12 paid holidays
  • Paid parental and family leave
  • Family building benefits
  • Back-up care
  • Enhanced family support
  • Fulltime
Read More
Arrow Right
New

Project Quantity Surveyor

We are recruiting for a Project Quantity Surveyor to join a growing engineering ...
Location
Location
United Kingdom , Rochester
Salary
Salary:
55000.00 - 65000.00 GBP / Year
socialvalueportal.com Logo
Social Value Portal Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 2 years’ experience in a Quantity Surveying or commercial role
  • Experience within construction, utilities, infrastructure, or water sectors
  • Strong understanding of cost reporting and forecasting
  • NEC3 and/or NEC4 contract experience
  • Good commercial awareness and negotiation skills
  • Ability to manage workload independently
  • Strong communication and stakeholder management skills
  • Proficient in Microsoft Excel and Microsoft Word
  • Degree or equivalent qualification in Quantity Surveying or related discipline preferred
Job Responsibility
Job Responsibility
  • Manage project commercial performance and cost control
  • Prepare and submit applications for payment
  • Administer subcontractor accounts and valuations
  • Support procurement of subcontractors and suppliers
  • Produce cost reports, forecasts, and cash flow updates
  • Monitor project budgets and commercial risks
  • Assist with change management and variations
  • Support final accounts and commercial close-out
  • Ensure compliance with NEC3 / NEC4 contracts
  • Liaise with clients, site teams, and key stakeholders
  • Fulltime
Read More
Arrow Right
New

Bids & Pursuits Manager - Disputes, Tax & Employment

Our Bids & Pursuits team are enthusiastic and passionate business development pr...
Location
Location
United Kingdom , East London
Salary
Salary:
Not provided
socialvalueportal.com Logo
Social Value Portal Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years' experience in a pursuit and bid role in legal, professional services or other relevant environments with a proven track record of winning strategic bids
  • Strong technical capability in leading pursuits, including stakeholder management and project management skills, and a proven ability to successfully manage work-winning processes
  • Experience supporting bid teams throughout the bid lifecycle, including critical meetings, presentations and commercial negotiations
  • Excellent organisational skills
  • ability to manage multiple opportunities simultaneously, work to timescales and deadlines set by clients
  • Experience of managing complex opportunities across practice, borders with multiple stakeholders and challenging timelines
Job Responsibility
Job Responsibility
  • Work with DIA fee-earners, CMG team members, and other stakeholders to define the bid strategy with a clear value proposition and win themes for all bids
  • Manage end to end process for DIA bids and pursuits from qualification to submissions, presentations and aftercare, including content management
  • Champion Ashurst's global bid & pursuit methodology, drive best practice and continuous improvement across pursuits and bids
  • Work closely with CMG colleagues to coach, advise and influence partners and other key stakeholders on opportunity qualification and pursuit strategy
  • Lead the development of win strategies, considering relationship, commerciality, competitive positioning and pursuit intelligence
What we offer
What we offer
  • competitive remuneration with the flexibility to reward high performance
  • flexible working
  • corporate health plans
  • a global professional development offering for all employees
  • an industry-leading programme that celebrates diversity and inclusion
  • Fulltime
Read More
Arrow Right
New

Merchandise Manager - Portmans

The Portmans team have two fantastic opportunities available for experienced Mer...
Location
Location
Australia , St Kilda Road
Salary
Salary:
Not provided
justgroup.com.au Logo
Just Group
Expiration Date
June 07, 2026
Flip Icon
Requirements
Requirements
  • Proven experience in fashion retail buying/product management and product development
  • Strong supplier management, negotiation, and relationship-building skills
  • Demonstrated leadership capability with a focus on team development and performance growth
  • Passion for coaching, developing, and growing high-performing teams
  • Tertiary qualification in Fashion, Design, Fabric Construction, or Pattern Making
  • Highly commercial mindset with strong retail and financial acumen
  • Track record of exceeding key buying and merchandising KPIs
  • Experience managing and delivering against sales, OTB, and gross profit budgets
  • Strong understanding of inventory management and expense control
  • Ability to curate and deliver commercially successful product ranges
Job Responsibility
Job Responsibility
  • Lead, coach, and develop a high-performing merchandising and product team, including succession planning
  • Lead seasonal buying strategies, budgets, and category financial performance (including OTB, GP targets, and sales outcomes)
  • Drive the design direction, trend forecasting, and development of commercially viable product ranges
  • Oversee product selection, styling, and fabric direction aligned with brand strategy
  • Collaborate cross-functionally (Executive, Finance, Retail, Supply Chain, Marketing, VM) to align on product, promotions, and business objectives
  • Build and manage supplier partnerships to ensure on-time, on-quality, and on-cost delivery, while identifying new product opportunities
  • Own in-season trading performance, driving promotional strategies and managing inventory to maximise sales and margin
  • Identify and act on growth opportunities including new categories, product innovations, and collaborations
  • Monitor market trends, competitor activity, and global insights to maintain a strong competitive edge
  • Conduct seasonal reviews and contribute to broader brand strategy and long-term planning
What we offer
What we offer
  • Team member discounts: Generous discounts in-store and online
  • Learning and development: Capability development workshops, succession planning, and internal career opportunities across Myer Group brands
  • Lifestyle benefits: Employee Assistance Program (EAP), volunteer leave, discounted health insurance, gym memberships and novated leasing for eligible employees
  • Fulltime
Read More
Arrow Right
New

Purchasing Coordinator

We are looking for a detail-oriented Purchasing Coordinator to support procureme...
Location
Location
United States , Hanford
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2+ years of experience in purchasing, procurement, or inventory coordination within a manufacturing environment
  • Working knowledge of purchase order processing, material purchasing, and general purchasing department operations
  • Experience supporting inventory control activities such as cycle counts, annual inventory, stock replenishment, or asset tracking
  • Ability to evaluate pricing and purchasing data to make practical, cost-conscious decisions
  • Strong organizational skills with the ability to manage multiple orders, deadlines, and supplier follow-up activities
  • Familiarity with GMP or similar regulated manufacturing standards
  • Clear communication skills for coordinating with suppliers and internal departments
  • Proficiency with standard business systems and tools used for procurement documentation and reporting
Job Responsibility
Job Responsibility
  • Source and purchase materials, services, equipment, repair items, tools, and plant supplies needed to support daily operations
  • Create and release purchase orders in alignment with company policies, approval requirements, and established procurement procedures
  • Monitor inventory activity by maintaining replenishment levels, reviewing non-stock requests, processing returns, and helping identify excess or obsolete materials
  • Partner with suppliers to secure competitive pricing, dependable service, and appropriate stock availability for production and facility requirements
  • Review requisitions, orders, and related purchasing documentation to confirm accuracy and compliance with internal standards
  • Track open orders, follow up on delayed shipments or backorders, and communicate expected delivery timing to internal teams
  • Analyze purchasing and inventory data to support cost savings, avoid unnecessary spend, and improve overall supply chain performance
  • Assess material demand for manufacturing and plant upkeep to help ensure critical items are available when needed
  • Assist with month-end and annual inventory counting activities while supporting inventory accuracy and reporting
  • Follow workplace safety requirements, good manufacturing practices, and food safety and quality standards in all purchasing-related activities
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • company 401(k) plan
Read More
Arrow Right
New

Merchandise Brand Development Assistant

Myer is seeking a proactive and detail-oriented Merchandise Brand Development As...
Location
Location
Australia , Docklands
Salary
Salary:
Not provided
justgroup.com.au Logo
Just Group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Highly organised with strong problem-solving skills
  • Strong attention to detail and ability to manage multiple priorities
  • Confident communicator with the ability to engage stakeholders at all levels
  • Intermediate skills in Microsoft Office (Outlook, Word, Excel, PowerPoint)
  • Comfortable using various systems and technology tools
  • Passion for cosmetics and retail environments
Job Responsibility
Job Responsibility
  • Coordinate communication and information flow between suppliers, support office, and stores
  • Maintain and update project timelines, WIP trackers, and master floorplan documentation
  • Collate and manage store requests for weekly review
  • Support new counter launches, including stock fit-outs
  • Track and coordinate pop-up installations and temporary space allocations
  • Manage supplier and store queries related to store development
  • Maintain accurate brand and stakeholder contact databases
  • Conduct competitor store visits and share best-in-class insights
  • Support VM planning and execution for vacant retail spaces
  • Build strong working relationships across Buying, Planning, Space Planning, Stores, and external partners
What we offer
What we offer
  • Team member discounts: Generous discounts in-store and online
  • Learning and development: Capability development workshops, succession planning, and internal career opportunities across Myer Group brands
  • Flexible working: Lifestyle leave, flexible work hours and hybrid working is available for Store Support Office roles
  • Lifestyle benefits: Employee Assistance Program (EAP), volunteer leave, discounted health insurance, gym memberships and novated leasing for eligible employees
  • Fulltime
Read More
Arrow Right