CrawlJobs Logo
Realign Logo Realign · IT - Software Development

Gcp engineer with Bigquery, Pyspark

United States, Phoenix 110000.00 USD / Year · Job Posted March 21, 2026
Apply Position
Job Link Share

Job Description

Role - GCP engineer with Bigquery, Pyspark

Requirements

  • 8+ years of professional experience as a Java Engineer
  • Strong knowledge of Java languages and web development frameworks like Spring, Hibernate, and Struts.
  • Expertise in developing web applications using front-end technologies (HTML, CSS, and JavaScript).
  • Develop and maintain SpringBoot applications using Java programming language.
  • Knowledge of RESTful web services and API development
  • Experience deploying microservice architecture, applications, and supporting services
  • Experience working on GCP application Migration for large enterprise
  • Familiar with software security best practices
  • Understanding of monitoring tools
  • Experience working within large-scale decoupled, service-oriented systems a plus
  • Strong analytical and problem-solving skills with organizational capabilities.
  • Familiarity with cloud technologies (Google Cloud).

Nice to have

Experience working within large-scale decoupled, service-oriented systems

Realign - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Gcp engineer with Bigquery, Pyspark

8 matching positions

Senior BigQuery Engineer

You will join the OneMIS stream, responsible for management, regulatory & risk r...
Location
Location
Romania , Bucharest
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • University degree in computer science or a comparable qualification
  • At least 5 years of experience as a Data Engineer, building scalable data pipelines and working with cloud-based data ecosystems
  • Strong expertise in SQL and hands‑on experience building performant datasets in BigQuery (or similar cloud data warehouses)
  • Proven experience with Python and PySpark for scalable data processing in distributed environments
  • Solid understanding of data modeling, ELT/ETL patterns, and data quality best practices
  • Experience with Google Cloud Platform, particularly BigQuery, Dataflow, Cloud Composer, GCS, or equivalent cloud data services
  • Hands‑on experience building scalable data pipelines (batch and near real‑time) in a cloud‑native environment
  • Proficiency with version control, CI/CD pipelines, and automated testing frameworks
  • Ability to troubleshoot and optimize performance across compute, storage, and processing layers
Job Responsibility
Job Responsibility
  • Build scalable data pipelines: Design and deliver batch and real-time ETL/ELT pipelines across cloud environments to support analytics and reporting
  • Develop SQL and BigQuery solutions: Write and optimize advanced SQL transformations and build performant, cost‑efficient BigQuery data models
  • Develop Python workflows: Implement scalable data processing solutions using Python and PySpark, ensuring maintainable and high‑quality code
  • Design data models and ensure quality: Build robust data models and apply validation practices to maintain accuracy and reliability
  • Build cloud‑native data solutions: Use GCP services such as BigQuery, Dataflow, Cloud Composer, Pub/Sub, and GCS to build and operate modern data platforms
  • Optimize performance and reliability: Troubleshoot complex pipeline issues and continuously improve compute, storage, and processing performance
  • Collaborate using strong engineering practices: Work with engineering, analytics, and business teams while contributing to CI/CD, code reviews, and testing standards
What we offer
What we offer
  • Smooth integration and a supportive mentor
  • Pick your working style: choose from Remote, Hybrid or Office work opportunities
  • Different working hours to suit your needs
  • Sponsored certifications, trainings and top e-learning platforms
  • Private Health Insurance custom-made for you
  • Individual coaching sessions
  • Accredited Coaching School
  • Parties or themed events
  • Fulltime
Read More
Arrow Right

Cloud Engineer - GCP & Databricks

As a Cloud Engineer, you are passionate about experience innovation and eager to...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
valtech.com Logo
Valtech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of hands-on professional experience in cloud data engineering or GCP platform roles
  • Bachelor's degree in Computer Science, Engineering, Information Systems, or equivalent practical experience
  • BigQuery with advanced SQL, partitioning, clustering, and cost optimization
  • Cloud Storage, Cloud Functions, Cloud Run
  • Dataflow (Apache Beam) for batch and streaming pipelines
  • Cloud Composer / Airflow for orchestration
  • Pub/Sub for event-driven architectures
  • Vertex AI exposure for model serving or pipelines
  • IAM, VPC, organization policies, and security governance
  • Terraform for infrastructure as code
Job Responsibility
Job Responsibility
  • Design GCP-native architectures using BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, Vertex AI, and Cloud Run
  • Build and maintain batch and streaming data pipelines using medallion architecture (Bronze, Silver, Gold)
  • Implement infrastructure as code using Terraform
  • Manage deployments through CI/CD pipelines such as Cloud Build
  • Define and enforce GCP landing zone standards including IAM, VPC, Shared VPC, Private Service Connect, and organization policies
  • Build end-to-end Databricks Lakehouse solutions on GCP
  • Design Delta Lake tables with proper governance using Unity Catalog
  • Develop and optimise PySpark and SQL workloads for large-scale transformations
  • Configure Databricks clusters, job scheduling, autoscaling, and cost controls
  • Implement Databricks Workflows and Asset Bundles for orchestration and CI/CD
What we offer
What we offer
  • Flexibility, with remote and hybrid work options (country-dependent)
  • Career advancement, with international mobility and professional development programs
  • Learning and development, with access to cutting-edge tools, training and industry experts
  • Fulltime
Read More
Arrow Right

Senior Data Software Engineer (Python & PySpark) - Vice President

The Senior Data Software Engineer is a senior level position responsible for est...
Location
Location
Singapore , Singapore
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
  • 7+ years of experience in data engineering, with a strong focus on Python and big data technologies
  • Proven expertise in designing and implementing large-scale data processing solutions using PySpark
  • Extensive experience with distributed computing frameworks like Apache Spark
  • Strong understanding of data warehousing concepts, dimensional modeling, and ETL/ELT principles
  • Proficiency in SQL and experience with various relational and NoSQL databases
  • Experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, ADLS, Google Cloud Storage, Redshift, Snowflake, BigQuery, Databricks)
  • Familiarity with workflow orchestration tools (e.g., Apache Airflow, Azure Data Factory, AWS Step Functions)
  • Experience with version control systems (e.g., Git)
  • Excellent problem-solving, analytical, and communication skills.
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
  • Fulltime
Read More
Arrow Right

GCP Data Engineer

We are seeking a GCP Data Engineer who will design, build, and operationalise cl...
Location
Location
India , Pune
Salary
Salary:
Not provided
vodafone.com Logo
Vodafone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experienced in GCP tools including BigQuery, Data Fusion, Dataproc, Cloud Composer, Workflows, and Cloud Scheduler
  • Skilled in programming languages such as Python, Spark, PySpark, or Java
  • Knowledgeable in Apache Airflow, GCP Dataproc clusters, and Dataflow
  • Possess 2–4 years of overall experience, with at least 2–3 years working on cloud platforms such as GCP, AWS, or Azure
  • Preferably certified as a Google Cloud Professional Data Engineer
  • Hold a technical qualification such as B.E./B.Tech, BCA/MCA, or BSc/MSc in Computer Science
Job Responsibility
Job Responsibility
  • Build and operationalise data processing systems based on low‑level design requirements
  • Apply strong working knowledge of the Spark framework and hands‑on experience with Dataproc
  • Use GCP Data Fusion, BigQuery, Airflow, and related tools to support optimised and scalable development approaches
  • Apply cloud‑based data pipeline patterns and propose innovative solutions to navigate platform constraints
  • Design, test, and maintain data pipelines aligned with data modelling, data warehousing, and industry‑standard data manipulation techniques
  • Design, develop, and maintain programmes written in Python, Spark, Scala, Java, or related technologies
  • Contribute to organisational improvements through process adoption, resource optimisation, and the use of tools that enhance productivity and quality
  • Recommend approaches to improve data reliability, operational efficiency, and overall solution quality
Read More
Arrow Right

GCP Data Engineer

We are seeking a skilled GCP Data Engineer to design, build, and operationalise ...
Location
Location
India , Pune
Salary
Salary:
Not provided
vodafone.com Logo
Vodafone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experienced in GCP tools including BigQuery, Data Fusion, Dataproc, Cloud Composer, Workflows, and Cloud Scheduler
  • Skilled in programming languages such as Python, Spark, PySpark, or Java
  • Knowledgeable in Apache Airflow, Dataproc clusters, and Dataflow
  • Possess 2–4 years of overall experience, including at least 2–3 years in cloud platforms (GCP/AWS/Azure)
  • Preferably certified as a Google Cloud Professional Data Engineer
  • Hold a relevant degree such as B.E./B.Tech, BCA/MCA, or BSc/MSc in Computer Science
Job Responsibility
Job Responsibility
  • Build and operationalise data processing systems based on detailed design specifications
  • Apply strong Spark knowledge and hands-on experience with Dataproc
  • familiarity with Dataflow is beneficial
  • Utilise GCP Data Fusion, BigQuery, Airflow, and related tools to deliver optimised data solutions
  • Apply cloud-based data pipeline patterns and contribute creative approaches to navigate platform limitations
  • Design, test, and maintain data pipelines following data modelling, warehousing, and manipulation standards
  • Design and develop programmes using languages such as Python, Spark, PySpark, Scala, or Java
  • Contribute to process adoption, resource and tool optimisation, and continuous quality uplift
  • Recommend improvements to enhance data reliability, operational efficiencies, and solution quality
Read More
Arrow Right

Data Engineer - VOIS

We are seeking an experienced Data Engineering Specialist with strong hands-on e...
Location
Location
India , Pune
Salary
Salary:
Not provided
vodafone.com Logo
Vodafone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong expertise in Databricks on GCP including Delta Lake, notebooks/jobs, Unity Catalog, and cluster policies
  • Experienced in Cloud Data Fusion design, including pipeline management, error handling, and orchestration
  • Skilled in Dataproc Spark with experience building PySpark jobs, configuring ephemeral clusters, and handling initialisation actions
  • Proficient in Python for data engineering including packaging, unit testing, type hints, and linting
  • Strong SQL skills, specifically with BigQuery including performance tuning, partitioning, and clustering
  • Familiar with GCP services such as Cloud Storage, Pub/Sub, and Cloud Composer/Airflow
  • Holds a qualification such as B.E., B.Tech, BCA, MCA, BSc, or MSc in Computer Science or a related field.
Job Responsibility
Job Responsibility
  • Design and build data pipelines on GCP using Databricks (Delta Lake and Unity Catalog) for orchestration, Dataproc for Spark execution, supporting both ETL/ELT and feature engineering workloads
  • Engineer declarative, modular, and reusable pipelines in Python, following configuration-as-code principles and CI/CD practices including Git-based promotion, testing, and deployment
  • Implement and maintain data quality and observability practices using validation frameworks, logging, metrics, and alerts
  • Optimise pipeline performance, reliability, and cost through techniques such as cluster sizing, auto-termination, Z-ordering, caching, and partitioning strategies
  • Apply robust error handling, parameterisation, and triggers within Cloud Data Fusion pipelines
  • Ensure operational excellence by maintaining monitoring, performance tuning, and continuous improvements across data products and workloads.
What we offer
What we offer
  • The opportunity to build and scale data solutions using leading GCP and Databricks technologies
  • Exposure to enterprise-level CI/CD, observability, and configuration-as-code practices
  • A collaborative environment where innovation, continuous learning, and technical excellence are encouraged
  • The chance to contribute to high-impact global data platforms.
  • Fulltime
Read More
Arrow Right

Backend & Data Engineering Specialist

We are seeking a Backend & Data Engineering Specialist to design, build and oper...
Location
Location
India , Pune
Salary
Salary:
Not provided
vodafone.com Logo
Vodafone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Python
  • Flask
  • FastAPI
  • BigQuery
  • SQL
  • Docker
  • Git
  • machine learning
  • time-series forecasting
  • regression models
Job Responsibility
Job Responsibility
  • Design, develop and maintain backend services using Python frameworks such as Flask and FastAPI
  • Build, optimise and support data pipelines and analytics workflows using BigQuery and SQL
  • Develop containerised applications using Docker and manage source code through Git
  • Implement and maintain machine learning solutions, including time-series forecasting and regression models
  • Support ETL and data transformation workflows, with exposure to PySpark-based processing
  • Deploy, monitor and manage applications on Google Cloud Platform (GCP)
  • Ensure applications comply with security, governance and data protection standards
  • Actively participate in Agile ceremonies including sprint planning, daily stand-ups, sprint reviews and retrospectives
  • Collaborate closely with Product Owners, DevOps, Data and Security teams within an Agile/Scrum setup
  • Take ownership of deliverables and contribute to continuous improvement in Agile execution
What we offer
What we offer
  • Opportunity to work on cloud-native backend and data platforms using Google Cloud technologies
  • Exposure to analytics-driven and machine learning–enabled solutions
  • Collaboration with cross-functional teams in a modern Agile delivery environment
  • A role that balances engineering ownership with continuous improvement and learning
Read More
Arrow Right

Data Engineer

We are proactively seeking experienced Data Engineers for upcoming contract and ...
Location
Location
United States , Charlotte
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of Data Engineering or related experience
  • Strong problem-solving and analytical skills
  • Excellent communication and collaboration abilities
  • Ability to work independently and manage multiple priorities in fast-paced environments
  • Strong SQL development and data modeling experience
  • Experience with cloud data platforms such as Azure Data Factory, Databricks, Snowflake, Synapse, Redshift, or BigQuery
  • Experience developing or supporting BI/reporting solutions using Power BI, Tableau, or similar tools
  • Programming experience with Python, PySpark, Scala, or related technologies
  • Experience with data warehousing, dimensional modeling, and enterprise analytics environments
  • Familiarity with Agile development methodologies and DevOps practices
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ETL/ELT pipelines and data integration processes
  • Develop and optimize cloud-based data solutions within Azure, AWS, or GCP environments
  • Support enterprise BI and analytics initiatives by delivering clean, reliable, analytics-ready datasets
  • Work with large datasets across data warehouses, data lakes, and operational systems
  • Collaborate with business stakeholders, analysts, and development teams to gather requirements and deliver data solutions
  • Optimize SQL queries, database performance, and data processing workflows
  • Support data governance, quality, validation, and automation initiatives
  • Participate in data architecture, modernization, and migration projects
What we offer
What we offer
  • Competitive compensation and benefits
  • free online training
  • medical, vision, dental, and life and disability insurance
  • enrollment in company 401(k) plan
Read More
Arrow Right