This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
The Big Data Engineer is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications systems analysis and programming activities.
Job Responsibility:
Partner with multiple management teams to ensure appropriate integration of functions
Resolve variety of high impact problems/projects through in-depth evaluation
Provide expertise in area and advanced knowledge of applications programming
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts
Appropriately assess risk when business decisions are made
Requirements:
Data Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines
Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
Proficiency in programming languages like Python, or Scala
Strong expertise in data processing frameworks such as Apache Spark, Hadoop
Expertise in Data Lakehouse technologies (Apache Iceberg, Apache Hudi, Trino)
Experience with cloud data platforms like AWS (Glue, EMR, Redshift), Azure (Synapse), or GCP (BigQuery)
Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)
Experience with data orchestration tools like Apache Airflow or Prefect
Familiarity with containerization (Docker, Kubernetes)
Bachelor's degree/University degree or equivalent experience
Nice to have:
Distributed caching solutions (Hazelcast or Redis)
Prior experience with building distributed, multi-tier applications
Experience with building apps which are highly performant and scalable
Welcome to CrawlJobs.com – Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
We use cookies to enhance your experience, analyze traffic, and serve personalized content. By clicking “Accept”, you agree to the use of cookies.