This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
To develop, implement, and optimize complex Data Warehouse (DWH) and Data Lakehouse solutions using the Databricks platform to ensure a scalable, high-performance, and governed data foundation for analytics, reporting, and Machine Learning.
Job Responsibility:
Design and implement robust, scalable, and high-performance ETL/ELT data pipelines using PySpark/Scala and Databricks SQL on the Databricks platform
expertise in implementing and optimizing the Medallion architecture (Bronze, Silver, Gold) using Delta Lake
design and implement real-time/near-real-time data processing solutions using Spark Structured Streaming and Delta Live Tables (DLT)
implement Unity Catalog for centralized data governance, fine-grained security (row/column-level security), and data lineage
develop and manage complex workflows using Databricks Workflows (Jobs) or external tools (Azure Data Factory, Airflow) to automate pipelines
integrate Databricks pipelines into CI/CD processes using tools like Git, Databricks Repos, and Bundles
work closely with Data Scientists, Analysts, and Architects to deliver optimal technical solutions
provide technical guidance and mentorship to junior developers.
Requirements:
Proven, expert-level experience with the entire Databricks ecosystem (Workspace, Cluster Management, Notebooks, Databricks SQL)
in-depth knowledge of Spark architecture (RDD, DataFrames, Spark SQL) and advanced optimization techniques
expertise in implementing and managing Delta Lake (ACID properties, Time Travel, Merge, Optimize, Vacuum)
advanced/expert-level proficiency in Python (with PySpark) and/or Scala (with Spark)
advanced/expert-level skills in SQL and Data Modeling (Dimensional, 3NF, Data Vault)
solid experience with a major Cloud platform (AWS, Azure, or GCP), especially with storage services (S3, ADLS Gen2, GCS) and networking
bachelor’s degree in Computer Science, Engineering, Mathematics, or a relevant technical field
minimum of 5+ years of experience in Data Engineering, with at least 3+ years of experience working with Databricks and Spark at scale.
Nice to have:
Hands-on experience with implementing and managing Unity Catalog
experience with Delta Live Tables (DLT) and Databricks Workflows
understanding of basic MLOps concepts and experience with MLflow to facilitate integration with Data Science teams
experience with Terraform or equivalent tools for Infrastructure as Code (IaC)
Databricks certifications (e.g., Databricks Certified Data Engineer Professional).
What we offer:
Full access to foreign language learning platform
personalized access to tech learning platforms
tailored workshops and trainings to sustain your growth
medical insurance
meal tickets
monthly budget to allocate on flexible benefit platform
Welcome to CrawlJobs.com – Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
We use cookies to enhance your experience, analyze traffic, and serve personalized content. By clicking “Accept”, you agree to the use of cookies.