Databricks Engineer Job at 84 Recruitment Limited (Auckland)

Databricks Engineer

We’re working with multiple organisations across Auckland and Wellington seeking...

Location

New Zealand , Wellington

Salary:

Not provided

84 Recruitment Limited

Expiration Date

Until further notice

Requirements

Strong hands-on experience with Databricks
Proven background in ETL/ELT pipeline development
Experience with cloud platforms (AWS, Azure, or GCP)
Solid understanding of modern data architecture and best practices

Job Responsibility

Design, build, and maintain ETL/ELT pipelines on Databricks
Implement and optimise Medallion Architecture data layers
Ensure data quality, governance, and performance across pipelines
Collaborate with stakeholders to deliver scalable data solutions

Databricks Engineer

Databricks Engineer (Contract) – Auckland & Wellington & Christchurch. We’re wor...

Location

New Zealand , Christchurch

Salary:

Not provided

84 Recruitment Limited

Expiration Date

Until further notice

Requirements

Strong hands-on experience with Databricks or similar cloud platform
Proven background in ETL/ELT pipeline development
Experience with cloud platforms (AWS, Azure, or GCP)
Solid understanding of modern data architecture and best practices

Job Responsibility

Design, build, and maintain ETL/ELT pipelines on Databricks
Implement and optimise Medallion Architecture data layers
Ensure data quality, governance, and performance across pipelines
Collaborate with stakeholders to deliver scalable data solutions

Databricks Engineer

Job Title: Databricks Engineer; Location – Remote; Fulltime; Skill: Databricks E...

Location

United States

Salary:

140000.00 USD / Year

Realign

Expiration Date

Until further notice

Requirements

Minimum 6-8 years of relevant experience
Develop and maintain CI/CD pipelines for Azure Databricks deployments (Azure DevOps/YAML and related tools)
Automate deployment and configuration of Databricks clusters, jobs, libraries, notebooks, and environment promotions
Implement and manage the Databricks environment for performance, cost efficiency, and scalability
optimize cluster sizing and autoscaling
Collaborate with Data Engineers/Scientists/Software Engineers to design, deploy, and scale data pipelines and models on Databricks
Monitor and troubleshoot clusters, pipelines, jobs, and associated workflows
integrate Azure Monitor/Log Analytics for visibility and metrics
Implement Infrastructure as Code (IaC) using Terraform, ARM templates, or Bicep to manage Azure resources and Databricks artifacts
Design and maintain backup, recovery, and DR strategies for Databricks environments

Job Responsibility

Workspace & environment engineering: Standardize Dev/UAT/Prod workspaces (network/Private Link, VNets, secure egress), service principals, secret scopes, and Key Vault integrations
Unity Catalog & governance: Configure catalogs/schemas, RBAC, lineage, and data access patterns aligned to our guardrails
CI/CD for Databricks: Implement YAML based Azure DevOps pipelines to automate notebook/job deployments, dependencies, environment promotions, and approvals/compliance checks
IaC for Databricks & Azure: Author reusable Bicep/Terraform modules for workspaces, clusters/pools, UC objects, and supporting Azure resources
Observability & reliability: Establish monitoring/alerting for jobs, clusters, SLAs, autoscaling, and cost controls
and automation for disaster recovery scenarios
Documentation & handover: Patterns, pipeline templates, IaC modules, and operational runbooks for BAU, plus KT during the first two releases

Fulltime

Databricks Engineer

Our client is revolutionizing the field of cell therapy manufacturing by develop...

Location

Salary:

Not provided

Coherent Solutions

Expiration Date

Until further notice

Requirements

5+ years of experience in Data Engineering with strong technical expertise
Proven hands-on experience with the Databricks Data Platform and Delta Lake
Experience building and managing Databricks Lakehouse solutions
Knowledge of Delta Live Tables or similar frameworks for real-time data ingestion is a strong plus
Ability to define processes from scratch and establish development workflows in a new or evolving team
Familiarity with data testing best practices and collaboration with QA teams to ensure data quality
Strong problem-solving mindset, initiative, and readiness to work in a dynamic, evolving environment
Ability to work with a time shift, ensuring overlap with the client until approximately 10:30 AM Pacific Time for meetings and collaboration
English level: Upper-Intermediate (written and spoken)

Job Responsibility

Design, build, and maintain data pipelines using Databricks and Delta Live Tables for real-time and batch data processing
Collaborate with cross-functional teams to ensure smooth data flow from diverse log-based sources
Participate in both individual and collaborative work, ensuring scalability, reliability, and performance of data solutions
Define and implement best practices for data development and deployment processes on the Databricks platform
Proactively address technical challenges in a project environment, proposing and implementing effective solutions

What we offer

Technical and non-technical training for professional and personal growth
Internal conferences and meetups to learn from industry experts
Support and mentorship from an experienced employee to help you professional grow and development
Internal startup incubator
Health insurance
English courses
Sports activities to promote a healthy lifestyle
Flexible work options, including remote and hybrid opportunities
Referral program for bringing in new talent
Work anniversary program and additional vacation days

Databricks Engineer

We are seeking a Databricks Engineer to design, build, and operate a Data & AI p...

Location

United States , Leesburg

Salary:

Not provided

WINTrio

Expiration Date

Until further notice

Requirements

Hands-on experience with Databricks, Delta Lake, and Apache Spark
Deep understanding of ELT pipeline development, orchestration, and monitoring in cloud-native environments
Experience implementing Medallion Architecture (Bronze/Silver/Gold) and working with data versioning and schema enforcement in enterprise grade environments
Strong proficiency in SQL, Python, or Scala for data transformations and workflow logic
Proven experience integrating enterprise platforms (e.g., PeopleSoft, Salesforce, D2L) into centralized data platforms
Familiarity with data governance, lineage tracking, and metadata management tools

Job Responsibility

Data & AI Platform Engineering (Databricks-Centric): Design, implement, and optimize end-to-end data pipelines on Databricks, following the Medallion Architecture principles
Build robust and scalable ETL/ELT pipelines using Apache Spark and Delta Lake to transform raw (bronze) data into trusted curated (silver) and analytics-ready (gold) data layers
Operationalize Databricks Workflows for orchestration, dependency management, and pipeline automation
Apply schema evolution and data versioning to support agile data development
Platform Integration & Data Ingestion: Connect and ingest data from enterprise systems such as PeopleSoft, D2L, and Salesforce using APIs, JDBC, or other integration frameworks
Implement connectors and ingestion frameworks that accommodate structured, semi-structured, and unstructured data
Design standardized data ingestion processes with automated error handling, retries, and alerting
Data Quality, Monitoring, and Governance: Develop data quality checks, validation rules, and anomaly detection mechanisms to ensure data integrity across all layers
Integrate monitoring and observability tools (e.g., Databricks metrics, Grafana) to track ETL performance, latency, and failures
Implement Unity Catalog or equivalent tools for centralized metadata management, data lineage, and governance policy enforcement

Staff Engineer I – Azure Databricks Engineer

Location

United States , Irving

Salary:

Not provided

Software Resources

Expiration Date

Until further notice

Requirements

5+ years of related experience in data analytics administration and development
2+ years of Databricks related experience
Bachelor's degree in related field required
Hands-on experience in Azure Databricks (Workspace management, Clusters, Jobs, Unity Catalog, Delta Lake, User access management, Rest APIs and SDKs)
Understanding of Azure infrastructure and data services, including Azure Data Lake, Azure Data Factory, Azure SQL, Azure Synapse Analytics, Azure Key Vault, Azure Monitor, networking
Experience with CI/CD pipelines (Azure DevOps preferred)
Strong programming skills in SQL, Python, and/or PySpark
Proven experience in leading cross-functional teams and managing multiple projects simultaneously
Intermediate to advanced ability to see the big picture and align projects with organizational goals
Capable of leading and motivating cross-functional teams

Job Responsibility

Delivery and operations of technologies and platforms required to model, transform, analyze, report, visualize data
Building, optimizing, streamlining and automating the Azure Databricks platform to enable analytical workloads like Machine learning (ML), Model Development, Data Insights, Data Science etc
Partner with ML engineers, data scientists, data analysts, and enterprise architects to enforce best practices, train and enable users using the Azure Databricks platform
Take assignments that can be worked on individually without supervision and manage work effort from concept to completion
Build, optimize and maintain the Azure Databricks platform, ensuring scalability, security, governance and performance
Implement and manage Azure Databricks workspaces, clusters, jobs, access management
Implement and manage policies, monitoring and observability for the Azure Databricks platform
Ensure compliance with IT policies, procedures, and industry standards, including reviewing and refining IT control enhancements
Work closely with business and analytics teams to ensure reliable and governed data access for data needed for analytics
Troubleshoot platform issues, optimize performance, and ensure uptime for critical Databricks services

What we offer

Competitive salaries
An ownership stake in the company
Medical and dental insurance
Time off
A great 401k matching program
Tuition assistance program
An employee volunteer program
A wellness program

Staff Engineer I – Azure Databricks Engineer

Software Resources has an immediate, direct hire job opportunity for a Staff Eng...

Location

United States , Irving, Dallas

Salary:

Not provided

Software Resources

Expiration Date

Until further notice

Requirements

5+ years of related experience in data analytics administration and development
2+ years of Databricks related experience
Bachelor’s degree in related field required
Hands-on experience in Azure Databricks (Workspace management, Clusters, Jobs, Unity Catalog, Delta Lake, User access management, Rest APIs and SDKs)
Understanding of Azure infrastructure and data services, including Azure Data Lake, Azure Data Factory, Azure SQL, Azure Synapse Analytics, Azure Key Vault, Azure Monitor, networking
Experience with CI/CD pipelines (Azure DevOps preferred)
Strong programming skills in SQL, Python, and/or PySpark
Proven experience in leading cross-functional teams and managing multiple projects simultaneously
Intermediate to advanced ability to see the big picture and align projects with organizational goals. Capable of leading and motivating cross-functional teams. Expertise in resolving conflicts and addressing challenges as well as skilled at identifying and mitigating risks at the project level
Good communication skills, excellent teamwork experience

Job Responsibility

Responsible for delivery and operations of technologies and platforms required to model, transform, analyze, report, visualize data
Work in a mid-level role with proficiency needed in building, optimizing, streamlining and automating the Azure Databricks platform to enable analytical workloads like Machine learning (ML), Model Development, Data Insights, Data Science etc
Partner with ML engineers, data scientists, data analysts, and enterprise architects to enforce best practices, train and enable users using the Azure Databricks platform
Take assignments that can be worked on individually without supervision and manage work effort from concept to completion
Build, optimize and maintain the Azure Databricks platform, ensuring scalability, security, governance and performance
Implement and manage Azure Databricks workspaces, clusters, jobs, access management
Implement and manage policies, monitoring and observability for the Azure Databricks platform
Ensure compliance with IT policies, procedures, and industry standards, including reviewing and refining IT control enhancements
Work closely with business and analytics teams to ensure reliable and governed data access for data needed for analytics
Troubleshoot platform issues, optimize performance, and ensure uptime for critical Databricks services

What we offer

Competitive salaries
An ownership stake in the company
Medical and dental insurance
Time off
A great 401k matching program
Tuition assistance program
An employee volunteer program
A wellness program
Opportunity to bolster your business knowledge, learning the ins and outs of how successful companies operate and manage their finances, giving you invaluable hands-on experience to help grow your career

Fulltime

New

Senior / Staff Full-Stack Data Engineer – Databricks

At Codvo, we are committed to building scalable, future-ready data platforms tha...

Location

India , Pune

Salary:

Not provided

Codvo AI

Expiration Date

Until further notice

Requirements

Strong hands-on experience with Databricks, Apache Spark, and Delta Lake
Proven experience building and operating production-grade data pipelines
Experience operationalizing machine learning models and inference pipelines
Strong understanding of data reliability, observability, and monitoring practices
Experience with CI/CD, DevOps, and MLOps workflows
Experience working with cloud platforms (AWS or Azure)
Familiarity with Unity Catalog and enterprise data governance concepts
Experience with spec-driven development and coding agents

Job Responsibility

Design, build, and maintain ETL/ELT pipelines on Databricks using Spark, Delta Lake, and Databricks Workflows
Build and operate batch and real-time data pipelines for ingestion, transformation, and orchestration
Operationalize machine learning inference pipelines authored by data scientists (batch and real-time)
Ensure consistency between model training and inference environments
Implement data quality checks, validation rules, monitoring, alerting, and automated recovery
Collaborate with data scientists to productionize models and optimize inference performance and cost
Implement CI/CD, DevOps, and MLOps best practices for data pipelines and ML workflows
Optimize compute, storage, and job configurations for performance and cost efficiency
Implement and manage enterprise data governance using Unity Catalog (schemas, lineage, ownership, documentation)
Work with Databricks infrastructure and platform configurations

Fulltime

Select Country

Databricks Engineer

Job Description

Job Responsibility

Requirements

Nice to have

Looking for more opportunities?

Databricks Engineer

Databricks Engineer

Databricks Engineer

Databricks Engineer

Databricks Engineer

Databricks Engineer

Staff Engineer I – Azure Databricks Engineer

Staff Engineer I – Azure Databricks Engineer

Senior / Staff Full-Stack Data Engineer – Databricks

Our AI answers in your language