CrawlJobs Logo
Robert Half Logo Robert Half · IT - Software Development

Senior Databricks Engineer

United States, Cincinnati · Job Posted June 09, 2026
Apply Position
Job Link Share

Job Description

We are looking for an experienced Senior Databricks Engineer to support enterprise data initiatives. This Long-term Contract position will focus on building and enhancing scalable data solutions, including extracting data from ServiceNow and structuring it for downstream ARDoc consumption within a broader enterprise architecture program. The role will partner with engineering, analytics, and platform teams to deliver reliable, governed, and high-performing data pipelines on Databricks.

Job Responsibility

  • Develop and maintain robust data pipelines in Databricks using Apache Spark to support enterprise reporting and analytical use cases
  • Extract and transform data from ServiceNow, then create and manage structured tables that supply ARDoc as part of a larger architecture initiative
  • Design layered data models using Delta Lake practices to improve scalability, data usability, and long-term maintainability
  • Configure and manage Databricks jobs, workflows, and compute resources to improve processing efficiency and platform stability
  • Investigate performance issues across Spark workloads and implement tuning strategies that improve speed, reliability, and cost control
  • Establish data governance practices through cataloging, lineage visibility, and secure access controls within the Databricks environment
  • Work closely with data scientists, analysts, and platform teams to deploy production-ready data assets and support advanced analytics needs
  • Provide technical guidance to other engineers by promoting engineering standards, reviewing designs, and sharing platform best practices

Requirements

  • At least 5 years of experience in data engineering working with large-scale or distributed data processing platforms
  • Minimum 3 years of hands-on experience building solutions in Databricks within production environments
  • Strong coding ability in Python, especially PySpark, with additional SQL proficiency for data transformation and analysis
  • Solid knowledge of Apache Spark concepts, including execution behavior, performance tuning, and resource management
  • Practical experience with Delta Lake features such as schema evolution, versioned data management, and reliable transactional processing
  • Familiarity with cloud-based data ecosystems and orchestration tools used to schedule, monitor, and manage data workloads
  • Experience using version control and CI/CD practices to support repeatable and controlled data engineering deployments
  • ServiceNow knowledge is highly valued, especially for roles involving data extraction and integration into downstream platforms

Nice to have

ServiceNow knowledge is highly valued, especially for roles involving data extraction and integration into downstream platforms

What we offer

  • medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
Robert Half - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Databricks Engineer

8 matching positions

Senior Data Engineer (DATABRICKS)

As a Senior Databricks Administrator, you will be responsible for the setup, con...
Location
Location
Ireland , Dublin 18
Salary
Salary:
Not provided
mastercard.com Logo
Mastercard
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of experience in Databricks administration on AWS or multi-cloud environments
  • Deep understanding of Databricks workspace architecture, Unity Catalog, and cluster configuration best practices
  • Strong experience in managing IAM policies, SCIM integration, and access provisioning workflows
  • Hands-on experience with monitoring, cost optimization, and governance of large-scale Databricks deployments
  • Hands-on experience with infrastructure-as-code (Terraform) and CI/CD pipelines
  • Experience with ETL orchestration and collaboration with engineering teams (Databricks Jobs, Workflows, Airflow)
Job Responsibility
Job Responsibility
  • Databricks Platform Administration (AWS) Provision, configure, and manage Databricks Workspaces across multiple environments (Dev, UAT, Prod)
  • Define and enforce cluster policies, compute tagging, and usage controls to optimize cost and resource utilization
  • Administer Unity Catalog, including creation and management of catalogs, schemas, access control, and data lineage
  • Manage SCIM integrations for user/group provisioning with IdPs like Okta or Azure/AWS SSO
  • Configure and maintain CI/CD pipelines for notebook deployment and workflow promotion using GitHub, GitLab, or similar tools
  • Security, Governance & Compliance Implement and enforce role-based access controls (RBAC) and fine-grained data permissions using Unity Catalog and Lake Formation
  • Ensure auditability and lineage tracking for compliance with Open Banking, GDPR, and PSD2 regulations
  • Configure and manage token scopes, secrets management, and credential passthrough for secure access to underlying data
  • Work with InfoSec and compliance teams to ensure platform security aligns with regulatory frameworks
  • Monitoring, Support & Troubleshooting Monitor workspace performance, cluster health, job execution, and user activity using Databricks native tools, CloudWatch, and third-party integrations
  • Fulltime
Read More
Arrow Right

Senior Databricks Data Engineer

To develop, implement, and optimize complex Data Warehouse (DWH) and Data Lakeho...
Location
Location
Romania , Bucharest
Salary
Salary:
Not provided
https://www.inetum.com Logo
Inetum
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven, expert-level experience with the entire Databricks ecosystem (Workspace, Cluster Management, Notebooks, Databricks SQL)
  • in-depth knowledge of Spark architecture (RDD, DataFrames, Spark SQL) and advanced optimization techniques
  • expertise in implementing and managing Delta Lake (ACID properties, Time Travel, Merge, Optimize, Vacuum)
  • advanced/expert-level proficiency in Python (with PySpark) and/or Scala (with Spark)
  • advanced/expert-level skills in SQL and Data Modeling (Dimensional, 3NF, Data Vault)
  • solid experience with a major Cloud platform (AWS, Azure, or GCP), especially with storage services (S3, ADLS Gen2, GCS) and networking
  • bachelor’s degree in Computer Science, Engineering, Mathematics, or a relevant technical field
  • minimum of 5+ years of experience in Data Engineering, with at least 3+ years of experience working with Databricks and Spark at scale.
Job Responsibility
Job Responsibility
  • Design and implement robust, scalable, and high-performance ETL/ELT data pipelines using PySpark/Scala and Databricks SQL on the Databricks platform
  • expertise in implementing and optimizing the Medallion architecture (Bronze, Silver, Gold) using Delta Lake
  • design and implement real-time/near-real-time data processing solutions using Spark Structured Streaming and Delta Live Tables (DLT)
  • implement Unity Catalog for centralized data governance, fine-grained security (row/column-level security), and data lineage
  • develop and manage complex workflows using Databricks Workflows (Jobs) or external tools (Azure Data Factory, Airflow) to automate pipelines
  • integrate Databricks pipelines into CI/CD processes using tools like Git, Databricks Repos, and Bundles
  • work closely with Data Scientists, Analysts, and Architects to deliver optimal technical solutions
  • provide technical guidance and mentorship to junior developers.
What we offer
What we offer
  • Full access to foreign language learning platform
  • personalized access to tech learning platforms
  • tailored workshops and trainings to sustain your growth
  • medical insurance
  • meal tickets
  • monthly budget to allocate on flexible benefit platform
  • access to 7 Card services
  • wellbeing activities and gatherings.
  • Fulltime
Read More
Arrow Right

Senior Databricks Data Engineer

To develop, implement, and optimize complex Data Warehouse (DWH) and Data Lakeho...
Location
Location
Romania , Bucharest
Salary
Salary:
Not provided
https://www.inetum.com Logo
Inetum
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven, expert-level experience with the entire Databricks ecosystem (Workspace, Cluster Management, Notebooks, Databricks SQL)
  • In-depth knowledge of Spark architecture (RDD, DataFrames, Spark SQL) and advanced optimization techniques
  • Expertise in implementing and managing Delta Lake (ACID properties, Time Travel, Merge, Optimize, Vacuum)
  • Advanced/expert-level proficiency in Python (with PySpark) and/or Scala (with Spark)
  • Advanced/expert-level skills in SQL and Data Modeling (Dimensional, 3NF, Data Vault)
  • Solid experience with a major Cloud platform (AWS, Azure, or GCP), especially with storage services (S3, ADLS Gen2, GCS) and networking.
Job Responsibility
Job Responsibility
  • Design and implement robust, scalable, and high-performance ETL/ELT data pipelines using PySpark/Scala and Databricks SQL on the Databricks platform
  • Expertise in implementing and optimizing the Medallion architecture (Bronze, Silver, Gold) using Delta Lake to ensure data quality, consistency, and historical tracking
  • Efficient implementation of the Lakehouse architecture on Databricks, combining best practices from DWH and Data Lake
  • Optimize Databricks clusters, Spark operations, and Delta tables to reduce latency and computational costs
  • Design and implement real-time/near-real-time data processing solutions using Spark Structured Streaming and Delta Live Tables
  • Implement and manage Unity Catalog for centralized data governance, data security and data lineage
  • Define and implement data quality standards and rules to maintain data integrity
  • Develop and manage complex workflows using Databricks Workflows or external tools to automate pipelines
  • Integrate Databricks pipelines into CI/CD processes
  • Work closely with Data Scientists, Analysts, and Architects to understand business requirements and deliver optimal technical solutions
What we offer
What we offer
  • Full access to foreign language learning platform
  • Personalized access to tech learning platforms
  • Tailored workshops and trainings to sustain your growth
  • Medical insurance
  • Meal tickets
  • Monthly budget to allocate on flexible benefit platform
  • Access to 7 Card services
  • Wellbeing activities and gatherings.
  • Fulltime
Read More
Arrow Right

Senior Databricks Data Engineer

At LeverX, we have had the privilege of delivering over 1,500 projects for vario...
Location
Location
Georgia
Salary
Salary:
Not provided
leverx.com Logo
LeverX
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years in Data Engineering
  • Strong experience with Databricks: Spark (PySpark + SQL), Delta Lake internals, Jobs, Workflows, SQL Warehouses, Unity Catalog
  • Excellent SQL skills (analytical queries, performance tuning)
  • Strong Python for data engineering
  • Experience with dbt (models, tests, exposures, CI/CD)
  • Experience with orchestration tools (Airflow, Dagster)
  • Experience designing scalable lakehouse architectures
  • Understanding of cost optimization in Databricks (clusters, warehouses, storage layout)
  • English B2+
Job Responsibility
Job Responsibility
  • Design and build scalable data pipelines on Databricks using PySpark, Databricks SQL, and Delta Lake
  • Develop, optimize, and troubleshoot Spark workloads for performance and reliability
  • Manage Delta Lake tables, including Z-Ordering, compaction, schema evolution, retention, and file layout optimization
  • Configure and administer Databricks Jobs, clusters, SQL Warehouses, and workflows
  • Own data governance using Unity Catalog, including permissions, lineage, and catalog/schema management
  • Implement data transformations and medallion architecture (bronze–silver–gold) using dbt
  • Build CI/CD pipelines and ensure data quality, observability, SLAs, auditing, and monitoring
  • Collaborate with and mentor engineers, partner with analytics and ML teams, and drive continuous platform improvements
What we offer
What we offer
  • Projects in different domains: healthcare, manufacturing, e-commerce, fintech, etc.
  • Projects for every taste: Startup products, enterprise solutions, research & development initiatives, and projects at the crossroads of SAP and the latest web technologies
  • Global clients based in Europe and the US, including Fortune 500 companies
  • Employment security: We hire for our team, not just a specific project. If your project ends, we will find you a new one
  • Healthy work atmosphere: On average, our employees stay with the company for 4+ years
  • Market-based compensation and regular performance reviews
  • Internal expert communities and courses
  • Perks to support your growth and well-being
Read More
Arrow Right

Principal Engineer I – Senior Azure Databricks Administrator

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Phoenix
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of related experience in data analytics administration and development
  • 4+ years of Databricks related experience
  • Bachelor’s degree in related field required
  • Advanced proven experience in Azure Databricks (Workspace management, Clusters, Jobs, Unity Catalog, Delta Lake, User access management, Rest APIs and SDKs)
  • Knowledge of MLFlow & MLOps
  • Deep understanding of Azure infrastructure and data services, including Azure Data Lake, Azure Data Factory, Azure SQL, Azure Synapse Analytics, Azure Key Vault, Azure Monitor, networking
  • Experience with CI/CD pipelines (Azure DevOps preferred)
  • Strong programming skills in SQL, Python, and/or PySpark
  • Advanced proven experience in leading cross-functional teams and managing multiple projects simultaneously
  • Advanced ability to see the big picture and align projects with organizational goals
Job Responsibility
Job Responsibility
  • Responsible for delivery and operations of technologies and platforms required to model, transform, analyze, report, visualize data
  • Provide SME expertise in designing, building, optimizing, streamlining and automating the Azure Databricks platform
  • Partner with ML engineers, data scientists, data analysts, and enterprise architects to provide frameworks, set standards, enforce best practices, train and enable users
  • Develop technical skills of one or more junior team-members
  • Take assignments that can be worked on individually without supervision, and manage work effort from concept to completion
  • Design, build, optimize, automate and maintain the Azure Databricks platform, ensuring scalability, security, governance and performance
  • Design, implement and manage Azure Databricks workspaces, clusters, jobs, access management
  • Design, implement and manage policies, monitoring and observability
  • Implement data analytics principles aimed at business enablement, reliability practices and sound recovery procedures
  • Ensure compliance with IT policies, procedures, and industry standards
What we offer
What we offer
  • Competitive salaries
  • An ownership stake in the company
  • Medical and dental insurance
  • Time off
  • A great 401k matching program
  • Tuition assistance program
  • An employee volunteer program
  • A wellness program
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Azure Databricks

Do you love working with data, artificial intelligence, and large distributed sy...
Location
Location
United States , Multiple Locations
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, OR Java, JavaScript, or Python
  • OR equivalent experience.
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Designing and building distributed systems that interact between different cloud technology services and infrastructures.
  • Conducting research and maintain an awareness of industry knowledge and identifies ways the information can be applied.
  • Mentoring other teammates and cross teams, providing technical direction for the engineering organization.
  • Writing high quality documentations including design documents, troubleshooting guides and software specs.
  • Participating in onboarding, code/design reviews, and regular meetings with the engineering teams that develop and manage those products.
  • Designs, develops, and maintains telemetry pipelines and monitoring tools that detail operations metrics.
  • Conduct deployment of the software products through Azure DevOps and CI/CD (continuous integration/continuous delivery) process.
  • Responds to incidents during regular on-call rotations. Improves and optimizes supportability of the product through innovative ideas.
  • Monitor performance and reliability of services and troubleshoot problem areas as needed.
  • Fulltime
Read More
Arrow Right

Senior Data Engineer with Databricks

At KMD, we build digital solutions used by millions of people around the world. ...
Location
Location
Poland , Warsaw
Salary
Salary:
160.00 - 180.00 PLN / Hour
kmd.net Logo
KMD Poland
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of commercial experience in implementing, developing, or maintaining data load systems (ETL/ELT)
  • Strong programming skills in Python, with a deep understanding of data-related challenges
  • Hands-on experience with Apache Spark and Databricks
  • Familiar with MSSQL databases
  • Experience working with distributed systems on a cloud platform
  • Worked on large-scale systems and understands performance optimization
  • Comfortable with Git and CI/CD practices, and can design or implement deployment processes for data pipelines
  • Proactive, solution-oriented, and has a strong can-do attitude
  • Communicates fluently in English and Polish, both written and spoken
  • Team player with excellent collaboration and communication skills
Job Responsibility
Job Responsibility
  • Develop and maintain data delivery pipelines for a leading IT solution in the energy market, leveraging Apache Spark, Databricks, Delta Lake, and Python
  • Have end-to-end responsibility for the full lifecycle of features you develop
  • Design technical solutions for business requirements from the product roadmap
  • Ensure optimal performance
  • Refactor existing code and enhance system architecture to improve maintainability and scalability
  • Design and evolve the test automation strategy, including technology stack and solution architecture
  • Prepare reviews, participate in retrospectives, estimate user stories, and refine features ensuring their readiness for development
What we offer
What we offer
  • KMD (an NEC company) is committed to providing equal opportunities. Hence, we invite all qualified interested applicants to apply for career opportunities. At KMD all aspects of employment and cooperation including the decision to hire/cooperate with will be based on merit, competence, performance, and business needs without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, or any other status protected under local anti-discrimination legislation
Read More
Arrow Right

Senior Associate Software Engineer Databricks Developer

Amgen is looking for highly motivated expert Senior Data Engineer who can own th...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Any degree with 5 - 9 years of experience in Computer Science, IT or related field
  • Strong solution design and problem solving skills
  • Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, Redshift, and Scaled Agile methodologies
  • Proficiency in workflow orchestration, performance tuning on big data processing
  • Strong understanding of AWS services
  • Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures
  • Ability to quickly learn, adapt and apply new technologies
  • Strong problem-solving and analytical skills
  • Excellent communication and teamwork skills
  • Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable data pipelines using AWS (Redshift, S3, Glue, Lambda) and Databricks (Spark, Delta Lake) to support enterprise analytics and reporting
  • Architect and implement robust data models (dimensional and normalized) to enable high-performance querying and optimized reporting in Redshift
  • Build and optimize batch and real-time data processing frameworks, leveraging Spark Structured Streaming and cloud-native services
  • Lead data ingestion, transformation, and orchestration workflows ensuring data quality, reliability, and performance at scale
  • Perform advanced data analytics and root cause analysis to troubleshoot data discrepancies, performance issues, and pipeline failures
  • Ensure data security, compliance, and role-based access control (RBAC) across data environments
  • Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets
  • Develop and drive continuous improvements in CI/CD pipelines for automated data pipeline deployments, automated testing, version control, and monitoring for data platforms in a cloud-native environment
  • Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals
  • Stay up to date with emerging data technologies and best practices, ensuring continuous improvement
Read More
Arrow Right