CrawlJobs Logo

Senior Databricks Administrator

dutechsystems.com Logo

Dutech Systems

Location Icon

Location:
United States , Austin

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

We are seeking an experienced Senior Databricks Administrator to manage and optimize our Databricks platform in an AWS cloud environment. The ideal candidate will have strong expertise in Databricks workspace administration, Apache Spark, cloud integrations, and data platform governance. This role focuses on ensuring platform performance, security, scalability, and cost optimization for enterprise data and AI workloads.

Job Responsibility:

  • Administer and manage Databricks workspaces in AWS cloud environments
  • Configure and optimize Databricks clusters, job scheduling, and workspace settings
  • Manage user access, roles, and permissions using IAM, SCIM, and RBAC
  • Implement and enforce cluster policies and governance standards
  • Monitor platform health, performance, and availability
  • Optimize Apache Spark workloads for performance and scalability
  • Integrate Databricks with cloud storage services (e.g., Amazon S3)
  • Manage and support Databricks SQL, notebooks, and job orchestration
  • Ensure data security, encryption, and compliance requirements are met
  • Automate infrastructure and workflows using Terraform, scripting, and CI/CD pipelines
  • Support AI/ML workloads using Databricks ML and MLflow (as needed)

Requirements:

  • 8+ years of experience in Databricks administration in AWS environments
  • Strong expertise in Databricks cluster configuration and workspace management
  • Hands-on experience with Apache Spark (performance tuning & troubleshooting)
  • Experience managing IAM, SCIM, and role-based access control (RBAC)
  • Experience integrating Databricks with cloud storage (S3 or similar)
  • Strong understanding of data security, encryption, and compliance
  • Experience with DevOps tools (Terraform, CI/CD pipelines, scripting)

Nice to have:

  • Experience in enterprise or government environments
  • Familiarity with Databricks Unity Catalog for governance
  • Experience with cost optimization strategies for Databricks workloads
  • Knowledge of data lake/lakehouse architectures
  • Experience supporting AI/ML workloads (MLflow)
  • Programming knowledge in Python, SQL, or Scala

Additional Information:

Job Posted:
April 19, 2026

Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Senior Databricks Administrator

Senior Data Engineer (DATABRICKS)

As a Senior Databricks Administrator, you will be responsible for the setup, con...
Location
Location
Ireland , Dublin 18
Salary
Salary:
Not provided
mastercard.com Logo
Mastercard
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of experience in Databricks administration on AWS or multi-cloud environments
  • Deep understanding of Databricks workspace architecture, Unity Catalog, and cluster configuration best practices
  • Strong experience in managing IAM policies, SCIM integration, and access provisioning workflows
  • Hands-on experience with monitoring, cost optimization, and governance of large-scale Databricks deployments
  • Hands-on experience with infrastructure-as-code (Terraform) and CI/CD pipelines
  • Experience with ETL orchestration and collaboration with engineering teams (Databricks Jobs, Workflows, Airflow)
Job Responsibility
Job Responsibility
  • Databricks Platform Administration (AWS) Provision, configure, and manage Databricks Workspaces across multiple environments (Dev, UAT, Prod)
  • Define and enforce cluster policies, compute tagging, and usage controls to optimize cost and resource utilization
  • Administer Unity Catalog, including creation and management of catalogs, schemas, access control, and data lineage
  • Manage SCIM integrations for user/group provisioning with IdPs like Okta or Azure/AWS SSO
  • Configure and maintain CI/CD pipelines for notebook deployment and workflow promotion using GitHub, GitLab, or similar tools
  • Security, Governance & Compliance Implement and enforce role-based access controls (RBAC) and fine-grained data permissions using Unity Catalog and Lake Formation
  • Ensure auditability and lineage tracking for compliance with Open Banking, GDPR, and PSD2 regulations
  • Configure and manage token scopes, secrets management, and credential passthrough for secure access to underlying data
  • Work with InfoSec and compliance teams to ensure platform security aligns with regulatory frameworks
  • Monitoring, Support & Troubleshooting Monitor workspace performance, cluster health, job execution, and user activity using Databricks native tools, CloudWatch, and third-party integrations
  • Fulltime
Read More
Arrow Right

Principal Engineer I – Senior Azure Databricks Administrator

Software Resources has an immediate, direct hire job opportunity for a Principal...
Location
Location
United States , Phoenix
Salary
Salary:
Not provided
softwareresources.com Logo
Software Resources
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of related experience in data analytics administration and development
  • 4+ years of Databricks related experience
  • Bachelor’s degree in related field required
  • Advanced proven experience in Azure Databricks (Workspace management, Clusters, Jobs, Unity Catalog, Delta Lake, User access management, Rest APIs and SDKs)
  • Knowledge of MLFlow & MLOps
  • Deep understanding of Azure infrastructure and data services, including Azure Data Lake, Azure Data Factory, Azure SQL, Azure Synapse Analytics, Azure Key Vault, Azure Monitor, networking
  • Experience with CI/CD pipelines (Azure DevOps preferred)
  • Strong programming skills in SQL, Python, and/or PySpark
  • Advanced proven experience in leading cross-functional teams and managing multiple projects simultaneously
  • Advanced ability to see the big picture and align projects with organizational goals
Job Responsibility
Job Responsibility
  • Responsible for delivery and operations of technologies and platforms required to model, transform, analyze, report, visualize data
  • Provide SME expertise in designing, building, optimizing, streamlining and automating the Azure Databricks platform
  • Partner with ML engineers, data scientists, data analysts, and enterprise architects to provide frameworks, set standards, enforce best practices, train and enable users
  • Develop technical skills of one or more junior team-members
  • Take assignments that can be worked on individually without supervision, and manage work effort from concept to completion
  • Design, build, optimize, automate and maintain the Azure Databricks platform, ensuring scalability, security, governance and performance
  • Design, implement and manage Azure Databricks workspaces, clusters, jobs, access management
  • Design, implement and manage policies, monitoring and observability
  • Implement data analytics principles aimed at business enablement, reliability practices and sound recovery procedures
  • Ensure compliance with IT policies, procedures, and industry standards
What we offer
What we offer
  • Competitive salaries
  • An ownership stake in the company
  • Medical and dental insurance
  • Time off
  • A great 401k matching program
  • Tuition assistance program
  • An employee volunteer program
  • A wellness program
  • Fulltime
Read More
Arrow Right

Senior Azure Administrator II

Location
Location
India , Bangalore, Kolkata
Salary
Salary:
Not provided
inxiteout.ai Logo
InXiteOut
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of solid experience as Azure Admin or DevOps Engineer or Platform Engineer, building CI/CD pipelines and deploying AI / ETL workloads involving various Azure services including VM, Databricks, ADF, ADLS etc. as well as microservices developed in MERN stack
  • CI/CD Pipeline Mastery: Azure Pipelines: Deep expertise in building and managing both Classic and YAML-based pipelines, including multi-stage deployments, release strategies, and integration
  • GitHub Actions and Azure DevOps Services for orchestrating automated builds, tests, and deployments across microservices
  • Infrastructure as Code (IaC): Proficiency in Terraform, Azure Resource Manager (ARM) templates, and Bicep for declarative infrastructure provisioning
  • Enables consistent, scalable, and auditable environments for AKS clusters and supporting services
  • Containerization & Orchestration: Strong command of Docker for image creation and management
  • Expertise in Kubernetes Container Distribution through pipeline, especially Azure Kubernetes Service (AKS) and scaling strategies
  • Familiarity with secure image storage and deployment using Azure Container Registry (ACR) and Azure Container Apps (ACA)
  • Deep expertise of deploying AI and ETL workloads including VM, Databricks, ADF, ADLS etc. involving Python and PySpark modules
  • Deep expertise of deploying web applications / microservices developed in MERN stack
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Platform

We’re looking for a seasoned software engineer to join Parafin’s Infrastructure ...
Location
Location
United States , San Francisco
Salary
Salary:
230000.00 - 265000.00 USD / Year
parafin.com Logo
Parafin
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of experience in software engineering with a strong background in data infrastructure, pipelines, and distributed systems
  • Advanced proficiency in Python and SQL
  • Hands-on Spark development experience
  • Expertise with modern cloud data stacks—AWS (S3, RDS), Databricks, and Airflow—and lakehouse architectures
  • Hands‑on experience with foundational data‑infrastructure technologies such as Hadoop, Hive, Kafka (or similar streaming platforms), Delta Lake/Iceberg, and distributed query engines like Trino/Presto
  • Familiarity with ingestion frameworks, developer‑experience tooling, and best practices for data versioning, lineage, partitioning, and clustering
  • Strong problem-solving skills and a proactive attitude toward ownership and platform health
  • Excellent communication and collaboration skills, especially in cross-functional settings
Job Responsibility
Job Responsibility
  • Design and build robust, highly scalable data pipelines and lakehouse infrastructure with PySpark, Databricks, and Airflow on AWS
  • Improve the data platform development experience for Engineering, Data Science, and Product by creating intuitive abstractions, self‑service tooling, and clear documentation
  • Own and maintain core data pipelines and models that power internal dashboards, ML models, and customer-facing products
  • Own the Data & ML platform infrastructure using Terraform, including end‑to‑end administration of Databricks workspaces: manage user access, monitor performance, optimize configurations (e.g., clusters, lakehouse settings), and ensure high availability of data pipelines
  • Lead projects to improve data quality, testing, observability, and cost efficiency across existing pipelines and backend systems (e.g., migrating Databricks SQL pipelines to dbt, scaling data ingestion, improving data-lineage tracking, and enhancing monitoring)
  • Act as the primary engineering partner for the Data Science team—embedded closely to gather requirements, design scalable solutions, and provide end-to-end support on all engineering aspects of their work
  • Work closely with backend engineers and data scientists to design performant data models and support new product development initiatives
  • Share best practices and mentor other engineers working on data-centric systems
What we offer
What we offer
  • Equity grant
  • Medical, dental & vision insurance
  • Work from home flexibility
  • Unlimited PTO
  • Commuter benefits
  • Free lunches
  • Paid parental leave
  • 401(k)
  • Employee assistance program
  • Fulltime
Read More
Arrow Right

340B Senior Business Analyst

This role will be an integral part of the Commercial Analytics team within Healt...
Location
Location
United States
Salary
Salary:
100700.00 - 155100.00 USD / Year
cencora.com Logo
Cencora
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Broad training in fields such as business administration, health information management, computer sciences, statistics, advanced analytics or similar vocations generally obtained through completion of a four-year bachelor’s degree program
  • Three (3) or more years of experience with analyzing and/or reporting on 340B Program, pharmaceutical, supply chain and/or healthcare data is preferred
  • Three (3) or more years of data analyst experience preferred
  • A minimum of six (6) years directly related and progressively responsible experience applying advanced analysis techniques to business problems
  • Experience managing large datasets and performing advanced analytics (i.e. Excel modeling, Access)
  • Demonstrates an ability to work with large amounts of data: facts, figures, and number crunching
  • Demonstrates an ability to present complex data, findings, ideas, and/or information to enable understanding and take action
  • Demonstrates an ability to review the data for trends, drawing logical conclusions based on these findings
  • Demonstrates vigilance in precision of data analysis in order to draw the correct conclusion of findings
  • Demonstrates strong collaboration and communication skills
Job Responsibility
Job Responsibility
  • Serves as a subject matter expert for 340B data, reporting and financial performance
  • Possess a deep understanding of the levers and their impact on the 340B P&L and operations
  • Supports operational and commercial decision making via business intelligence processes
  • Demonstrates proficiency in working with large, complex data sets to identify patterns, correlations and outliers
  • Demonstrates the ability to distill relevant insights and showcase them through relevant visualizations
  • Proactively develops new reporting tools and models to support initiatives aimed at driving profitability, finding new market opportunities, improving operational efficiency and program reliability
  • Ensures consistency and high degree of accuracy across 340B data and reports
  • Effectively collaborates with other cross-functional teams for the purposes of making data requests or collaborating with them on 340B related projects
  • Develops and deploy educational processes, training and support for 340B data and reports
  • Perform related duties as assigned
What we offer
What we offer
  • Medical
  • dental
  • vision care
  • comprehensive suite of benefits that focus on the physical, emotional, financial, and social aspects of wellness
  • support for working families
  • backup dependent care
  • adoption assistance
  • infertility coverage
  • family building support
  • behavioral health solutions
  • Fulltime
Read More
Arrow Right

Senior Manager, Internal Audit

Reporting to the Head of Internal Audit, the Internal Audit Senior Manager will ...
Location
Location
United States , New York; San Francisco
Salary
Salary:
130000.00 - 215000.00 USD / Year
airwallex.com Logo
Airwallex
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree or above in Accounting, Finance, Business Administration, or a related field
  • At least 10 years of experience as an auditor in a Big 4 firm and/or financial industry practitioner (e.g., at a fintech, bank, card scheme, or money services business)
  • Proven track record in financial crime compliance, fraud, card scheme compliance (Issuing and Acquiring), and enterprise risk management
  • Deep experience in the US regulatory landscape, alongside good understanding of and Canadian or Latin America regulatory requirements
  • Detail-oriented, positive attitude, and self-motivated
  • Good interpersonal skills with the ability to present complex and sensitive issues to senior management and influence change
Job Responsibility
Job Responsibility
  • Develop comprehensive, risk-based audit plans and testing procedures for risk and compliance audits, prioritizing areas with the highest potential for non-compliance or financial loss
  • Execute audit fieldwork, assess the design and operating effectiveness of controls, and deliver insightful, actionable reports to senior management and the Audit Committee
  • Develop and implement data analysis tools (e.g. using Python, Databricks, Looker) to improve audit efficiency and effectiveness, and leverage analytics to identify emerging risks and anomalies
  • Manage internal business stakeholders, communicate complex risk and compliance issues clearly, and ensure that audit recommendations are understood and implemented
  • Monitor and track completion of management action plans, and follow up to ensure timely remediation of audit findings
  • Stay abreast of regulatory changes and industry best practices, and promote a culture of continuous improvement and operational excellence
What we offer
What we offer
  • Offers Equity
  • Offers Bonus
  • medical, dental, and vision insurance
  • a 401(k) plan
  • short-term and long-term disability
  • basic life insurance
  • well-being benefits
  • 20 paid days of vacation
  • 12 paid days of company holidays in a calendar year
  • Fulltime
Read More
Arrow Right

Cloud Platform Architect Manager

The role involves leading a team of architects and cloud platform administrators...
Location
Location
United States of America , Charlotte
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of professional experience in designing & architecting Cloud platform solutions, with a focus on Azure-based platforms and enterprise-scale systems
  • 3+ years’ experience leading teams of cloud platform architects and platform administrators within a global organization
  • Working experience in designing and architecting cloud platform solutions that leverage Databricks and Snowflake
  • Hands-on experience with Azure Cloud services (Azure VM, Azure functions, Container Services, IaaS, PaaS, Cosmos DB, Azure Storage, Messaging services, Azure Identity and Security)
  • Practical experience in designing systems to handle large data volumes
  • Practical experience in designing systems for large-scale processing with a focus on security, performance optimization and cost management
  • Working experience in designing and architecting solutions that comply with data security industry standards and regulations, including RBAC, data encryption (GDPR, PCI, etc.), and monitoring
  • Fluent English (all project communication in English).
Job Responsibility
Job Responsibility
  • Leading a team of platform architects to ensure work is assigned, understood and completed by the team members as per agreed timelines and in accordance with team members’ skillsets and company standards
  • Executing the required performance management requirements for your employee direct reports including but not limited to setting annual objects / Key Result Area (KRAs), mid-year and year-end progress reviews, and career growth conversations
  • Building roadmaps to meet short- and long-term business objectives within program timelines and overseeing the delivery of solutions based on business requirements
  • Communicate effectively with senior management, stakeholders, and teams on progress, risks and opportunities
  • Mentoring Data Engineering and Data Science teams on, for example, leveraging Azure platform capabilities, such as configuring SPNs for system connections, enabling monitoring/logging for pipelines, and implementing security protocols
  • Providing subject matter expertise in designing, architecting and maintaining Azure Cloud Platforms
  • Performing architectural assessments, defining solutions and producing detailed design documentation addressing complex business needs
  • Working with the Global BI, DE, and Analytics teams to define, drive and enforce architecture standards, best practices, and governance frameworks across the platform
  • Contributing and collaborating with other architects within the Enterprise Architecture team to get the required approval from the Architecture Review Board for new initiatives
  • Providing technical direction on Azure platform services, including best practices for system configurations, integration patterns, and support for business requirements
What we offer
What we offer
  • Equal Opportunity Employer policies
  • Complies with the Americans with Disabilities Act (ADA) and state/local disability laws
  • Reasonable accommodations as per the ADA and local laws.
  • Fulltime
Read More
Arrow Right

Senior Manager of Data Engineering

At GEICO Technology Solutions, we are on a journey to revolutionize the Insurtec...
Location
Location
United States , Palo Alto
Salary
Salary:
150000.00 - 300000.00 USD / Year
geico.com Logo
Geico
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of progressive experience in data engineering, database administration, or data platform development
  • 4+ years of people management experience leading technical teams (5+ direct reports)
  • Proven track record of building and scaling data pipelines in production environments
  • Experience managing enterprise database platforms (Oracle, PostgreSQL, or equivalent)
  • Deep expertise in relational databases (Oracle, PostgreSQL, MSSQL)
  • familiarity with NoSQL (MongoDB, Cassandra)
  • Working knowledge of Kafka, Kinesis, or Pulsar for real-time data streaming
  • Strong experience with Azure or AWS, data services
  • Hands-on experience with Snowflake, BigQuery, Redshift, or Databricks
  • Proficiency with Apache Airflow, dbt, or similar orchestration tools
Job Responsibility
Job Responsibility
  • Build and maintain scalable data pipelines using modern orchestration tools
  • Implement data quality frameworks, monitoring, and alerting
  • Create self-service data products that reduce time-to-insight for business users
  • Collaborate with Architecture, Security, and Compliance teams on data governance
  • Champion data engineering best practices: version control, CI/CD, infrastructure-as-code
  • Partner with Analytics and Data Science teams to deliver ML-ready datasets
  • Report to leadership on platform health, roadmap progress, and strategic opportunities
What we offer
What we offer
  • Comprehensive Total Rewards program
  • 401K savings plan vested from day one that offers a 6% match
  • performance and recognition-based incentives
  • tuition assistance
  • mental healthcare
  • fertility and adoption assistance
  • workplace flexibility
  • GEICO Flex program, which offers the ability to work from anywhere in the US for up to four weeks per year
  • Fulltime
Read More
Arrow Right