CrawlJobs Logo
Realign Logo Realign · IT - Software Development

Senior Databricks Developer

United States 200000.00 USD / Year · Job Posted March 21, 2026
Apply Position
Job Link Share

Job Description

Must Have Technical/Functional Skills; Roles & Responsibilities

Job Responsibility

  • Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and Spark SQL
  • Optimize Spark jobs—including partitioning, caching, cluster sizing, shuffle minimization, and cost-efficient workload design
  • Build and manage workflows using Databricks Jobs, Repos, Delta Live Tables, and Unity Catalog
  • Develop and refine DBT models, tests, seeds, macros, and documentation to support standardized transformation layers
  • Implement modular, version-controlled DBT pipelines aligned with data governance and quality practices
  • Partner with data consumers to ensure models align with business definitions, lineage, and auditability
  • Create curated, reusable, and well-governed data assets (gold/silver/bronze layers) for analytics, reporting, and ML use cases
  • Continuously refine and optimize data assets for consistency, reliability, and usability across teams
  • Drive standardization of data patterns, frameworks, and reusable components
  • Identify and implement engineering efficiencies across Databricks and Spark workloads—cluster optimization, code improvements, auto-scaling patterns, and job orchestration enhancements
  • Collaborate with platform engineering to enhance DevOps automation, CI/CD pipelines, and environment management
  • Improve cost governance through workload analysis, optimization, and proactive cost monitoring
  • Conduct Spark job tuning and pipeline performance optimization to improve processing speed and reduce compute spend
  • Troubleshoot production issues and deliver durable fixes that improve long term reliability
  • Implement best practices for Delta Lake performance (ZORDER, auto-optimize, vacuum, retention tuning)
  • Implement end-to-end observability for data pipelines, including logging, metrics, tracing, and alerting
  • Integrate Databricks with monitoring ecosystems (e.g., Azure Monitor, CloudWatch, Datadog)
  • Ensure pipeline SLAs/SLOs are clearly defined and consistently met
  • Work closely with data architects, analysts, business SMEs, and platform teams
  • Provide technical leadership, review code, mentor junior engineers, and advocate for engineering excellence
  • Translate business requirements into scalable, production-quality data solutions

Requirements

  • 7+ years of experience in Data Engineering, with 3–5+ years on Databricks
  • Advanced proficiency in Apache Spark, PySpark, SQL, and distributed data processing
  • Strong experience with DBT (Core or Cloud) for building robust transformation layers
  • Hands-on expertise in data asset modeling, curation, optimization, and lifecycle management
  • Proven experience with job tuning, performance debugging, and cluster optimization
  • Experience implementing observability solutions for data pipelines
  • Solid understanding of Delta Lake, lakehouse architecture, and data governance
  • Experience with cloud platforms (Azure preferred
  • AWS/GCP acceptable)
  • Strong Git-based development workflows and CI/CD experience
Realign - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Databricks Developer

8 matching positions

Senior Databricks & Apache Spark Developer - Vice President

We are looking for a highly skilled Senior Databricks Engineer to contribute to ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years in data engineering or distributed systems
  • Strong expertise in Apache Spark (JavaSpark / PySpark), Databricks on AWS, and Delta Lake
  • Experience with AWS services and large‑scale distributed data processing
  • Experience modernizing or refactoring legacy data platforms into cloud‑based architectures
  • Strong background in Spark performance tuning and large‑scale batch optimization
  • Ability to translate architecture into implementable designs
  • Understanding of data modeling and pipeline orchestration patterns
  • Strong problem‑solving mindset for complex distributed systems
  • Comfortable working in time‑bound, high‑impact environments
  • Proactive, accountable, and collaborative
Job Responsibility
Job Responsibility
  • Platform Engineering & Modernization: Refactor and modernize existing Spark pipelines to Databricks native architectures
  • Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
  • Enhance and extend existing processing logic using optimized Spark (JavaSpark / PySpark) on Databricks
  • Databricks Native Development: Build and optimize solutions using Databricks features, including Delta Lake, Databricks Workflows for orchestration and Auto scaling and job clusters
  • Design & Solution Engineering: Contribute to low and mid level architecture and design
  • Translate high level architecture into detailed technical designs
  • Define data models, pipeline patterns, and reusable components
  • Ensure solutions are scalable, maintainable, and production ready
  • Performance Optimization & Simplification: Analyze, improve Spark job performance and simplify complex or over engineered pipelines into standardized, efficient patterns
  • Engineering Standards & Best Practices: Follow and contribute to Databricks and Spark engineering standards
  • Fulltime
Read More
Arrow Right

Senior Associate Software Engineer Databricks Developer

Amgen is looking for highly motivated expert Senior Data Engineer who can own th...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Any degree with 5 - 9 years of experience in Computer Science, IT or related field
  • Strong solution design and problem solving skills
  • Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, Redshift, and Scaled Agile methodologies
  • Proficiency in workflow orchestration, performance tuning on big data processing
  • Strong understanding of AWS services
  • Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures
  • Ability to quickly learn, adapt and apply new technologies
  • Strong problem-solving and analytical skills
  • Excellent communication and teamwork skills
  • Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable data pipelines using AWS (Redshift, S3, Glue, Lambda) and Databricks (Spark, Delta Lake) to support enterprise analytics and reporting
  • Architect and implement robust data models (dimensional and normalized) to enable high-performance querying and optimized reporting in Redshift
  • Build and optimize batch and real-time data processing frameworks, leveraging Spark Structured Streaming and cloud-native services
  • Lead data ingestion, transformation, and orchestration workflows ensuring data quality, reliability, and performance at scale
  • Perform advanced data analytics and root cause analysis to troubleshoot data discrepancies, performance issues, and pipeline failures
  • Ensure data security, compliance, and role-based access control (RBAC) across data environments
  • Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets
  • Develop and drive continuous improvements in CI/CD pipelines for automated data pipeline deployments, automated testing, version control, and monitoring for data platforms in a cloud-native environment
  • Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals
  • Stay up to date with emerging data technologies and best practices, ensuring continuous improvement
Read More
Arrow Right

Senior Developer

We are looking for a Senior Developer to support analytics and reporting initiat...
Location
Location
United States , Glendale
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Engineering, or a related technical field
  • At least 6 years of experience working with MicroStrategy in enterprise reporting environments
  • Minimum 5 years of experience with data warehousing concepts, structures, and reporting integration
  • Strong hands-on expertise in SQL, Python, and React for analytics and application development
  • At least 3 years of experience using Databricks or Snowflake in data-focused environments
  • Practical experience with AI-assisted coding tools in software development workflows
  • Demonstrated ability to solve complex technical problems independently with strong analytical judgment
Job Responsibility
Job Responsibility
  • Design, build, and enhance dashboards, reporting products, and data visualizations that support operational teams and executive decision-making
  • Develop and maintain MicroStrategy reporting assets while improving usability, accuracy, and overall system performance
  • Create and evolve custom web-based analytics applications using React to deliver modern reporting experiences
  • Convert business questions into technical specifications for data models, visual layers, and underlying database structures
  • Support the modernization of legacy analytics solutions by moving selected reporting capabilities into a React-based framework
  • Apply AI-assisted development tools to speed up prototyping, documentation, and coding tasks while preserving quality and review standards
  • Partner with architects and technical leads to introduce new development practices, technology approaches, and scalable reporting solutions
  • Optimize semantic models, queries, and reporting performance to ensure dependable and efficient analytics delivery
  • Collaborate effectively with distributed teammates and end users to refine requirements, troubleshoot issues, and deliver useful reporting outcomes
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right

Senior Databricks Engineer

We are looking for an experienced Senior Databricks Engineer to support enterpri...
Location
Location
United States , Cincinnati
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 5 years of experience in data engineering working with large-scale or distributed data processing platforms
  • Minimum 3 years of hands-on experience building solutions in Databricks within production environments
  • Strong coding ability in Python, especially PySpark, with additional SQL proficiency for data transformation and analysis
  • Solid knowledge of Apache Spark concepts, including execution behavior, performance tuning, and resource management
  • Practical experience with Delta Lake features such as schema evolution, versioned data management, and reliable transactional processing
  • Familiarity with cloud-based data ecosystems and orchestration tools used to schedule, monitor, and manage data workloads
  • Experience using version control and CI/CD practices to support repeatable and controlled data engineering deployments
  • ServiceNow knowledge is highly valued, especially for roles involving data extraction and integration into downstream platforms
Job Responsibility
Job Responsibility
  • Develop and maintain robust data pipelines in Databricks using Apache Spark to support enterprise reporting and analytical use cases
  • Extract and transform data from ServiceNow, then create and manage structured tables that supply ARDoc as part of a larger architecture initiative
  • Design layered data models using Delta Lake practices to improve scalability, data usability, and long-term maintainability
  • Configure and manage Databricks jobs, workflows, and compute resources to improve processing efficiency and platform stability
  • Investigate performance issues across Spark workloads and implement tuning strategies that improve speed, reliability, and cost control
  • Establish data governance practices through cataloging, lineage visibility, and secure access controls within the Databricks environment
  • Work closely with data scientists, analysts, and platform teams to deploy production-ready data assets and support advanced analytics needs
  • Provide technical guidance to other engineers by promoting engineering standards, reviewing designs, and sharing platform best practices
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
Read More
Arrow Right

Pyspark Big Data Senior Developer - Vice President

We are building an A-team of highly skilled and autonomous engineers, and we are...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of extensive, hands-on experience as a Senior Big Data Developer, with a strong emphasis on PySpark and the Apache Spark ecosystem, operating as a player/coach
  • Expert proficiency in Python, with a proven track record of developing robust, scalable, and high-performance PySpark applications for large-scale data processing
  • Deep understanding and extensive hands-on experience with Apache Spark (Spark Core, Spark SQL, Spark Streaming) and its ecosystem
  • Experience with distributed computing frameworks such as Hadoop (HDFS, YARN)
  • Expert proficiency in SQL and extensive experience with data warehousing concepts and technologies (e.g., Hive, Snowflake, Redshift, Databricks SQL)
  • Proven experience with various data storage formats (e.g., Parquet, ORC, Avro) and data lake solutions (e.g., Delta Lake, Iceberg)
  • Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
  • Strong experience with Apache Kafka for building real-time data pipelines and event-driven architectures
  • Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
  • Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is a mandatory requirement
Job Responsibility
Job Responsibility
  • Operate end-to-end in the design, development, and implementation of robust big data solutions, ensuring optimal performance, scalability, data quality, and security
  • Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for big data processing and analytical solutions
  • Act as a player/coach within the team, mentoring junior members and leading by example in the development of efficient and innovative big data architectures
  • Design, develop, and optimize large-scale data pipelines using PySpark for data ingestion, transformation, and aggregation, always with an eye towards efficiency and domain relevance
  • Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka
  • Design and implement sophisticated data warehousing solutions and dimensional models for efficient data storage and retrieval, ensuring alignment with business needs
  • Work with various distributed data storage technologies, including distributed file systems (e.g., HDFS, S3) and NoSQL databases (e.g., MongoDB, Cassandra), selecting the right tool for the right problem
  • Implement efficient data processing and storage strategies to optimize the performance and scalability of big data applications, with a strong focus on the 'why' behind the technology choices
  • Champion best practices in software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines
  • Demonstrate high autonomy and agency in driving projects forward, making informed decisions, and proactively identifying areas for improvement
  • Fulltime
Read More
Arrow Right

Salesforce Senior Developer

The Senior Salesforce Developer role involves designing, coding, and deploying c...
Location
Location
Mexico , Guadalajara
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 8-10+ years experience as a Salesforce Developer in Salesforce ecosystem (e.g., Sales Cloud, Service Cloud)
  • 5+ years of experience in Salesforce Service Cloud Voice and Amazon Connect i.e. Salesforce and Amazon Connect ecosystem (application, telephony, routing, AI, and DevOps layers)
  • Proficient in Apex, Visualforce, and Lightning Web Components
  • Strong understanding of Salesforce architecture and best practices
  • Experience with integrating Salesforce with other systems
  • Experience with Agile development methodologies and tools i.e. Jira
  • Ability to design and implement complex data models
  • Excellent problem-solving and analytical skills
  • Must have Salesforce Certified Platform Developer I and II, Certified Sales Cloud and Service Cloud and/or Architect certifications
Job Responsibility
Job Responsibility
  • Design, Coding/Programming , develop, test and deploy custom applications using Salesforce
  • Extend Salesforce capabilities to support various business functions, including custom applications and integrations
  • Management of user roles, permissions, profiles, flows, validation rules, and declarative automation
  • Leverage of Salesforce AI tools to automate administrative workflows and improve operational efficiency
  • Work closely with cross-functional teams to understand requirements and implement solutions
  • Document technical specifications and maintain updated system documentation
  • Collaborate with cross-functional teams to identify and prioritize business requirements and ensure successful implementation of Salesforce solutions
  • Provide technical leadership and mentorship to junior developers
  • Contribute to the development of best practices for Salesforce development
  • Experience in Databricks
  • Fulltime
Read More
Arrow Right

Senior Developer

This role is responsible for developing, enhancing, and maintaining enterprise a...
Location
Location
United States , Anaheim
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of MicroStrategy (Strategy) experience
  • 5+ years of data warehouse experience
  • 3+ years of experience with Databricks or Snowflake
  • Advanced SQL and Python skills
  • Experience building dashboards or applications using React
  • Hands‑on experience with AI‑assisted coding tools
  • Strong analytical, troubleshooting, and problem‑solving skills
  • High attention to detail and commitment to data accuracy
  • Bachelor’s degree in Computer Science, Engineering, or a related technical discipline
Job Responsibility
Job Responsibility
  • Design, develop, and maintain MicroStrategy dashboards, reports, and semantic/data models
  • Support and enhance custom analytics dashboards and web applications built with React
  • Migrate select legacy reporting solutions to modern React‑based frameworks
  • Perform data modeling, SQL development, and performance tuning
  • Translate business and reporting requirements into technical specifications and analytics solutions
  • Use AI‑assisted coding tools (e.g., Cursor, Claude Code) to accelerate development, prototyping, and documentation while maintaining coding standards
  • Collaborate with technical leads and architects to adopt new tools, standards, and best practices
  • Optimize reporting performance across datasets, dashboards, and databases
  • Work effectively with remote and cross‑functional team members
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • company 401(k) plan
  • free online training
Read More
Arrow Right

Senior Databricks Data Engineer

To develop, implement, and optimize complex Data Warehouse (DWH) and Data Lakeho...
Location
Location
Romania , Bucharest
Salary
Salary:
Not provided
https://www.inetum.com Logo
Inetum
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven, expert-level experience with the entire Databricks ecosystem (Workspace, Cluster Management, Notebooks, Databricks SQL)
  • in-depth knowledge of Spark architecture (RDD, DataFrames, Spark SQL) and advanced optimization techniques
  • expertise in implementing and managing Delta Lake (ACID properties, Time Travel, Merge, Optimize, Vacuum)
  • advanced/expert-level proficiency in Python (with PySpark) and/or Scala (with Spark)
  • advanced/expert-level skills in SQL and Data Modeling (Dimensional, 3NF, Data Vault)
  • solid experience with a major Cloud platform (AWS, Azure, or GCP), especially with storage services (S3, ADLS Gen2, GCS) and networking
  • bachelor’s degree in Computer Science, Engineering, Mathematics, or a relevant technical field
  • minimum of 5+ years of experience in Data Engineering, with at least 3+ years of experience working with Databricks and Spark at scale.
Job Responsibility
Job Responsibility
  • Design and implement robust, scalable, and high-performance ETL/ELT data pipelines using PySpark/Scala and Databricks SQL on the Databricks platform
  • expertise in implementing and optimizing the Medallion architecture (Bronze, Silver, Gold) using Delta Lake
  • design and implement real-time/near-real-time data processing solutions using Spark Structured Streaming and Delta Live Tables (DLT)
  • implement Unity Catalog for centralized data governance, fine-grained security (row/column-level security), and data lineage
  • develop and manage complex workflows using Databricks Workflows (Jobs) or external tools (Azure Data Factory, Airflow) to automate pipelines
  • integrate Databricks pipelines into CI/CD processes using tools like Git, Databricks Repos, and Bundles
  • work closely with Data Scientists, Analysts, and Architects to deliver optimal technical solutions
  • provide technical guidance and mentorship to junior developers.
What we offer
What we offer
  • Full access to foreign language learning platform
  • personalized access to tech learning platforms
  • tailored workshops and trainings to sustain your growth
  • medical insurance
  • meal tickets
  • monthly budget to allocate on flexible benefit platform
  • access to 7 Card services
  • wellbeing activities and gatherings.
  • Fulltime
Read More
Arrow Right