CrawlJobs Logo
Wissen Logo Wissen · IT - Software Development

Lead Databricks Developer

India, Bangalore South · Job Posted March 05, 2026
Apply Position
Job Link Share

Job Description

We are seeking an experienced Lead Databricks Developer with deep expertise in Spark, PySpark, and SQL to design, build, and optimize large-scale data processing solutions. The ideal candidate will have strong hands-on experience in cloud-based data environments (preferably AWS) and will be capable of leading end-to-end data engineering initiatives. This role involves collaborating with cross‑functional teams, ensuring high data quality, and building robust pipelines to support business analytics and data-driven decision-making.

Job Responsibility

  • Design, develop, and maintain scalable data processing applications using Python, PySpark, and Apache Spark
  • Work in cloud-based environments—AWS preferred, with flexibility to work on Azure Databricks
  • Collaborate closely with data engineers, data scientists, analysts, and business stakeholders to understand requirements and deliver effective solutions
  • Ensure data integrity, reliability, and performance across data pipelines and workflows
  • Write clean, well-structured, and maintainable code following industry best practices
  • Perform exploratory data analysis and implement data validation frameworks
  • Monitor and troubleshoot performance bottlenecks in Spark jobs and data workflows
  • Implement and manage CI/CD pipelines for automated build, test, and deployment of data pipelines
  • Optimize data pipelines for cost, performance, and scalability in cloud environments

Requirements

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
  • 8–12 years of experience in data engineering with a strong focus on Databricks and Spark/PySpark
  • Strong proficiency in SQL and experience working with relational databases
  • Advanced hands‑on experience in Python for data engineering
  • Solid understanding of cloud platforms (AWS preferred
  • Azure Databricks acceptable)
  • Experience with Git or other version control systems
  • Strong analytical thinking, debugging, and problem‑solving skills
  • Excellent communication and cross‑team collaboration abilities

Nice to have

  • Experience with Delta Lake, Unity Catalog, or advanced Databricks features
  • Exposure to data warehousing concepts and dimensional modeling
  • Familiarity with Airflow, Glue, or other workflow orchestration tools
  • Knowledge of data security, governance, and compliance best practices
  • Understanding of DevOps, containerization (Docker), or Kubernetes
  • Experience integrating Databricks with BI tools (Power BI, Tableau, etc.)
  • Knowledge of machine learning workflows or MLOps within Databricks is a plus
Wissen - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Lead Databricks Developer

8 matching positions

Senior Associate Software Engineer Databricks Developer

Amgen is looking for highly motivated expert Senior Data Engineer who can own th...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Any degree with 5 - 9 years of experience in Computer Science, IT or related field
  • Strong solution design and problem solving skills
  • Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, Redshift, and Scaled Agile methodologies
  • Proficiency in workflow orchestration, performance tuning on big data processing
  • Strong understanding of AWS services
  • Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures
  • Ability to quickly learn, adapt and apply new technologies
  • Strong problem-solving and analytical skills
  • Excellent communication and teamwork skills
  • Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable data pipelines using AWS (Redshift, S3, Glue, Lambda) and Databricks (Spark, Delta Lake) to support enterprise analytics and reporting
  • Architect and implement robust data models (dimensional and normalized) to enable high-performance querying and optimized reporting in Redshift
  • Build and optimize batch and real-time data processing frameworks, leveraging Spark Structured Streaming and cloud-native services
  • Lead data ingestion, transformation, and orchestration workflows ensuring data quality, reliability, and performance at scale
  • Perform advanced data analytics and root cause analysis to troubleshoot data discrepancies, performance issues, and pipeline failures
  • Ensure data security, compliance, and role-based access control (RBAC) across data environments
  • Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets
  • Develop and drive continuous improvements in CI/CD pipelines for automated data pipeline deployments, automated testing, version control, and monitoring for data platforms in a cloud-native environment
  • Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals
  • Stay up to date with emerging data technologies and best practices, ensuring continuous improvement
Read More
Arrow Right

Data & Databricks Practice Lead

We are looking for a dynamic and visionary Practice Lead to build and scale our ...
Location
Location
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in data engineering, architecture, and platform modernization
  • 3+ years of hands-on experience with Databricks, Apache Spark, Iceberg, and Scala
  • Deep expertise in cloud-native data platforms (Azure Data Fabric, Amazon EMR, Redshift)
  • Proven track record in building ML pipelines, implementing MLOps (MLflow), and integrating Redshift ML
  • Strong understanding of data governance, observability, and schema evolution
  • Excellent leadership, communication, and stakeholder management skills
Job Responsibility
Job Responsibility
  • Define and execute the strategic roadmap for the Data & AI practice, aligning with evolving trends in Lakehouse, cloud-native platforms, and AI engineering
  • Build reusable solution accelerators (e.g., Iceberg ingestion templates, Spark ETL frameworks, MLflow pipelines)
  • Establish and manage strategic partnerships with Databricks, AWS, Azure, and other ecosystem players
  • Represent the practice in industry forums, client workshops, and thought leadership initiatives
  • Architect scalable Lakehouse solutions using Delta Lake or Iceberg, optimized for performance and cost
  • Lead the design of unified batch + streaming pipelines using Spark Structured Streaming and Scala
  • Oversee multi-cloud data platform modernization using Azure Data Fabric, EMR, and Redshift
  • Guide implementation of data governance frameworks using Unity Catalog, Azure Purview, and Iceberg metadata
  • Mentor and grow a high-performing team of data engineers, ML engineers, and platform specialists
  • Define career paths, training plans, and certification goals aligned with Databricks, AWS, and Azure
  • Fulltime
Read More
Arrow Right

Technical Lead – Azure Databricks with ETL (Informatica)

We are seeking an experienced Technical Lead specializing in Azure Databricks an...
Location
Location
Canada , Toronto
Salary
Salary:
150000.00 USD / Year
realign-llc.com Logo
Realign
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience with Azure Databricks and big data processing
  • Expertise in ETL development using Informatica
  • Proficiency in SQL, Python, or Scala for data processing
  • Experience with data modeling, data warehousing, and analytics pipelines
  • Strong leadership, mentoring, and communication skills
  • Experience working in Agile development environments
Job Responsibility
Job Responsibility
  • Lead the design and implementation of ETL pipelines using Informatica
  • Develop and maintain data processing workflows in Azure Databricks
  • Collaborate with architects, data engineers, and business analysts to define technical solutions
  • Ensure data quality, performance, and scalability of ETL and data workflows
  • Mentor and guide team members on best practices in data engineering
  • Troubleshoot and resolve technical issues related to ETL and data processing
  • Document technical designs, processes, and procedures
  • Fulltime
Read More
Arrow Right

Lead Power BI Developer

At 3Cloud, we hire people who aren’t afraid to experiment or fail. We hire peopl...
Location
Location
Philippines
Salary
Salary:
Not provided
3cloudsolutions.com Logo
3Cloud
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in computer, engineering, data science or a related field
  • or equivalent work experience
  • Experience managing / leading a team
  • 8+ years working with data and business intelligence solutions on the Microsoft tool stack for Business Intelligence (SSRS, Power BI, or PowerPivot (Excel) or PerformancePoint), or other vendor Business Intelligence tools (Business Objects, MicroStrategy, Qlik, SAP)
  • Working knowledge of TSQL, data modeling, reporting using Power BI or other Microsoft tools
  • Experience creating complete Power BI report solutions based on business questions with linked content, drilldowns, and nested parameters
  • Dynamic measure designs using advanced DAX techniques
  • Knowledge of data mashups and transformations using Power Query/M language
  • Utilize Databricks platform to develop, implement, and optimize data pipelines and workflows.
  • Ability to report visual choices and visual interactions
Job Responsibility
Job Responsibility
  • Conduct customer discovery meetings to determine reporting requirements
  • Understand and work with customer on Power BI Licensing, and understand the limits/advantages of each licensing platform
  • Create well performing star schema data models for use with Power BI, including proper calculated columns and measures
  • Creating Power BI solutions for customers including importing data from multiple sources, building reports and dashboards, and creating appropriate data elements using DAX and source queries
  • Guide customers on deploying Power BI solutions to the Power BI service (Gateways, DirectQuery, DataFlows, Data Updates)
  • Fulltime
Read More
Arrow Right

Lead Power BI Developer

Lead consultant specializing in Power BI with a focus on data. Must have advance...
Location
Location
Philippines
Salary
Salary:
Not provided
3cloudsolutions.com Logo
3Cloud
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in computer, engineering, data science or a related field
  • or equivalent work experience
  • Experience managing / leading a team
  • 8+ years working with data and business intelligence solutions on the Microsoft tool stack for Business Intelligence (SSRS, Power BI, PowerPivot, PerformancePoint) or other vendor BI tools (Business Objects, MicroStrategy, Qlik, SAP)
  • Working knowledge of TSQL, data modeling, reporting using Power BI or other Microsoft tools
  • Experience creating complete Power BI report solutions with linked content, drilldowns, and nested parameters
  • Dynamic measure designs using advanced DAX techniques
  • Knowledge of data mashups and transformations using Power Query/M language
  • Utilize Databricks platform to develop, implement, and optimize data pipelines and workflows
  • Ability to report visual choices and visual interactions
Job Responsibility
Job Responsibility
  • Conduct customer discovery meetings to determine reporting requirements
  • Understand and work with customer on Power BI Licensing
  • Create well performing star schema data models for use with Power BI
  • Creating Power BI solutions for customers including importing data, building reports and dashboards
  • Guide customers on deploying Power BI solutions to the Power BI service (Gateways, DirectQuery, DataFlows, Data Updates)
  • Fulltime
Read More
Arrow Right

Lead Veeva/Salesforce CRM Developer

The Specialist Software Engineer collaborates with Salesforce/Veeva CRM Admins, ...
Location
Location
United States
Salary
Salary:
137457.00 - 171183.00 USD / Year
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Doctorate degree
  • OR Master’s degree and 2 years of computer science, IT, or related field
  • OR Bachelor’s degree and 4 years of computer science, IT, or related field
  • OR Associate’s degree and 8 years of computer science, IT, or related field
  • OR High school diploma / GED and 10 years of computer science, IT, or related field
  • 5+ years of experience as a Salesforce CRM Developer
  • In-depth knowledge of Salesforce development, including experience working with MuleSoft or Databricks integrations
  • Hands-on experience with the Salesforce security model, including OWD, sharing rules, profiles, and permissions
  • Experience with Data Loader, data migration tools, and CI/CD tools
  • Salesforce Developer 2 (required)
Job Responsibility
Job Responsibility
  • Leads large CRM migration to ensure configurations, customizations, and data migration are successfully implemented in the new CRM
  • Drives development in Salesforce/Veeva CRM to build and configure new and existing features
  • Works closely with Product Owners and cross-functional Development Teams to plan and schedule releases, providing guidance on release dependencies, technical constraints, and risks
  • Supports the development teams in automating build, test, and deployment processes, enabling continuous integration and continuous delivery in the Salesforce/Veeva ecosystem
  • Provides ongoing support and troubleshooting for post-release issues, ensuring timely resolution of any problems that arise
  • Maintains and optimizes existing configurations and customizations by limiting or eliminating technical debt to streamline the system
  • Partners with Development Teams to ensure agile software delivery and DevOps practices are applied to move most effectively within our CI/CD pipeline
  • Researches Veeva/Salesforce capabilities as needed to suit business requirements and provides gap analysis
  • Works closely with other developers in the team, including Product Owners, Business Analysts, Salesforce Admins, QA Analysts, and Scrum Masters, across cross-functional teams
  • Develops innovative solutions using generative AI technologies
What we offer
What we offer
  • A comprehensive employee benefits package, including a Retirement and Savings Plan with generous company contributions, group medical, dental and vision coverage, life and disability insurance, and flexible spending accounts
  • A discretionary annual bonus program, or for field sales representatives, a sales-based incentive plan
  • Stock-based long-term incentives
  • Award-winning time-off plans
  • Flexible work models, including remote and hybrid work arrangements, where possible
  • Fulltime
Read More
Arrow Right
New

Sr Manager IT Quality Assurance

The IT Senior Manager, CI/CD Engineering & DevOps is responsible for owning and ...
Location
Location
United States
Salary
Salary:
Not provided
valvoline.com Logo
Valvoline
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree (prefer Software Engineering, Information Systems, business or related field)
  • 8+ years of progressive experience in software delivery, DevOps engineering, or release management
  • 5+ years managing enterprise-scale CI/CD pipelines in a heterogeneous technology environment
  • Demonstrated experience with automated testing at enterprise scale (functional, regression, integration, performance)
  • Experience supporting large ERP platform delivery (SAP ECC and/or S/4HANA strongly preferred)
  • Proven track record leading production cutover, failover validation, and hypercare operations for major releases
  • Expert-level proficiency with Jenkins, GitHub Actions, Azure DevOps, or equivalent modern pipeline orchestration tools
  • Strong knowledge of pipeline-as-code (Jenkinsfile, YAML workflows), artifact management (Nexus, JFrog), and environment promotion patterns
  • Experience with GitOps principles, branching strategies (trunk-based, GitFlow), and PR governance controls
  • Deep expertise in enterprise automated testing frameworks: Selenium, Playwright, Tosca, REST Assured, OpenText UFT One
Job Responsibility
Job Responsibility
  • Own the enterprise CI/CD pipeline strategy, architecture, and execution across all VGO platforms
  • Drive adoption of modern pipeline tools (Jenkins, GitHub Actions, cALM, SAP Solution Manager, Azure DevOps, ArgoCD, other pipelines)
  • Enforce pipeline-as-code standards, enforce branch strategy governance, and ensure automated build, test, and deploy processes operate across SAP BTP, Salesforce, Databricks, and custom application stacks
  • Lead the shift-left engineering culture embedding quality and security at every pipeline stage
  • Define and govern the enterprise automated testing strategy encompassing functional, integration, regression, performance, and security test automation
  • Own the automation-first mandate
  • Establish code coverage thresholds, regression gates, and test-data management standards
  • Oversee toolchain selection and governance (Selenium, Playwright, Tosca, UFT One, OpenText ALM, REST Assured) aligned to VGO's SAP S/4HANA, Salesforce, and API integration landscape
  • Establish and enforce enterprise code quality standards and automated code review practices using static analysis tooling (SonarQube, CodeQL, ESLint, ABAP Code Inspector)
  • Govern pull request workflows, peer review SLAs, and automated gate enforcement within GitHub Enterprise
  • Fulltime
Read More
Arrow Right

Principal Data Scientists

Principal Data Scientists located in Bellevue, WA will implement and maintain mo...
Location
Location
United States , Bellevue
Salary
Salary:
196914.00 - 224000.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master’s degree in Applied Economics, Economics, Mathematics, Operations Research, Statistics, Finance, or related, or its foreign equivalent and 5 years of relevant work experience
  • Bachelor’s degree in Applied Economics, Economics, Mathematics, Operations Research, Statistics, Finance, or related, or its foreign equivalent and 7 years of relevant work experience
  • At least 18 years of age
  • Legally authorized to work in the United States
  • Using SQL and Python or other statistical/analytical programming languages to manipulate large amounts of data, extract key insights from the data, and then clearly and concisely communicate actionable recommendations based upon insight
  • Working independently to identify new segmentation opportunities using statistical methods including decision tree, clustering, leading to enhancements to decision process and policies
  • Developing predictive analytical models using the appropriate statistical methodologies, including logistics regression, experimental design, and hypothesis testing
  • Extracting, loading, and transforming data from multiple sources necessary for statistical, reporting and ad-hoc analysis
  • Building complex machine learning algorithms with automated model parameter tuning
  • Working with a cloud computing environment including Azure Databricks and AWS
Job Responsibility
Job Responsibility
  • Implement and maintain modeling pipelines in Python, ensuring statistical accuracy and version control in collaboration with data engineering teams
  • Communicate complex findings clearly to technical and non-technical stakeholders through presentations, documentation, and data visualizations that support decision-making
  • Stay current with advances in forecasting, attribution modeling, and statistical methods by engaging in professional development and applied learning
  • Design, lead and innovate the development of advanced statistical and machine learning models to forecast business outcomes such as service activations, digital and retail traffic, and related KPIs
  • Contribute to the design, innovation and refinement of media attribution models including Marketing Mix Modeling and Multi-Touch Attribution to evaluate marketing effectiveness and align attribution insights with forecasting strategies
  • Guide the development and deployment of scalable modeling pipelines in Python, providing oversight to ensure reproducibility, rigor, and operational readiness
  • Mentor and review the work of junior data scientists providing methodological direction, feedback, and quality control
  • Collaborate cross-functionally with marketing, analytics, and data engineering teams to ensure that forecasting and attribution outputs meet business needs and are integrated into decision-making processes
What we offer
What we offer
  • Annual stock grant
  • Employee stock purchase plan
  • 401(k)
  • Access to free, year-round money coaches
  • Annual bonus or periodic sales incentive or bonus
  • Medical, dental and vision insurance
  • Flexible spending account
  • Paid time off and up to 12 paid holidays
  • Paid parental and family leave
  • Family building benefits
  • Fulltime
Read More
Arrow Right