CrawlJobs Logo

Lead Databricks Developer

votredircom.fr Logo

Wissen

Location Icon

Location:
India , Bangalore South

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

We are seeking an experienced Lead Databricks Developer with deep expertise in Spark, PySpark, and SQL to design, build, and optimize large-scale data processing solutions. The ideal candidate will have strong hands-on experience in cloud-based data environments (preferably AWS) and will be capable of leading end-to-end data engineering initiatives. This role involves collaborating with cross‑functional teams, ensuring high data quality, and building robust pipelines to support business analytics and data-driven decision-making.

Job Responsibility:

  • Design, develop, and maintain scalable data processing applications using Python, PySpark, and Apache Spark
  • Work in cloud-based environments—AWS preferred, with flexibility to work on Azure Databricks
  • Collaborate closely with data engineers, data scientists, analysts, and business stakeholders to understand requirements and deliver effective solutions
  • Ensure data integrity, reliability, and performance across data pipelines and workflows
  • Write clean, well-structured, and maintainable code following industry best practices
  • Perform exploratory data analysis and implement data validation frameworks
  • Monitor and troubleshoot performance bottlenecks in Spark jobs and data workflows
  • Implement and manage CI/CD pipelines for automated build, test, and deployment of data pipelines
  • Optimize data pipelines for cost, performance, and scalability in cloud environments

Requirements:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
  • 8–12 years of experience in data engineering with a strong focus on Databricks and Spark/PySpark
  • Strong proficiency in SQL and experience working with relational databases
  • Advanced hands‑on experience in Python for data engineering
  • Solid understanding of cloud platforms (AWS preferred
  • Azure Databricks acceptable)
  • Experience with Git or other version control systems
  • Strong analytical thinking, debugging, and problem‑solving skills
  • Excellent communication and cross‑team collaboration abilities

Nice to have:

  • Experience with Delta Lake, Unity Catalog, or advanced Databricks features
  • Exposure to data warehousing concepts and dimensional modeling
  • Familiarity with Airflow, Glue, or other workflow orchestration tools
  • Knowledge of data security, governance, and compliance best practices
  • Understanding of DevOps, containerization (Docker), or Kubernetes
  • Experience integrating Databricks with BI tools (Power BI, Tableau, etc.)
  • Knowledge of machine learning workflows or MLOps within Databricks is a plus

Additional Information:

Job Posted:
March 05, 2026

Employment Type:
Fulltime
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Lead Databricks Developer

Lead Power BI Developer

Lead consultant specializing in Power BI with a focus on data. Must have advance...
Location
Location
Philippines
Salary
Salary:
Not provided
3cloudsolutions.com Logo
3Cloud
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in computer, engineering, data science or a related field
  • or equivalent work experience
  • Experience managing / leading a team
  • 8+ years working with data and business intelligence solutions on the Microsoft tool stack for Business Intelligence (SSRS, Power BI, PowerPivot, PerformancePoint) or other vendor BI tools (Business Objects, MicroStrategy, Qlik, SAP)
  • Working knowledge of TSQL, data modeling, reporting using Power BI or other Microsoft tools
  • Experience creating complete Power BI report solutions with linked content, drilldowns, and nested parameters
  • Dynamic measure designs using advanced DAX techniques
  • Knowledge of data mashups and transformations using Power Query/M language
  • Utilize Databricks platform to develop, implement, and optimize data pipelines and workflows
  • Ability to report visual choices and visual interactions
Job Responsibility
Job Responsibility
  • Conduct customer discovery meetings to determine reporting requirements
  • Understand and work with customer on Power BI Licensing
  • Create well performing star schema data models for use with Power BI
  • Creating Power BI solutions for customers including importing data, building reports and dashboards
  • Guide customers on deploying Power BI solutions to the Power BI service (Gateways, DirectQuery, DataFlows, Data Updates)
  • Fulltime
Read More
Arrow Right

Lead Veeva/Salesforce CRM Developer

The Specialist Software Engineer collaborates with Salesforce/Veeva CRM Admins, ...
Location
Location
United States
Salary
Salary:
137457.00 - 171183.00 USD / Year
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Doctorate degree
  • OR Master’s degree and 2 years of computer science, IT, or related field
  • OR Bachelor’s degree and 4 years of computer science, IT, or related field
  • OR Associate’s degree and 8 years of computer science, IT, or related field
  • OR High school diploma / GED and 10 years of computer science, IT, or related field
  • 5+ years of experience as a Salesforce CRM Developer
  • In-depth knowledge of Salesforce development, including experience working with MuleSoft or Databricks integrations
  • Hands-on experience with the Salesforce security model, including OWD, sharing rules, profiles, and permissions
  • Experience with Data Loader, data migration tools, and CI/CD tools
  • Salesforce Developer 2 (required)
Job Responsibility
Job Responsibility
  • Leads large CRM migration to ensure configurations, customizations, and data migration are successfully implemented in the new CRM
  • Drives development in Salesforce/Veeva CRM to build and configure new and existing features
  • Works closely with Product Owners and cross-functional Development Teams to plan and schedule releases, providing guidance on release dependencies, technical constraints, and risks
  • Supports the development teams in automating build, test, and deployment processes, enabling continuous integration and continuous delivery in the Salesforce/Veeva ecosystem
  • Provides ongoing support and troubleshooting for post-release issues, ensuring timely resolution of any problems that arise
  • Maintains and optimizes existing configurations and customizations by limiting or eliminating technical debt to streamline the system
  • Partners with Development Teams to ensure agile software delivery and DevOps practices are applied to move most effectively within our CI/CD pipeline
  • Researches Veeva/Salesforce capabilities as needed to suit business requirements and provides gap analysis
  • Works closely with other developers in the team, including Product Owners, Business Analysts, Salesforce Admins, QA Analysts, and Scrum Masters, across cross-functional teams
  • Develops innovative solutions using generative AI technologies
What we offer
What we offer
  • A comprehensive employee benefits package, including a Retirement and Savings Plan with generous company contributions, group medical, dental and vision coverage, life and disability insurance, and flexible spending accounts
  • A discretionary annual bonus program, or for field sales representatives, a sales-based incentive plan
  • Stock-based long-term incentives
  • Award-winning time-off plans
  • Flexible work models, including remote and hybrid work arrangements, where possible
  • Fulltime
Read More
Arrow Right

Sr. SQL Database Developer

We are seeking a highly skilled Sr. Database Developer with 4+ years of experien...
Location
Location
Salary
Salary:
Not provided
honorvettech.com Logo
HonorVet Technologies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of hands-on experience with Microsoft SQL Server and related database technologies
  • Strong expertise in designing and developing high-performance databases supporting large-scale concurrent transactions
  • Advanced knowledge of relational databases, logical/physical data modeling, and database architecture
  • Proficiency in SQL, stored procedures, triggers, and query optimization for large and complex datasets
  • Experience with data warehousing, OLTP, OLAP, operational data stores, and decision support systems
  • Hands-on experience with Snowflake, SQL Server, and Databricks
  • Skilled in developing SSRS reports with advanced reporting features and optimizations
  • Strong understanding of Agile development methodologies
  • Excellent communication and collaboration skills
  • Bachelor's Degree in Computer Science, Information Systems, or related field, or equivalent combination of education and experience
Job Responsibility
Job Responsibility
  • Collaborate with application, product, QA, and PMO teams to deliver database solutions that align with business requirements
  • Translate business requirements into technical specifications and partner with developers to deliver scalable results
  • Design, develop, and implement complex database systems, tools, and applications with a focus on performance and scalability
  • Write optimized queries, stored procedures, triggers, and leverage advanced SQL coding techniques
  • Utilize data modeling tools to design logical and physical database models
  • Lead database architecture and design initiatives, including performance optimization and data loading strategies from Snowflake to SQL Server via Databricks
  • Provide technical leadership, share best practices, and mentor team members
  • Contribute to project planning and provide accurate development estimates
  • Develop and optimize SSRS reports, including sub-reports, parameterized reports, linked reports, charts, and datasets
  • Configure and optimize SSRS report performance, including grouping and page-break functionality
Read More
Arrow Right

Lead Data Engineer

Alimentation Couche-Tard Inc., (ACT) is a global Fortune 200 company. A leader i...
Location
Location
India , Gurugram
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or master’s degree in computer science, Engineering, or related field
  • 7-9 years of data engineering experience with strong hands-on delivery using ADF, SQL, Python, Databricks, and Spark
  • Experience designing data pipelines, warehouse models, and processing frameworks using Snowflake or Azure Synapse
  • Proficient with CI/CD tools (Azure DevOps, GitHub) and observability practices
  • Solid grasp of data governance, metadata tagging, and role-based access control
  • Proven ability to mentor and grow engineers in a matrixed or global environment
  • Strong verbal and written communication skills, with the ability to operate cross-functionally
  • Certifications in Azure, Databricks, or Snowflake are a plus
  • Strong Knowledge of Data Engineering concepts (Data pipelines creation, Data Warehousing, Data Marts/Cubes, Data Reconciliation and Audit, Data Management)
  • Working Knowledge of Dev-Ops processes (CI/CD), Git/Jenkins version control tool, Master Data Management (MDM) and Data Quality tools
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable pipelines across ADF, Databricks, Snowflake, and related platforms
  • Lead the technical execution of non-domain specific initiatives (e.g. reusable dimensions, TLOG standardization, enablement pipelines)
  • Architect data models and re-usable layers consumed by multiple downstream pods
  • Guide platform-wide patterns like parameterization, CI/CD pipelines, pipeline recovery, and auditability frameworks
  • Mentoring and coaching team
  • Partner with product and platform leaders to ensure engineering consistency and delivery excellence
  • Act as an L3 escalation point for operational data issues impacting foundational pipelines
  • Own engineering best practices, sprint planning, and quality across the Enablement pod
  • Contribute to platform discussions and architectural decisions across regions
  • Fulltime
Read More
Arrow Right

Lead Data Engineer

As a Lead Data Engineer at Rearc, you'll play a pivotal role in establishing and...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
rearc.io Logo
Rearc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in data engineering, data architecture, or related fields
  • Extensive experience in writing and testing Java and/or Python
  • Proven experience with data pipeline orchestration using platforms such as Airflow, Databricks, DBT or AWS Glue
  • Hands-on experience with data analysis tools and libraries like Pyspark, NumPy, Pandas, or Dask
  • Proficiency with Spark and Databricks is highly desirable
  • Proven track record of leading complex data engineering projects, including designing and implementing scalable data solutions
  • Hands-on experience with ETL processes, data warehousing, and data modeling tools
  • In-depth knowledge of data integration tools and best practices
  • Strong understanding of cloud-based data services and technologies (e.g., AWS Redshift, Azure Synapse Analytics, Google BigQuery)
  • Strong strategic and analytical skills
Job Responsibility
Job Responsibility
  • Understand Requirements and Challenges: Collaborate with stakeholders to deeply understand their data requirements and challenges
  • Implement with a DataOps Mindset: Embrace a DataOps mindset and utilize modern data engineering tools and frameworks, such as Apache Airflow, Apache Spark, or similar, to build scalable and efficient data pipelines and architectures
  • Lead Data Engineering Projects: Take the lead in managing and executing data engineering projects, providing technical guidance and oversight to ensure successful project delivery
  • Mentor Data Engineers: Share your extensive knowledge and experience in data engineering with junior team members, guiding and mentoring them to foster their growth and development in the field
  • Promote Knowledge Sharing: Contribute to our knowledge base by writing technical blogs and articles, promoting best practices in data engineering, and contributing to a culture of continuous learning and innovation
Read More
Arrow Right

Lead Data Engineer

Lead Data Engineer to serve as both a technical leader and people coach for our ...
Location
Location
India , Gurugram
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or master’s degree in computer science, Engineering, or related field
  • 8-10 years of data engineering experience with strong hands-on delivery using ADF, SQL, Python, Databricks, and Spark
  • Experience designing data pipelines, warehouse models, and processing frameworks using Snowflake or Azure Synapse
  • Proficient with CI/CD tools (Azure DevOps, GitHub) and observability practices
  • Solid grasp of data governance, metadata tagging, and role-based access control
  • Proven ability to mentor and grow engineers in a matrixed or global environment
  • Strong verbal and written communication skills, with the ability to operate cross-functionally
  • Strong Knowledge of Data Engineering concepts (Data pipelines creation, Data Warehousing, Data Marts/Cubes, Data Reconciliation and Audit, Data Management)
  • Working Knowledge of Dev-Ops processes (CI/CD), Git/Jenkins version control tool, Master Data Management (MDM) and Data Quality tools
  • Strong Experience in ETL/ELT development, QA and operation/support process (RCA of production issues, Code/Data Fix Strategy, Monitoring and maintenance)
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable pipelines across ADF, Databricks, Snowflake, and related platforms
  • Lead the technical execution of non-domain specific initiatives (e.g. reusable dimensions, TLOG standardization, enablement pipelines)
  • Architect data models and re-usable layers consumed by multiple downstream pods
  • Guide platform-wide patterns like parameterization, CI/CD pipelines, pipeline recovery, and auditability frameworks
  • Mentoring and coaching team
  • Partner with product and platform leaders to ensure engineering consistency and delivery excellence
  • Act as an L3 escalation point for operational data issues impacting foundational pipelines
  • Own engineering best practices, sprint planning, and quality across the Enablement pod
  • Contribute to platform discussions and architectural decisions across regions
  • Fulltime
Read More
Arrow Right

Lead Data Engineer

As a Lead Data Engineer at Rearc, you'll play a pivotal role in establishing and...
Location
Location
United States
Salary
Salary:
Not provided
rearc.io Logo
Rearc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in data engineering, data architecture, or related technical fields
  • Proven ability to design, build, and optimize large-scale data ecosystems
  • Strong track record of leading complex data engineering initiatives
  • Deep hands-on expertise in ETL/ELT design, data warehousing, and data modeling
  • Extensive experience with data integration frameworks and best practices
  • Advanced knowledge of cloud-based data services and architectures (AWS Redshift, Azure Synapse Analytics, Google BigQuery, or equivalent)
  • Strong strategic and analytical thinking
  • Proficiency with modern data engineering frameworks (Databricks, Spark, lakehouse technologies like Delta Lake)
  • Exceptional communication and interpersonal skills
Job Responsibility
Job Responsibility
  • Engage deeply with stakeholders to understand data needs, business challenges, and technical constraints
  • Translate stakeholder needs into scalable, high-quality data solutions
  • Implement with a DataOps mindset using tools like Apache Airflow, Databricks/Spark, Kafka
  • Build reliable, automated, and efficient data pipelines and architectures
  • Lead and execute complex projects
  • Provide technical direction and set engineering standards
  • Ensure alignment with customer goals and company principles
  • Mentor and develop data engineers
  • Promote knowledge sharing and thought leadership
  • Contribute to internal and external content
What we offer
What we offer
  • Comprehensive health benefits
  • Generous time away and flexible PTO
  • Maternity and paternity leave
  • Access to educational resources with reimbursement for continued learning
  • 401(k) plan with company contribution
Read More
Arrow Right

AI Developer Intern

AI Developer Intern role at Hewlett Packard Enterprise focusing on developing AI...
Location
Location
United States , Houston
Salary
Salary:
40.00 - 46.00 USD / Hour
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Currently pursuing a Master's degree in Computer Science, Data Science, Engineering, Applied Mathematics or a related technical field
  • Strong coding proficiency in Python and SQL
  • Deep understanding of algorithmic thinking, data structures, and statistical modeling
  • Familiarity with machine learning frameworks (e.g., TensorFlow, PyTorch)
  • Familiarity with large language models (LLMs), prompt engineering, fine-tuning techniques, and AI evaluation methodologies
Job Responsibility
Job Responsibility
  • Lead and contribute to the development of AI-driven analytics solutions focused on business performance and sales optimization
  • Work extensively with Databricks for scalable data engineering and Azure cloud services for model deployment, orchestration, and monitoring
  • Translate business requirements into scalable and performant technical solutions
  • Design, code, test, and assure the quality of complex AI-powered product features
  • Partner with a highly motivated and talented set of colleagues
  • Collaborate across teams and time zones, demonstrating flexibility and accountability
What we offer
What we offer
  • Health & Wellbeing benefits
  • Personal & Professional Development programs
  • Unconditional Inclusion environment
  • Fulltime
Read More
Arrow Right