CrawlJobs Logo

Data Architect/Databricks Consultant

genzeon.com Logo

Genzeon

Location Icon

Location:
India , Hyderabad

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

We are seeking a specialized Databricks Architect with deep expertise in cost optimization and migration strategies, particularly focused on transitioning away from Databricks platforms. The ideal candidate will have extensive experience in Spark cluster solutions and a proven track record of reducing Databricks operational costs while architecting successful migration paths to alternative platforms.

Job Responsibility:

  • Conduct comprehensive cost analysis and auditing of existing Databricks deployments across multiple workspaces
  • Develop and implement aggressive cost reduction strategies targeting 30-50% savings through cluster optimization
  • Design and deploy automated cost monitoring solutions with real-time alerts and budget controls
  • Optimize cluster configurations, auto-scaling policies, and job scheduling to minimize compute costs
  • Implement spot instance strategies and preemptible VM usage for non-critical workloads
  • Establish cost allocation frameworks and implement chargeback mechanisms for business unit accountability
  • Create cost governance policies and developer guidelines to prevent cost overruns
  • Analyze and optimize storage costs including Delta Lake table optimization and data lifecycle management
  • Lead strategic initiatives to migrate workloads away from Databricks to cost-effective alternatives
  • Assess existing Databricks implementations and create detailed migration roadmaps to target platforms
  • Design migration architectures for transitioning to open-source Spark on Kubernetes, EMR, or other platforms
  • Develop automated migration tools and frameworks to minimize business disruption
  • Create comprehensive migration strategies including data export, job conversion, and dependency mapping
  • Establish parallel running environments to ensure zero-downtime migrations
  • Lead post-migration validation and performance benchmarking against original Databricks solutions
  • Document lessons learned and create reusable migration playbooks for future projects
  • Design high-performance, cost-optimized Spark cluster architectures outside of Databricks ecosystem
  • Implement custom Spark solutions on Kubernetes, YARN, and standalone cluster managers
  • Optimize Spark job performance through advanced tuning of memory management, serialization, and parallelism
  • Develop custom Spark operators and applications for specialized business use cases
  • Troubleshoot complex Spark performance bottlenecks and implement optimization strategies
  • Create cluster auto-scaling solutions and dynamic resource allocation frameworks
  • Design fault-tolerant Spark architectures with disaster recovery and high availability
  • Implement monitoring and alerting for Spark cluster health and job performance metrics
  • Collaborate with finance teams to develop multi-year cost reduction roadmaps
  • Evaluate and recommend alternative platforms based on cost-benefit analysis
  • Create business cases for migration projects with detailed ROI calculations
  • Establish technical debt reduction strategies related to Databricks dependencies
  • Partner with procurement teams on contract negotiations and vendor management

Requirements:

  • 8+ years of experience in big data architecture with focus on cost optimization
  • 5+ years of hands-on Databricks experience with proven cost reduction achievements
  • Demonstrated experience architecting and executing complete platform migrations from Databricks to alternative solutions with successful outcomes
  • 6+ years of advanced Apache Spark development and cluster management experience
  • Track record of achieving significant cost savings (minimum 40%+) in cloud data platforms
  • Expert knowledge of Databricks pricing models, compute types, and cost drivers
  • Experience with FinOps practices and cloud cost management tools
  • Proven ability to implement automated cost controls and budget management systems
  • Knowledge of alternative platforms and their cost structures (EMR, HDInsight, GCP Dataproc, etc.)
  • Deep expertise in migrating complex data workloads between different Spark platforms
  • Advanced knowledge of Spark internals, catalyst optimizer, and performance tuning
  • Experience with Kubernetes-based Spark deployments and container orchestration
  • Proficiency in infrastructure-as-code for multi-cloud Spark cluster provisioning
  • Strong background in data pipeline migration and ETL/ELT conversion strategies
  • Expert-level proficiency in Scala, Python, and Java for Spark development
  • Advanced SQL skills and experience with multiple database technologies
  • Experience with open-source alternatives to Databricks (Apache Spark, Delta Lake OSS, MLflow OSS)
  • Knowledge of streaming platforms (Kafka, Kinesis, Pulsar) and real-time architectures
  • Proficiency with monitoring tools (Prometheus, Grafana, ELK stack)
  • Bachelor's degree in Computer Science, Engineering, Information Technology, or related field

Nice to have:

  • Databricks certifications combined with experience in competitive platforms
  • Cloud cost management certifications (AWS Cost Optimization, Azure Cost Management)
  • Experience with vendor negotiations and contract optimization
  • Background in building business cases for platform migrations
  • Knowledge of data governance during platform transitions
  • Experience with Apache Iceberg, Hudi, or other open table formats as Delta Lake alternatives
  • Master's degree preferred but not required with equivalent experience

Additional Information:

Job Posted:
December 08, 2025

Icon
Genzeon - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Data Architect/Databricks Consultant

Receptionist

We are looking for a detail-oriented Receptionist to join our team in Miami, Flo...
Location
Location
United States , Miami
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in using a multi-line phone system for managing calls
  • Strong customer service skills with the ability to handle inquiries professionally
  • Experience in data entry with attention to detail and accuracy
  • Ability to communicate effectively through email correspondence
  • Excellent interpersonal skills to interact with staff and visitors
  • Competence in Microsoft Excel, Outlook, and Word for administrative tasks
  • Organizational skills to manage files and maintain office order
  • Capability to schedule appointments and coordinate meetings efficiently
Job Responsibility
Job Responsibility
  • Oversee access to the office, ensuring security and proper protocols are followed
  • Manage the stocking and organization of supplies in the kitchens to maintain efficiency
  • Maintain the cleanliness and orderliness of the office environment to ensure a neat appearance
  • Handle incoming phone calls using a multi-line phone system, providing courteous and efficient service
  • Assist with scheduling appointments and coordinating meetings as needed
  • Perform accurate data entry tasks to support administrative functions
  • Organize and maintain files, ensuring easy accessibility and proper documentation
  • Communicate effectively via email to address inquiries and provide information
  • Execute various ad hoc projects and tasks as assigned to support office operations
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right

Psychiatrist

Astrya Global, a San Diego–based medical staffing agency, is hiring Psychiatrist...
Location
Location
United States , San Bernardino
Salary
Salary:
Not provided
astryaglobal.com Logo
Astrya Global
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Active CA licensure
  • Board certification
  • Active CA DEA
  • New graduates eligible
Job Responsibility
Job Responsibility
  • Evaluate and diagnose mental health disorders
  • Develop and implement treatment plans
  • See 15+ patients per day completing initial and follow up appointments
  • Prescribe and refill medications
  • Collaborate with up to 4 NPs as needed
What we offer
What we offer
  • Malpractice Insurance
  • Weekly pay
  • Full-service credentialing and licensing department
  • Dedicated corporate travel team with airfare, car rental and hotel booking
  • Referral Bonus up to $5,000
  • Fulltime
Read More
Arrow Right

Cook

Boston Pizza Cooks are the engine of the entire restaurant! They are responsible...
Location
Location
Canada , Campbell River
Salary
Salary:
17.85 - 22.00 CAD / Hour
bostonpizza.com Logo
Boston Pizza
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Collaborative
  • Team Player
  • Multi Tasker
  • Self-Starter
  • Detail Oriented
  • Efficient
Job Responsibility
Job Responsibility
  • Creating delicious menu items perfectly every time for every guest
  • Ensuring Guests have an excellent Boston Pizza experience from start to finish
  • Championing a fun and safe work environment
  • Contributing to a thriving workplace culture
  • Works alongside their team members
  • Builds relationships by finding common ground and working cohesively with all team members
  • Committed to championing the 'Heart of Hospitality', Boston Pizza's Brand Standards and Food Safety
  • Dedicated to providing Perfect Food Every Time to our Guests
  • Maintains composure and thrives in a fast pace environment
  • Works with a sense of purpose and always prioritize Guest’s orders
What we offer
What we offer
  • Flex scheduling, days or nights
  • A discount on Canada's best gourmet pizza
  • A company entrenched in community service with opportunities to give back
  • Staff parties, pizza recipe contests etc
  • Work as part of an amazing team
  • BPI Scholarship Awards
  • Flexible scheduling that fits your lifestyle and contributes to your life-work balance
  • Exciting opportunities for career growth within the restaurant or at Boston Pizza head office
  • Build relationships and give back to your local community
  • Boston Pizza Scholarship Program
Read More
Arrow Right

Specialist Education Support Assistant

Are you looking to get into supply or long‑term school-based support work? Are y...
Location
Location
United Kingdom , South Gloucestershire
Salary
Salary:
Not provided
affinityworkforce.com Logo
AFFINITY WORKFORCE
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hold an Enhanced Child DBS on the Update Service (or be willing to apply through AWS)
  • Have the ability to develop, encourage and promote learning
  • Have a passion for making a positive difference to children’s lives
  • Bring experience working within a UK school environment
  • Hold a valid safeguarding certificate
  • Demonstrate strong communication skills with children, staff and parents
Job Responsibility
Job Responsibility
  • Delivering targeted interventions
  • Supporting behaviour and pastoral needs
  • Helping pupils overcome barriers to learning
  • Providing lunchtime support
What we offer
What we offer
  • Competitive daily rate
  • Opportunities to gain valuable, varied school experience
  • Access to exclusive rewards hub offering training, discounts & cashback
  • Generous referral bonus
  • Flexible work schedule
  • Temp-to-perm opportunities
  • Weekly pay
Read More
Arrow Right

Psychiatrist

Astrya Global, a San Diego–based medical staffing agency, is looking for a psych...
Location
Location
United States , San Diego
Salary
Salary:
250.00 - 260.00 USD / Hour
astryaglobal.com Logo
Astrya Global
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • M.D. or D.O. with active, unrestricted California medical license
  • Active California DEA
  • 2 years of experience preferred
  • Credentialed with Optum and PAVE preferred
Job Responsibility
Job Responsibility
  • Conduct comprehensive psychiatric evaluations including DSM-5-TR diagnostic assessments and clinical formulation
  • Develop and oversee individualized treatment plans in collaboration with patients and multidisciplinary care teams
  • Prescribe, monitor, and adjust psychotropic medications while evaluating treatment response and side effects
  • Perform risk assessments for suicidality, homicidality, and danger to self or others, and implement appropriate safety planning
  • Coordinate care with psychologists, therapists, primary care providers, case managers, and community agencies to support integrated treatment
  • Provide crisis evaluation, stabilization, and psychiatric consultation as needed
  • Facilitate referrals to appropriate levels of care including inpatient, partial hospitalization (PHP), intensive outpatient (IOP), and residential treatment programs
  • Maintain accurate and timely clinical documentation in accordance with regulatory and organizational standards
What we offer
What we offer
  • Malpractice Insurance
  • Weekly pay
  • Full-service credentialing and licensing department
  • Dedicated corporate travel team with airfare, car rental and hotel booking
  • Referral Bonus up to $5,000
  • Fulltime
Read More
Arrow Right

Power Apps Developer

The Power Apps Developer is responsible for designing, developing, and maintaini...
Location
Location
United States , Riviera Beach
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Information Technology, Computer Science, Business Systems, or related field OR equivalent combination of certifications and relevant experience in lieu of degree
  • Minimum 4–6 years of experience developing business applications or enterprise solutions
  • Minimum 2+ years of hands-on Power Platform development experience
  • Strong experience with: Power Apps (Canvas and/or Model-Driven)
  • Power Automate workflows
  • SharePoint and Microsoft 365 integration
  • Data modeling and structured data design
  • Ability to translate business requirements into technical solutions
Job Responsibility
Job Responsibility
  • Design, develop, and deploy custom business applications using Microsoft Power Apps (Canvas and Model-Driven apps)
  • Build solutions that support operational workflows, approvals, data collection, and reporting needs
  • Create responsive applications that function across desktop, tablet, and mobile platforms
  • Ensure applications follow enterprise architecture and security standards
  • Develop automated workflows using Power Automate
  • Integrate applications with enterprise platforms such as: SharePoint
  • Microsoft 365 services
  • Dataverse
  • SQL databases
  • Oracle or other enterprise systems (as applicable)
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • company 401(k) plan
Read More
Arrow Right

Cook

Looking for Line Cooks to join our team here at Boston Pizza. We strive to creat...
Location
Location
Canada , Campbell River
Salary
Salary:
17.85 - 22.00 CAD / Hour
bostonpizza.com Logo
Boston Pizza
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Collaborative
  • Team Player
  • Multi Tasker
  • Self-Starter
  • Detail Oriented
  • Efficient
Job Responsibility
Job Responsibility
  • Works alongside their team members
  • Committed to championing the 'Heart of Hospitality', Boston Pizza's Brand Standards and Food Safety
  • Dedicated to providing Perfect Food Every Time to our Guests
  • Maintains composure and thrives in a fast pace environment
  • Works with a sense of purpose and always prioritize Guest’s orders
What we offer
What we offer
  • Flex scheduling, days or nights
  • A discount on Canada's best gourmet pizza
  • A company entrenched in community service with opportunities to give back
  • Staff parties, pizza recipe contests etc
  • Work as part of an amazing team
  • BPI Scholarship Awards
  • Flexible scheduling that fits your lifestyle and contributes to your life-work balance
  • Exciting opportunities for career growth
  • Management Training Programs, Leadership Development conferences and access to dynamic mentorship programs
  • Fantastic reward and recognition programs
Read More
Arrow Right

Principal Civil Engineer

BSP Consulting has an immediate vacancy for a Principal Civil Engineer Nottingha...
Location
Location
United Kingdom , Nottingham
Salary
Salary:
Not provided
bsp-consulting.co.uk Logo
BSP Consulting
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Specific experience in the use of AutoCAD for technical drawing
  • A strong background of Degree level Civil Engineering and a logical analytical approach to problem solving
  • Civil3D or PDS, InfoDrainage or MicroDrainage software experience
  • Knowledge of the construction industry and basic health and safety
  • 10+ years of relevant technical experience
  • Strong client‑care skills
  • Effective communicator in written English and verbal communication
  • Solid understanding of the construction industry, current design standards, and health and safety requirements
  • Experience with InfoDrainage, Civil 3D for design, and AutoCAD for technical drawing
  • A strong background of English, Mathematics and a logical analytical approach to problem solving
Job Responsibility
Job Responsibility
  • Management, design, and drafting of a wide range of civil engineering schemes
  • Section 278 / Section 38 highway works for adoption
  • Section 104 / Section 185 drainage works for adoption
  • Private levels and private drainage design
  • External works design
  • Cut and fill analysis and earthworks design
  • Sustainable drainage systems (SuDS) design
  • Liaison with local authorities, water companies, the Environment Agency, and project stakeholders
  • Design and drafting of civil engineering works across multiple sectors
  • Deputise for the team leader, supporting the planning and organisation of tasks, people, and resources
What we offer
What we offer
  • Mentorship towards ICE Chartership can be provided where required
  • Fulltime
Read More
Arrow Right
Remote jobs Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Canada Jobs Australia Jobs Germany Jobs Ireland Jobs Poland Jobs
Company
Blog Salaries Tools About Us FAQ Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
CrawlJobs Group
ITFlashcards AllDevBlogs
© 2026 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy