CrawlJobs Logo
Citi Logo Citi · IT - Software Development

Databricks AWS Engineer - Vice President

India, Pune · Job Posted June 15, 2026
Apply Position
Job Link Share

Job Description

We are looking for a highly skilled Senior Databricks Engineer to contribute to the engineering, modernization, and continuous evolution of data processing platform on Databricks on AWS. While supporting the transition from the legacy Cloudera Hadoop platform to Databricks on AWS, this role will continue to play a key part in enhancing performance, simplifying pipelines, and delivering new capabilities on the Databricks platform over the long term. The ideal candidate is a strong hands‑on Spark engineer with solid design experience, capable of contributing to architectural decisions while leading complex implementation and optimization efforts.

Job Responsibility

  • Refactor and modernize existing Spark pipelines to Databricks native architectures
  • Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
  • Enhance and extend existing processing logic using optimized Spark (JavaSpark / PySpark) on Databricks
  • Build and optimize solutions using Databricks features, including Delta Lake, Databricks Workflows for orchestration and Auto scaling and job clusters
  • Contribute to low and mid level architecture and design
  • Translate high level architecture into detailed technical designs
  • Define data models, pipeline patterns, and reusable components
  • Ensure solutions are scalable, maintainable, and production ready
  • Analyze, improve Spark job performance and simplify complex or over engineered pipelines into standardized, efficient patterns
  • Follow and contribute to Databricks and Spark engineering standards
  • Write clean, modular, and testable code
  • Contribute to shared frameworks, reusable libraries, and quality standards
  • Work closely with senior architects, platform teams, and DevOps engineers
  • Provide technical inputs, troubleshooting support, and implementation guidance
  • Participate in design discussions and technical decision making
  • Develop unit, integration, and data validation tests
  • Support production releases and post deployment validation

Requirements

  • 10+ years in data engineering or distributed systems
  • Strong expertise in Apache Spark (JavaSpark / PySpark), Databricks on AWS, and Delta Lake
  • Experience with AWS services and large‑scale distributed data processing
  • Experience modernizing or refactoring legacy data platforms into cloud‑based architectures
  • Strong background in Spark performance tuning and large‑scale batch optimization
  • Ability to translate architecture into implementable designs
  • Understanding of data modeling and pipeline orchestration patterns
  • Strong problem‑solving mindset for complex distributed systems
  • Comfortable working in time‑bound, high‑impact environments
  • Proactive, accountable, and collaborative
  • Clear communication skills across global teams
  • Bachelor’s degree/University degree or equivalent experience
Citi - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Databricks AWS Engineer - Vice President

8 matching positions

Data Engineer (Big Data, Cloud - AWS, Databricks) - Assistant Vice President

Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Scala, Spark/Pyspark is must, Hadoop ( BIG Data ), + AWS,Databricks
  • 8 to 11 years’ experience implementing data-intensive solutions using agile methodologies
  • Experience of relational databases and using SQL for data querying, transformation and manipulation
  • Experience of modelling data for analytical consumers
  • Ability to automate and streamline the build, test and deployment of data pipelines
  • Experience in cloud native technologies and patterns
  • A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
  • Excellent communication and problem-solving skills
  • An inclination to mentor
  • an ability to lead and deliver medium sized components independently
Job Responsibility
Job Responsibility
  • Developing and supporting scalable, extensible, and highly available data solutions
  • Deliver on critical business priorities while ensuring alignment with the wider architectural vision
  • Identify and help address potential risks in the data supply chain
  • Follow and contribute to technical standards
  • Design and develop analytical data models
  • Fulltime
Read More
Arrow Right

Principal Data Genai Platform Engineer - Senior Vice President

Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 12+ years of relevant experience in enterprise application development, data engineering, or AI platform engineering, with a strong track record of leadership in regulated environments
  • 8+ years of experience leading multi-team Agile organizations (20+ engineers), including managing distributed and hybrid AI-assisted teams
  • Advanced expertise in Python, PySpark, and Databricks ecosystem for large-scale data processing and ELT/ETL pipelines
  • Proven experience architecting and implementing enterprise AI/GenAI platforms, including agentic AI frameworks, LLM integrations, and prompt engineering
  • Hands-on experience with AI-assisted development tools such as Devin.AI and GitHub Copilot and integrating them into engineering workflows
  • Strong experience with microservices architecture, APIs, and cloud-native deployment (Kubernetes/OpenShift)
  • Strong experience with event-driven architectures and streaming platforms (Kafka)
  • Deep understanding of data architecture, data mesh, data federation, and regulatory data requirements
  • Exceptional leadership, communication, stakeholder management, and decision-making capabilities
  • Experience with cloud platforms (AWS, Azure, GCP, Databricks) and modern data ecosystems
Job Responsibility
Job Responsibility
  • Lead multiple agile scrum teams comprising ~15+ engineers, including hybrid teams of human engineers and AI-assisted development (Devin.AI, Copilot), ensuring delivery excellence and alignment with business priorities
  • Define and execute the enterprise strategy for Python engineering, AI agent platforms, and full-stack data applications, aligned with Retail and Wealth Risk objectives
  • Serve as the senior architect and technical authority for enterprise-scale AI agents, data engineering pipelines, and microservices-based applications, ensuring scalability, resilience, and security
  • Drive the adoption and operationalization of AI Product Development Lifecycle (AI PDLC), including model governance, evaluation, deployment, monitoring, and compliance with Model Risk Management (MRM)
  • Lead development of high-volume data pipelines and data federation layers using PySpark, Databricks, Kafka, and Data Mesh architecture to support regulatory reporting (CCAR, FDIC) and risk analytics
  • Architect and oversee GenAI agent ecosystems using LLMs (Google ADK, Gemini/Flash), implementing Human-in-the-Loop (HITL) frameworks to ensure explainability, auditability, and compliance
  • Drive AI-augmented software development lifecycle, integrating tools such as Devin.AI, GitHub Copilot, and MCP platforms through advanced prompt engineering and governance guardrails
  • Lead microservices and cloud-native architecture using FastAPI/Spring Boot, Kubernetes/OpenShift, and CI/CD pipelines, ensuring high availability and performance
  • Drive engineering efficiency and standardization by reusing and repurposing enterprise-level frameworks, platforms, and tools, reducing duplication and accelerating delivery across teams
  • Ensure all engineering solutions incorporate data governance and non-functional requirements, including Data Quality (DQ), data lineage, data tracing, and auditability, aligned with enterprise governance processes and regulatory expectations
  • Fulltime
Read More
Arrow Right

Vice President - Technology (Platform Engineer)

Our client's technology team is responsible for creating and continuously improv...
Location
Location
United States , New York
Salary
Salary:
175000.00 - 215000.00 USD / Year
rennerbrown.com Logo
Renner Brown
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in computer science, Computer Engineering, or related field
  • 8+ years in infrastructure engineering, cloud platform engineering, or data engineering
  • Azure expertise: Azure AI Foundry, Azure Data Factory, Azure Databricks, AKS, Azure API Management, Azure Key Vault, Azure Entra ID
  • Strong Python skills: backend services, REST APIs (FastAPI or Flask), and automation scripting
  • PowerShell for infrastructure tasks
  • Infrastructure-as-Code: Terraform and/or Bicep
  • container orchestration with Docker and Kubernetes
  • Experience integrating LLM APIs (Anthropic Claude, Azure OpenAI) in production including token cost management and observability
  • RAG pipeline experience: vector search (Azure AI Search or pgvector), document processing, and retrieval patterns
  • Familiarity with LLM application frameworks such as LangChain or Semantic Kernel
Job Responsibility
Job Responsibility
  • AI Platform & Developer Infrastructure: Design, build, and operate the firm's AI platform, enabling developers to build and deploy Python-based AI applications
  • Implement and manage Azure AI Foundry environments: model deployments, AI hubs, project workspaces, and access controls
  • Integrate and operationalize third-party AI APIs (Anthropic Claude API, Azure OpenAI) with secure access patterns, API gateway controls, rate limiting, and cost monitoring
  • Build internal developer tooling and SDK scaffolding to accelerate AI application development across the firm
  • Data Infrastructure & Pipelines: Build and maintain data pipelines using Azure Data Factory and Azure Databricks to serve AI application data needs
  • Implement vector search and document retrieval infrastructure (Azure AI Search) to support RAG-based applications
  • Manage structured and unstructured data stores including Azure Data Lake, Azure SQL, and Cosmos DB
  • Cloud Infrastructure & DevOps: Provision and maintain secure, scalable infrastructure on Azure (primary) and AWS using Infrastructure-as-Code (Terraform or Bicep)
  • Build and maintain CI/CD pipelines for AI application deployment via Azure DevOps or GitHub Actions
  • Manage containerized workloads using Docker and Kubernetes (AKS) for AI application hosting and API services
  • Fulltime
Read More
Arrow Right

Senior Data Software Engineer (Python & PySpark) - Vice President

The Senior Data Software Engineer is a senior level position responsible for est...
Location
Location
Singapore , Singapore
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
  • 7+ years of experience in data engineering, with a strong focus on Python and big data technologies
  • Proven expertise in designing and implementing large-scale data processing solutions using PySpark
  • Extensive experience with distributed computing frameworks like Apache Spark
  • Strong understanding of data warehousing concepts, dimensional modeling, and ETL/ELT principles
  • Proficiency in SQL and experience with various relational and NoSQL databases
  • Experience with cloud platforms (AWS, Azure, GCP) and their data services (e.g., S3, ADLS, Google Cloud Storage, Redshift, Snowflake, BigQuery, Databricks)
  • Familiarity with workflow orchestration tools (e.g., Apache Airflow, Azure Data Factory, AWS Step Functions)
  • Experience with version control systems (e.g., Git)
  • Excellent problem-solving, analytical, and communication skills.
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
  • Fulltime
Read More
Arrow Right

Digital Software Engineer Senior Analyst - Assistant Vice President

Location
Location
United States , Tampa
Salary
Salary:
96960.00 - 145440.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
August 31, 2026
Flip Icon
Requirements
Requirements
  • 5-8 years experience with PL/SQL, Oracle and integration with Java applications
  • Exposure to architecture experience in building horizontally scalable, highly available, highly resilient, and low latency applications
  • Experience with data engineering techniques like building data lakes and data warehouses, data mesh, data pipelines, ETL vs. ELT
  • Exposure to Cloud infrastructure both on-premise and public cloud (i.e., OpenShift, AWS, Snowflake, Databricks)
  • Exposure to Continuous Integration and Continuous Delivery (CI/CD) pipelines, either on-premise or public cloud (i.e., Tekton, Harness, CircleCI, Cloudbees Jenkins, etc.)
  • Exposure to API Management tools
  • Exposure to Infrastructure as Code tools (i.e., IntelliJ, Cloudformation, PiSpark, etc.)
  • Experience mentoring junior developers
Job Responsibility
Job Responsibility
  • Accountable for executing and driving mid-size feature application design and development efforts to completion, serving as a development lead on medium-scale projects and supporting the execution of larger efforts
  • Proficient at operating with considerable autonomy and discretion as you will significantly influence the way an application is designed and developed by providing subject specific expertise and an advanced level of understanding of application programming principles
  • Sought after due to ability to analyze and troubleshoot coding, application performance and design challenges
  • Capable of research in root cause of development and performance concerns as well as the resolution of defects
  • Have a deep understanding of the technical requirements for the solutions being built
  • Understand engineering needs including those required to build, maintain, and operate the system through all phases of its life
  • Proficient in information modeling, data structures and algorithms
  • Understand maintenance characteristics, runtime properties and dependencies that exist in support of your system’s software
  • Demonstrate an advanced understanding of supported main system flows and possess a comprehensive understanding of how the system and others collectively integrate to contribute towards achieving business objectives
  • Participate in design discussions as a Development Lead and as such will play the part of a key decision maker in driving design decisions
What we offer
What we offer
  • medical, dental & vision coverage
  • 401(k)
  • life, accident, and disability insurance
  • wellness programs
  • planned time off (vacation)
  • unplanned time off (sick leave)
  • paid holidays
  • Fulltime
Read More
Arrow Right

Senior Databricks & Apache Spark Developer - Vice President

We are looking for a highly skilled Senior Databricks Engineer to contribute to ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years in data engineering or distributed systems
  • Strong expertise in Apache Spark (JavaSpark / PySpark), Databricks on AWS, and Delta Lake
  • Experience with AWS services and large‑scale distributed data processing
  • Experience modernizing or refactoring legacy data platforms into cloud‑based architectures
  • Strong background in Spark performance tuning and large‑scale batch optimization
  • Ability to translate architecture into implementable designs
  • Understanding of data modeling and pipeline orchestration patterns
  • Strong problem‑solving mindset for complex distributed systems
  • Comfortable working in time‑bound, high‑impact environments
  • Proactive, accountable, and collaborative
Job Responsibility
Job Responsibility
  • Platform Engineering & Modernization: Refactor and modernize existing Spark pipelines to Databricks native architectures
  • Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns
  • Enhance and extend existing processing logic using optimized Spark (JavaSpark / PySpark) on Databricks
  • Databricks Native Development: Build and optimize solutions using Databricks features, including Delta Lake, Databricks Workflows for orchestration and Auto scaling and job clusters
  • Design & Solution Engineering: Contribute to low and mid level architecture and design
  • Translate high level architecture into detailed technical designs
  • Define data models, pipeline patterns, and reusable components
  • Ensure solutions are scalable, maintainable, and production ready
  • Performance Optimization & Simplification: Analyze, improve Spark job performance and simplify complex or over engineered pipelines into standardized, efficient patterns
  • Engineering Standards & Best Practices: Follow and contribute to Databricks and Spark engineering standards
  • Fulltime
Read More
Arrow Right

Vice President, Big Data Scala Engineer

We are seeking an experienced and highly skilled Vice President, Big Data Scala ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field
  • 12+ years of progressive experience in software development, with at least 5+ years focusing on big data technologies
  • 3+ years of experience in a leadership or senior architectural role
  • Extensive hands-on experience with Scala for big data processing
  • Demonstrated expertise with Apache Spark (Spark Core, Spark SQL, Spark Streaming)
  • Strong experience with distributed systems and big data ecosystems (e.g., Hadoop, Kafka, Cassandra, HBase, Delta Lake, Snowflake, Databricks)
  • Proficiency with cloud platforms (AWS, Azure, GCP) and their big data services (e.g., EMR, Redshift, Glue, DataProc, BigQuery)
  • Experience with containerization technologies (Docker, Kubernetes) and CI/CD pipelines
  • Solid understanding of data warehousing concepts, ETL/ELT processes, and data modeling
  • Familiarity with functional programming paradigms in Scala
Job Responsibility
Job Responsibility
  • Lead the architecture, design, and development of high-performance, scalable, and reliable big data processing systems using Scala and Apache Spark
  • Drive technical vision and strategy for big data initiatives
  • Evaluate and recommend new technologies and tools
  • Design, develop, and optimize data pipelines for ingestion, transformation, and storage of massive datasets
  • Implement robust and efficient data processing jobs using Scala and Spark (batch and streaming)
  • Ensure data quality, integrity, and security
  • Promote and enforce best practices in coding, testing, and deployment
  • Mentor and guide a team of talented big data engineers
  • Conduct code reviews, provide constructive feedback
  • Participate in the recruitment and hiring
What we offer
What we offer
  • Opportunity to work on cutting-edge big data technologies and impactful projects
  • A collaborative and innovative work environment
  • Competitive compensation and benefits package
  • Opportunities for professional growth and career advancement
  • Fulltime
Read More
Arrow Right

Vice President - Technology (Data & AI Infrastructure Engineer)

Our client's technology team is responsible for creating and continuously improv...
Location
Location
United States , New York
Salary
Salary:
175000.00 - 215000.00 USD / Year
rennerbrown.com Logo
Renner Brown
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in computer science, Computer Engineering, or related field (Master's degree is a plus)
  • 8+ years in infrastructure engineering, cloud platform engineering, or data engineering
  • Demonstrated experience building shared platforms or developer services in an enterprise environment
  • Azure expertise: Azure AI Foundry, Azure Data Factory, Azure Databricks, AKS, Azure API Management, Azure Key Vault, Azure Entra ID
  • Strong Python skills: backend services, REST APIs (FastAPI or Flask), and automation scripting
  • PowerShell for infrastructure tasks
  • Infrastructure-as-Code: Terraform and/or Bicep
  • container orchestration with Docker and Kubernetes
  • Experience integrating LLM APIs (Anthropic Claude, Azure OpenAI) in production including token cost management and observability
  • RAG pipeline experience: vector search (Azure AI Search or pgvector), document processing, and retrieval patterns
Job Responsibility
Job Responsibility
  • Design, build, and operate the firm's AI platform, enabling developers to build and deploy Python-based AI applications
  • Implement and manage Azure AI Foundry environments: model deployments, AI hubs, project workspaces, and access controls
  • Integrate and operationalize third-party AI APIs (Anthropic Claude API, Azure OpenAI) with secure access patterns, API gateway controls, rate limiting, and cost monitoring
  • Build internal developer tooling and SDK scaffolding to accelerate AI application development across the firm
  • Build and maintain data pipelines using Azure Data Factory and Azure Databricks to serve AI application data needs
  • Implement vector search and document retrieval infrastructure (Azure AI Search) to support RAG-based applications
  • Manage structured and unstructured data stores including Azure Data Lake, Azure SQL, and Cosmos DB
  • Provision and maintain secure, scalable infrastructure on Azure (primary) and AWS using Infrastructure-as-Code (Terraform or Bicep)
  • Build and maintain CI/CD pipelines for AI application deployment via Azure DevOps or GitHub Actions
  • Manage containerized workloads using Docker and Kubernetes (AKS) for AI application hosting and API services
  • Fulltime
Read More
Arrow Right