CrawlJobs Logo

Data Lake SME

https://www.hpe.com/ Logo

Hewlett Packard Enterprise

Location Icon

Location:
India , Bangalore

Category Icon

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

Not provided

Job Description:

We are looking for an experienced Data Lake / ETL Engineer with 7+ years of expertise in designing, developing, and managing large-scale data ingestion, transformation, and analytics pipelines. The role involves building scalable and secure data lake platforms, enabling business insights through efficient ETL/ELT frameworks, and ensuring data quality, performance, and governance across the enterprise ecosystem.

Job Responsibility:

  • Design and implement data ingestion pipelines for structured, semi-structured, and unstructured data
  • Develop and manage ETL/ELT processes for large-scale data processing
  • Optimize storage and retrieval strategies across on-prem and cloud-based data lakes
  • Integrate data from multiple sources (databases, APIs, streaming platforms)
  • Implement real-time and batch processing using Apache Spark, Kafka, or Flink
  • Support metadata management, data lineage, and cataloging
  • Tune queries and pipelines for high performance and cost efficiency
  • Implement partitioning, indexing, and caching strategies for large datasets
  • Automate routine ETL/ELT workflows for reliability and speed
  • Ensure compliance with data governance, privacy, and regulatory standards (GDPR, HIPAA, etc.)
  • Implement encryption, masking, and role-based access control (RBAC)
  • Collaborate with cybersecurity teams to align with Zero Trust and IAM policies
  • Partner with data scientists, analysts, and application teams for analytics enablement
  • Provide L2/L3 support for production pipelines and troubleshoot failures
  • Mentor junior engineers and contribute to best practices documentation

Requirements:

  • 7+ years of experience in data engineering, ETL/ELT development, or data lake management
  • Strong expertise in ETL tools (Informatica, Talend, dbt, SSIS, or similar)
  • Hands-on experience with big data ecosystems: Hadoop, Spark, Hive, Presto, Delta Lake, or Iceberg
  • Proficiency with SQL, Python, or Scala for data processing and transformation
  • Experience with cloud data platforms (AWS Glue, Redshift, Azure Synapse, GCP BigQuery)
  • Familiarity with workflow orchestration tools (Airflow, Temporal, Oozie)

Nice to have:

  • Exposure to real-time data streaming (Kafka, Kinesis, Pulsar)
  • Knowledge of data modeling (Kimball/Inmon), star schema, and dimensional modeling
  • Experience with containerized deployments (Docker, Kubernetes)
  • Informatica/Talend/dbt certifications
What we offer:
  • Health & Wellbeing
  • Personal & Professional Development
  • Unconditional Inclusion

Additional Information:

Job Posted:
October 08, 2025

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Data Lake SME

Pre-Sales Solution Engineer

We are seeking a highly skilled Enterprise focused Pre-Sales Solution Engineer t...
Location
Location
Salary
Salary:
Not provided
lakefs.io Logo
LakeFS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Engineering, or a related technical field
  • 5+ years of experience in a technical pre-sales or solution engineering role, preferably in the software industry
  • Excellent verbal and written communication skills, with the ability to deliver technical presentations to both technical and non-technical audiences
  • Hands-on experience with Kubernetes and containerized environments
  • Experience leading client workshops, sales enablement sessions, and technical training
  • Ability to work independently as well as collaboratively across sales, product, and engineering teams
  • Excellent problem-solving skills and a creative approach to architecting solutions
  • Experience in technical sales of open source products is preferred
  • Strong understanding and hands-on experience in the Data & AI ecosystem, including: Data lake and data warehouse technologies (e.g., S3, Delta Lake, Iceberg, Hive, Glue, Snowflake)
  • Data processing frameworks (e.g., Spark, Databricks, Flink, Trino, Presto)
Job Responsibility
Job Responsibility
  • Own the technical win from discovery through demo, architecture design, pilot/POC, security review, and a clean handoff to Customer Success
  • Design and present tailored demos that showcase branch‑based workflows, reproducible experiments, schema‑safe changes, and instant data rollback
  • Scope and execute pilots with clear success criteria
  • create sample repos, notebooks, and automation (Spark/Databricks jobs, Airflow DAGs, CI pipelines) that prove value quickly
  • Build reference architectures for lakeFS OSS and Cloud across AWS/Azure/GCP
  • document IAM roles, private networking, scaling, GC/performance tuning, and disaster recovery
  • Answer deep technical questions as a lakeFS SME
  • Handle RFPs and security questionnaires
  • map controls to customer requirements and recommend compliant deployment patterns
  • Partner with sales to quantify business impact (risk reduction, developer velocity, storage efficiency) and co‑create the ROI/TCO narrative with champions
Read More
Arrow Right

Senior Bigdata Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8 - 10 years of relevant experience
  • Experience in systems analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Programming Languages: Python, PySpark
  • Data Lake Table Format: Apache Iceberg
  • Data Orchestration: Apache Airflow
  • Data Visualization: Tableau
  • Big Data Processing: Apache Spark
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
What we offer
What we offer
  • Equal employment opportunity
  • Fulltime
Read More
Arrow Right

Senior Cloud Data Architect

As a Senior Cloud Architect, your role will focus on supporting users, collabora...
Location
Location
Spain , Barcelona
Salary
Salary:
Not provided
https://www.allianz.com Logo
Allianz
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong expertise in Azure cloud infrastructure, Data & AI technologies, and data platform management, with proficiency in Azure Synapse Analytics, Azure Machine Learning, Azure Data Lake, and Informatica Intelligent Data Management Cloud (IDMC)
  • Proven experience in modern Data Warehouse architectures (e.g., Lakehouse) and integrating machine learning models and AI capabilities using Azure services like Cognitive Services and Azure Bot Service for predictive analytics and automation
  • In-depth knowledge of data security and compliance practices using Azure AD, Azure Key Vault, and Informatica’s data governance tools, focusing on data privacy and regulatory standards
  • Expertise in optimizing resource usage, performance, and costs across Azure services and IDMC, leveraging tools like Azure Cost Management and Azure Monitor, and skilled in ETL/ELT tools and advanced SQL
  • Proficiency in data integration, machine learning, and generative AI from an architectural perspective, with hands-on experience in Python, SQL, Spark/Scala/PySpark, and container solutions like Docker and Kubernetes
  • Experience with CI/CD pipelines (e.g., GitHub Actions, Jenkins), microservices architectures, and APIs, with knowledge of architecture frameworks like TOGAF or Zachman, adept at managing multiple priorities in fast-paced environments, and excellent communication and presentation skills
  • Over 5 years of experience in cloud architecture focusing on Data & AI infrastructure, particularly in Azure, with expertise in building scalable, secure, and cost-effective solutions for data analytics and AI/ML environments.
Job Responsibility
Job Responsibility
  • Define and prioritize new functional and non-functional capabilities for the cloud-based data platform, ensuring alignment with business needs and Allianz's security, compliance, privacy, and architecture standards
  • Act as the platform SME for both potential and existing users, guiding them in the architecture of scalable, high-performance Data & AI solutions
  • Provide leadership and product guidance to engineering teams during the design, development, and implementation of new platform capabilities
  • Ensure all solutions meet defined quality standards and acceptance criteria
  • Work with stakeholders to co-create data solutions, optimizing business models and identifying opportunities for improved data usage
  • Lead the evaluation and selection of technologies and partners to implement data analytics use cases, focusing on proofs of concept and prototypes
  • Stay up to date with emerging trends in Data, Analytics, AI/ML, and cloud technologies
  • Leverage open-source technologies and cloud tools to drive innovation and cost-efficiency
  • Prepare materials for management briefings and public events
  • Represent the team in technical discussions, particularly regarding architecture and platform capabilities.
What we offer
What we offer
  • Hybrid work model which recognizes the value of striking a balance between in-person collaboration and remote working incl. up to 25 days per year working from abroad
  • Rewarding performance through company bonus scheme, pension, employee shares program, and multiple employee discounts
  • Career development and digital learning programs to international career mobility
  • Flexible working, health and wellbeing offers (including healthcare and parental leave benefits)
  • Support for balancing family and career and helping employees return from career breaks with experience that nothing else can teach.
  • Fulltime
Read More
Arrow Right

Solution Architect

The Solution Architect role involves driving the architectural transformation fo...
Location
Location
Ireland , Dublin
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant experience in Data modeling, Data lineage analysis, Operational reporting, preferably in a global organization
  • Proven architecture experience in solutioning of horizontally scalable, highly available, highly resilient data distribution platforms
  • Proficient in message queuing, stream processing, and highly scalable ‘big data’ data stores
  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
  • Strong analytic skills related to working with unstructured datasets
  • Extensive experience with Data Integration patterns
  • Extensive experience with Real/Near Real time streaming patterns
  • Strong background in Data Management, Data Governance, Transformation initiatives preferred
  • Preferred Experience/Familiarity with one or more of these tools: Big data platforms - Hadoop, Apache Kafka, Relational SQL, NoSQL, and Cloud Native databases - Postgres, Cassandra, Snowflake, Experience with data pipeline and orchestration tools - Azkaban, Luigi, or Airflow, Experience with stream-processing engines - Apache Spark, Apache Storm, or Apache Flink and ETL tools - Talend, Ab Initio, Experience with Data Analytics/visualization tools - Looker, Mode, or Tableau
Job Responsibility
Job Responsibility
  • Re-engineering the interaction of incoming and outgoing data flows from the Core Accounts DDA platform to Reference Data platforms, Data Warehouse, Data Lake as well as other local reporting systems which consume data from Core Accounts
  • Drive data architecture and roadmap for eliminating non-strategic point-to-point connections and batch handoffs
  • Define canonical data models for key entities and events related to Customer, Account, Core DDA in line with the Data Standards
  • Assess opportunities to simplify/rationalize/refactor the existing database schemas paving way for modularization of the existing stack
  • Provide technical guidance to Data Engineers responsible for designing an Operational Data Store for intra-day and end-of-day reporting
  • Implementing data strategies and developing logical and physical data models
  • Formulate efficient approach to rationalize and formulate strategy to migrate reports
  • Build and nurture a strong engineering organization to deliver value to internal and external clients
  • Acts as SME to senior stakeholders in business, operations, and technology divisions across upstream and downstream Organizations
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
What we offer
What we offer
  • Competitive base salary (annually reviewed)
  • Hybrid working model (up to 2 days working at home per week)
  • Additional benefits supporting you and your family
  • Fulltime
Read More
Arrow Right
New

CRM Developer

We’re Fever, the world’s leading tech platform for culture and live entertainmen...
Location
Location
Spain , Madrid
Salary
Salary:
Not provided
https://feverup.com/fe Logo
Fever
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • You have a bachelor's in engineering (civil, industrial, telecommunications, or aeronautics) or a background in maths, statistics or data analytics
  • At least +3 years of experience in a CRM relative role
  • A high level of English. Speaking Spanish is a plus
  • Proven experience in Salesforce Marketing Cloud or any other similar marketing automation platform
  • Experience in strategy consulting and strategy or analytics roles at top tech firms is a significant advantage
  • Data-driven person who brings numbers to strategic and business decisions
  • Generate high business impact based on your decisions
  • Excellent interpersonal skills with a proven ability to manage and influence multiple stakeholders while working effectively across teams, disciplines, and business cases
  • Understanding any coding language is preferred (SQL, R, Python…)
  • Proactive problem-solver and an opportunity-seeker
Job Responsibility
Job Responsibility
  • Lead global CRM initiatives focusing on its growth and development
  • Strategic Campaign Design & Execution: Lead the planning, creation, and execution of CRM campaigns across multiple channels (email, push notifications, SMS, WhatsApp), aimed at engaging, retaining, and growing the user base and the revenue generated through these channels
  • Data-Driven Optimization: Analyze user behavior, campaign performance, and customer feedback to continuously improve messaging, segmentation, and overall CRM effectiveness
  • Cross-Functional Collaboration: Work closely with marketing, product, and data teams to align CRM strategies with business goals, ensuring a seamless and personalized user journey across all touchpoints
  • Being a Growth Manager at Fever means taking full ownership of the results, evolution, goals, and strategic decisions within a specific business line
  • Take part in the exponential growth of fever by taking the lead on global projects that will define the future of the entertainment industry
  • Enhance the overall company performance across different business departments which work together in the creation of new plans and events
  • Define and establish new strategies to increase performance using analytical, business and strategic skills
  • Design end-to-end solutions with daily high impact
  • Create, develop and manage new projects like new business streams, workflows, strategies, and specific departments' growth
What we offer
What we offer
  • Have a real impact on the company's growth and evolution
  • Excellent working conditions with a location in the heart of the Madrid scene
  • Home-office friendly
  • Great work environment with a young, international team of talented people to work with
  • 40% Discount on all events on Fever
  • Health insurance and other benefits such as Flexible remuneration with a 100% tax exemption through Cobee
  • English Lessons
  • We have free food, drink, and fruit at the office!
  • Possibility to receive in advance part of your salary by Payflow
  • Gympass Membership
  • Fulltime
Read More
Arrow Right
New

Tender Writer

We are seeking an experienced Tender Writer / Technical Author to lead and write...
Location
Location
United Kingdom , Cardiff; Greater London
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Must be able to write all content for bids / tender responses as apposed to a ‘Bid Manager’
  • Could have an APMP qualification or be willing to undergo this certification
  • Must be able to pass an SC Security Check (UK Based, Passport holder 5+ yrs minimum)
Job Responsibility
Job Responsibility
  • Lead and write responses to UK Public Sector tenders for UK Sovereign IT services
  • Work closely with technical, legal, and commercial teams to deliver winning proposals
  • Write all of the content required for winning IT bid submissions and tenders for the UK Public Sector
  • Work solo on bids as well as in teams when required
  • Create repeatable content within your bid writing team for assisting in future bids
  • Fulltime
Read More
Arrow Right
New

Senior Technical Product Manager

As a Senior Product Manager for the Care Internal Tools at EarnIn, you will dire...
Location
Location
Thailand , Bangkok
Salary
Salary:
Not provided
earnin.com Logo
EarnIn
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of product management at a consumer technology product company
  • Strong product/design sense developed by shipping delightful and innovative products
  • Strong analytical skills in leveraging data to drive product decisions and improve metrics
  • Deep user empathy and experience in building products based on user needs
  • Excellent written and verbal communication skills
  • Experience with a culture of rapid experimentation to accelerate learning and iteration
  • Ability to influence and build alignment between different teams and functions to execute your roadmap and deliver against goals successfully
  • Ability to get the job done in a highly constrained space, even if that means rolling up your sleeves for your team
  • Ability to think outside the box to solve complex problems with creative and radical ideas
  • Prior experience working on web technologies
Job Responsibility
Job Responsibility
  • Maintain a constant pulse on our community members to understand their needs and the problems they face in solving those needs
  • Create tight feedback loops with our community members and create a deep understanding of Product Issues and the path to resolution
  • Raise the bar for design quality and user experience to make the experience of using Earnin delightful and engaging for our community members
  • Lead a cross-functional team of engineers, designers, researchers, and data analysts to build and scale products that make the most impact on our community members
  • Define OKRs and set stretch goals for your product and team
  • Establish, monitor, and analyze metrics to track progress toward product goals
  • Leverage user research and data analysis to identify and prioritize new product opportunities for your roadmap
  • Design, develop, and execute features and experiments to iterate your way to the product’s success rapidly
  • Communicate the product vision, roadmap, and results to your product team, stakeholders, including the executive team, and community members
  • Collaborate with key stakeholders across marketing, risk, customer support, finance, and legal to ship sustainable and compliant products
What we offer
What we offer
  • Hybrid setup with flexibility
  • 80m THB/year International health insurance (IPD/OPD/Dental/Optical)
  • Working with international teams and potential opportunity for travel to our Mountain View HQ
  • Flexible annual learning budget
  • Home office setup allowance
  • Phone, internet, and transportation allowance
  • Healthcare
  • Internet/cell phone reimbursement
  • A learning and development stipend
  • Opportunities to travel to our Mountain View HQ
  • Fulltime
Read More
Arrow Right
New

Product Evangelist

We are looking for a dynamic and highly knowledgeable Product Evangelist to beco...
Location
Location
United Kingdom , London; Cardiff; Birmingham
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Deep technical understanding
  • Exceptional communication skills
  • Previous experience in Marketing is a great advantage
Job Responsibility
Job Responsibility
  • Become the public voice and subject‑matter expert for Sovereign Cloud products
  • Educate, inspire, and influence customers, partners, and the wider market
  • Work closely with the marketing team to shape and execute the overall marketing strategy
  • Deliver high‑quality content through blogs, online events, webinars, and in‑person presentations
  • Act as the primary spokesperson and thought leader for the product suite
  • Produce high‑quality written content including blogs, articles, whitepapers, and social media posts
  • Represent Rackspace Sovereign Cloud on stage at industry events, conferences, and customer engagements
  • Fulltime
Read More
Arrow Right