CrawlJobs Logo

Data Lake SME

https://www.hpe.com/ Logo

Hewlett Packard Enterprise

Location Icon

Location:
India , Bangalore

Category Icon

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

Not provided

Job Description:

We are looking for an experienced Data Lake / ETL Engineer with 7+ years of expertise in designing, developing, and managing large-scale data ingestion, transformation, and analytics pipelines. The role involves building scalable and secure data lake platforms, enabling business insights through efficient ETL/ELT frameworks, and ensuring data quality, performance, and governance across the enterprise ecosystem.

Job Responsibility:

  • Design and implement data ingestion pipelines for structured, semi-structured, and unstructured data
  • Develop and manage ETL/ELT processes for large-scale data processing
  • Optimize storage and retrieval strategies across on-prem and cloud-based data lakes
  • Integrate data from multiple sources (databases, APIs, streaming platforms)
  • Implement real-time and batch processing using Apache Spark, Kafka, or Flink
  • Support metadata management, data lineage, and cataloging
  • Tune queries and pipelines for high performance and cost efficiency
  • Implement partitioning, indexing, and caching strategies for large datasets
  • Automate routine ETL/ELT workflows for reliability and speed
  • Ensure compliance with data governance, privacy, and regulatory standards (GDPR, HIPAA, etc.)
  • Implement encryption, masking, and role-based access control (RBAC)
  • Collaborate with cybersecurity teams to align with Zero Trust and IAM policies
  • Partner with data scientists, analysts, and application teams for analytics enablement
  • Provide L2/L3 support for production pipelines and troubleshoot failures
  • Mentor junior engineers and contribute to best practices documentation

Requirements:

  • 7+ years of experience in data engineering, ETL/ELT development, or data lake management
  • Strong expertise in ETL tools (Informatica, Talend, dbt, SSIS, or similar)
  • Hands-on experience with big data ecosystems: Hadoop, Spark, Hive, Presto, Delta Lake, or Iceberg
  • Proficiency with SQL, Python, or Scala for data processing and transformation
  • Experience with cloud data platforms (AWS Glue, Redshift, Azure Synapse, GCP BigQuery)
  • Familiarity with workflow orchestration tools (Airflow, Temporal, Oozie)

Nice to have:

  • Exposure to real-time data streaming (Kafka, Kinesis, Pulsar)
  • Knowledge of data modeling (Kimball/Inmon), star schema, and dimensional modeling
  • Experience with containerized deployments (Docker, Kubernetes)
  • Informatica/Talend/dbt certifications
What we offer:
  • Health & Wellbeing
  • Personal & Professional Development
  • Unconditional Inclusion

Additional Information:

Job Posted:
October 08, 2025

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Data Lake SME

Pre-Sales Solution Engineer

We are seeking a highly skilled Enterprise focused Pre-Sales Solution Engineer t...
Location
Location
Salary
Salary:
Not provided
lakefs.io Logo
LakeFS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Engineering, or a related technical field
  • 5+ years of experience in a technical pre-sales or solution engineering role, preferably in the software industry
  • Excellent verbal and written communication skills, with the ability to deliver technical presentations to both technical and non-technical audiences
  • Hands-on experience with Kubernetes and containerized environments
  • Experience leading client workshops, sales enablement sessions, and technical training
  • Ability to work independently as well as collaboratively across sales, product, and engineering teams
  • Excellent problem-solving skills and a creative approach to architecting solutions
  • Experience in technical sales of open source products is preferred
  • Strong understanding and hands-on experience in the Data & AI ecosystem, including: Data lake and data warehouse technologies (e.g., S3, Delta Lake, Iceberg, Hive, Glue, Snowflake)
  • Data processing frameworks (e.g., Spark, Databricks, Flink, Trino, Presto)
Job Responsibility
Job Responsibility
  • Own the technical win from discovery through demo, architecture design, pilot/POC, security review, and a clean handoff to Customer Success
  • Design and present tailored demos that showcase branch‑based workflows, reproducible experiments, schema‑safe changes, and instant data rollback
  • Scope and execute pilots with clear success criteria
  • create sample repos, notebooks, and automation (Spark/Databricks jobs, Airflow DAGs, CI pipelines) that prove value quickly
  • Build reference architectures for lakeFS OSS and Cloud across AWS/Azure/GCP
  • document IAM roles, private networking, scaling, GC/performance tuning, and disaster recovery
  • Answer deep technical questions as a lakeFS SME
  • Handle RFPs and security questionnaires
  • map controls to customer requirements and recommend compliant deployment patterns
  • Partner with sales to quantify business impact (risk reduction, developer velocity, storage efficiency) and co‑create the ROI/TCO narrative with champions
Read More
Arrow Right

Senior Bigdata Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8 - 10 years of relevant experience
  • Experience in systems analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Programming Languages: Python, PySpark
  • Data Lake Table Format: Apache Iceberg
  • Data Orchestration: Apache Airflow
  • Data Visualization: Tableau
  • Big Data Processing: Apache Spark
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
What we offer
What we offer
  • Equal employment opportunity
  • Fulltime
Read More
Arrow Right

Senior Cloud Data Architect

As a Senior Cloud Architect, your role will focus on supporting users, collabora...
Location
Location
Spain , Barcelona
Salary
Salary:
Not provided
https://www.allianz.com Logo
Allianz
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong expertise in Azure cloud infrastructure, Data & AI technologies, and data platform management, with proficiency in Azure Synapse Analytics, Azure Machine Learning, Azure Data Lake, and Informatica Intelligent Data Management Cloud (IDMC)
  • Proven experience in modern Data Warehouse architectures (e.g., Lakehouse) and integrating machine learning models and AI capabilities using Azure services like Cognitive Services and Azure Bot Service for predictive analytics and automation
  • In-depth knowledge of data security and compliance practices using Azure AD, Azure Key Vault, and Informatica’s data governance tools, focusing on data privacy and regulatory standards
  • Expertise in optimizing resource usage, performance, and costs across Azure services and IDMC, leveraging tools like Azure Cost Management and Azure Monitor, and skilled in ETL/ELT tools and advanced SQL
  • Proficiency in data integration, machine learning, and generative AI from an architectural perspective, with hands-on experience in Python, SQL, Spark/Scala/PySpark, and container solutions like Docker and Kubernetes
  • Experience with CI/CD pipelines (e.g., GitHub Actions, Jenkins), microservices architectures, and APIs, with knowledge of architecture frameworks like TOGAF or Zachman, adept at managing multiple priorities in fast-paced environments, and excellent communication and presentation skills
  • Over 5 years of experience in cloud architecture focusing on Data & AI infrastructure, particularly in Azure, with expertise in building scalable, secure, and cost-effective solutions for data analytics and AI/ML environments.
Job Responsibility
Job Responsibility
  • Define and prioritize new functional and non-functional capabilities for the cloud-based data platform, ensuring alignment with business needs and Allianz's security, compliance, privacy, and architecture standards
  • Act as the platform SME for both potential and existing users, guiding them in the architecture of scalable, high-performance Data & AI solutions
  • Provide leadership and product guidance to engineering teams during the design, development, and implementation of new platform capabilities
  • Ensure all solutions meet defined quality standards and acceptance criteria
  • Work with stakeholders to co-create data solutions, optimizing business models and identifying opportunities for improved data usage
  • Lead the evaluation and selection of technologies and partners to implement data analytics use cases, focusing on proofs of concept and prototypes
  • Stay up to date with emerging trends in Data, Analytics, AI/ML, and cloud technologies
  • Leverage open-source technologies and cloud tools to drive innovation and cost-efficiency
  • Prepare materials for management briefings and public events
  • Represent the team in technical discussions, particularly regarding architecture and platform capabilities.
What we offer
What we offer
  • Hybrid work model which recognizes the value of striking a balance between in-person collaboration and remote working incl. up to 25 days per year working from abroad
  • Rewarding performance through company bonus scheme, pension, employee shares program, and multiple employee discounts
  • Career development and digital learning programs to international career mobility
  • Flexible working, health and wellbeing offers (including healthcare and parental leave benefits)
  • Support for balancing family and career and helping employees return from career breaks with experience that nothing else can teach.
  • Fulltime
Read More
Arrow Right

Solution Architect

The Solution Architect role involves driving the architectural transformation fo...
Location
Location
Ireland , Dublin
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant experience in Data modeling, Data lineage analysis, Operational reporting, preferably in a global organization
  • Proven architecture experience in solutioning of horizontally scalable, highly available, highly resilient data distribution platforms
  • Proficient in message queuing, stream processing, and highly scalable ‘big data’ data stores
  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
  • Strong analytic skills related to working with unstructured datasets
  • Extensive experience with Data Integration patterns
  • Extensive experience with Real/Near Real time streaming patterns
  • Strong background in Data Management, Data Governance, Transformation initiatives preferred
  • Preferred Experience/Familiarity with one or more of these tools: Big data platforms - Hadoop, Apache Kafka, Relational SQL, NoSQL, and Cloud Native databases - Postgres, Cassandra, Snowflake, Experience with data pipeline and orchestration tools - Azkaban, Luigi, or Airflow, Experience with stream-processing engines - Apache Spark, Apache Storm, or Apache Flink and ETL tools - Talend, Ab Initio, Experience with Data Analytics/visualization tools - Looker, Mode, or Tableau
Job Responsibility
Job Responsibility
  • Re-engineering the interaction of incoming and outgoing data flows from the Core Accounts DDA platform to Reference Data platforms, Data Warehouse, Data Lake as well as other local reporting systems which consume data from Core Accounts
  • Drive data architecture and roadmap for eliminating non-strategic point-to-point connections and batch handoffs
  • Define canonical data models for key entities and events related to Customer, Account, Core DDA in line with the Data Standards
  • Assess opportunities to simplify/rationalize/refactor the existing database schemas paving way for modularization of the existing stack
  • Provide technical guidance to Data Engineers responsible for designing an Operational Data Store for intra-day and end-of-day reporting
  • Implementing data strategies and developing logical and physical data models
  • Formulate efficient approach to rationalize and formulate strategy to migrate reports
  • Build and nurture a strong engineering organization to deliver value to internal and external clients
  • Acts as SME to senior stakeholders in business, operations, and technology divisions across upstream and downstream Organizations
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
What we offer
What we offer
  • Competitive base salary (annually reviewed)
  • Hybrid working model (up to 2 days working at home per week)
  • Additional benefits supporting you and your family
  • Fulltime
Read More
Arrow Right
New

Admin/inventory worker in warehouse

As an admin worker in warehouse, you will play a key role in maintaining the eff...
Location
Location
Canada , Pointe-Claire, Montreal
Salary
Salary:
20.00 CAD / Hour
https://www.randstad.com Logo
Randstad
Expiration Date
February 13, 2026
Flip Icon
Requirements
Requirements
  • Hold permanent residence or Canadian citizenship
  • Warehouse experience: receiving, putting away, picking, packing, shipping, and inventory management
  • Knowledge and experience using RF scanners and handheld tools
  • Functional proficiency with WMS systems, specifically SAP and/or Blue Yonder
  • Proficiency in the use of office applications, including Microsoft Sheets and Google Sheets
  • Ability to operate a manual pallet jack
  • Excellent communication skills and ability to follow detailed instructions
  • Ability to work independently within a team dedicated to Manufacturing and Logistics
Job Responsibility
Job Responsibility
  • Performing all logistics processes in the supply chain: receiving, storing, picking, packing, and shipping
  • Maintaining inventory accuracy
  • Effectively using warehouse management systems (WMS), including SAP and Blue Yonder, to record and track stock movements
  • Operating handling equipment, including pallet jacks, safely and efficiently
  • Use RF scanners for inventory tracking and hand tools for equipment assembly or preparation
  • Prepare shipping and inventory documentation using office tools such as Microsoft and Google Sheets
What we offer
What we offer
  • Weekly pay
  • Day shift
  • Annual pay raises
  • Work equipment provided by the company (pants, jackets, gloves, hats)
  • Free parking
  • Accessible by public transportation
  • Fulltime
Read More
Arrow Right
New

Warehouse associate

We are looking for experienced and dependable Reach Truck Forklift Operators to ...
Location
Location
Canada , Pointe-Claire, Montreal
Salary
Salary:
20.00 CAD / Hour
https://www.randstad.com Logo
Randstad
Expiration Date
March 01, 2026
Flip Icon
Requirements
Requirements
  • Have a valid Reach Forklift Certification
  • Have at least 1 year of experience operating a reach truck
  • Be comfortable working in a refrigerated and freezer warehouse environment
  • Strong Safety background
  • Pass a forklift assessment and complete one week of paid training
  • Safety Shoes will be required
  • Have a valid EXTERNAL forklift operator's license from a recognized training center in Ontario
Job Responsibility
Job Responsibility
  • Operate Reach Trucks safely and efficiently
  • Pick orders by pallet or case
  • Use RF scanners to manage and track inventory
  • Work in cold storage environments (PPE provided)
  • Carrying out shipping, receiving and storage tasks
What we offer
What we offer
  • Weekly pay
  • Day shift
  • Annual pay raises
  • Company-provided freezer gear (pants, jackets, gloves, toques)
  • Free parking
  • Accessible by public transportation
  • Fulltime
Read More
Arrow Right
New

Scooter delivery driver

Do you own a scooter and want to use it to earn a nice salary as a scooter deliv...
Location
Location
Netherlands , Zwolle
Salary
Salary:
14.71 EUR / Hour
https://www.randstad.com Logo
Randstad
Expiration Date
August 03, 2026
Flip Icon
Requirements
Requirements
  • You are 16 or older
  • Speak and understand English on a basic level
  • You have your own scooter and a valid driver’s license
  • You own a smartphone with sufficient data and a fully charged powerbank
Job Responsibility
Job Responsibility
  • Delivering tasty meals to customers in Zwolle
  • Using the Uber Eats app to guide you to your next delivery
  • Picking up orders at restaurants in the city
  • Navigating and delivering meals to hungry customers in Zwolle
What we offer
What we offer
  • You decide when and how much you work
  • Work outdoors and explore Zwolle on a scooter
  • Earn a km allowance of € 0,23 per km driven
  • Job for a longer period of time
  • Great colleagues and employer
  • With each order you deliver during dinner hours (17:00 – 21:00), you will earn an extra euro
  • Parttime
Read More
Arrow Right
New

Delivery rider

Are you looking for a flexible job as a delivery rider in Alphen a/d rijn? Then ...
Location
Location
Netherlands , Alphen aan den rijn
Salary
Salary:
15.71 EUR / Hour
https://www.randstad.com Logo
Randstad
Expiration Date
August 03, 2029
Flip Icon
Requirements
Requirements
  • You have your own (e-)bike
  • You speak English or Dutch at a good level
  • You are older then 16
  • You love work outdoors
  • You have a powerbank
  • You have a smartphone with internet access
Job Responsibility
Job Responsibility
  • Receive an order via the Uber Eats app
  • Bike to the address to pick up the order
  • Drop off the meals at the consumer’s house
  • Press finish in the app after delivery
What we offer
What we offer
  • Flexible schedule, choose when you want to work
  • 8% holiday pay
  • Work outdoors
  • Learning your way around your own city
  • Equipment from Uber Eats
  • Dinner bonus: a bonus up to €4,60 extra for every order you deliver dinner hours (16:00-20:00)
  • Parttime
Read More
Arrow Right