CrawlJobs Logo
GT Logo GT · IT - Software Development

Data Engineer

Poland, Warsaw · Job Posted February 18, 2026
Apply Position
Job Link Share

Job Description

The Senior Data Engineer will design, build, and optimize the batch data pipelines and algorithms that power this matching engine. This is a hands-on role focused on scale, performance, and accuracy, working with Spark, SQL, and modern orchestration tools. This is not a traditional analytics role — it is a deep data platform and algorithmic engineering at massive scale.

Job Responsibility

  • Build Data Pipelines: Design and maintain robust, scalable ETL/ELT pipelines to ingest and process third-party and first-party datasets
  • Data Quality & Enrichment: Apply transformation, normalization, and enrichment rules to ensure data consistency and usability
  • Collaborate Across Teams: Work with product managers, data architects, and content experts from Coro and Helix to align data structure with business needs
  • Operationalize Matching & Merging Logic: Support the implementation of data matching and entity resolution processes using AI/ML tools and proprietary frameworks
  • Monitor & Troubleshoot Pipelines: Build alerts, logs, and metrics to ensure data flows remain healthy and issues are identified and resolved quickly
  • Documentation & Standards: Contribute to documentation, code quality standards, and internal best practices to ensure maintainability

Requirements

  • 5+ years of experience in Big Data, Data Platform, or ETL engineering roles
  • Proficient in SQL, Python, and experience with Spark(PySpark or Scala), Airflow, Snowflake, and Azure Data Lake or similar technologies
  • Familiarity with Azure (preferred) or other major cloud platforms
  • Proven experience designing and operating large-scale batch data pipelines
  • Solid understanding of distributed systems and algorithms (partitioning, shuffles, joins, scalability trade-offs)
  • Proactive, detail-oriented, and eager to take ownership of projects and continuously improve systems
  • Comfortable working in a cross-functional environment and open to learning from and supporting teammates

Nice to have

  • Experience with, or strong interest in, fuzzy and semantic matching techniques
  • Exposure to ML-assisted data pipelines
  • Familiarity with search or retrieval systems (e.g., Elasticsearch, OpenSearch, vector databases)

What we offer

  • Join a fast-growing, high-impact team
  • Contribute to an ambitious effort to create the highest quality, most comprehensive business directory in the world
  • Be part of a startup-style group within the company that’s redefining how they deliver consulting through productization and data innovation
  • Work with cutting-edge data tools, including AI/ML enrichment, semantic matching, and modern cloud-based infrastructure
GT - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Data Engineer

8 matching positions

Data Engineer

We are currently seeking a Data Engineer to join our team in Bangalore, Karnātak...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 9+ years in Data Engineering / Big Data / Platform Engineering with end to end delivery of large enterprise programs
  • Multi tenant database and data modeling expertise—designing tenant aware schemas and delivering efficient, robust database code and designs for websites and products
  • Advanced SQL across Oracle, Azure SQL Managed Instance (SQL MI), MySQL, and Snowflake
  • strong performance tuning and query optimization
  • Hands on with cloud data platforms: Azure (preferred), including big data services and integrations
  • Deep experience building ETL/ELT pipelines using Informatica, SnapLogic, Azure Functions, Python targeting Snowflake and other cloud data warehouses
  • Proven track record designing, developing, and deploying batch data processing pipelines and feature generation platforms using Azure, Python, and SQL
  • Strong DataOps/CI/CD for data and code: GitHub based workflows, automated database releases, and source control best practices across the SDLC
  • Experience with event/streaming: Azure Event Hubs, Kafka, or Amazon Kinesis
  • Azure platform proficiency, including: Azure Data Factory, Azure Databricks, AKS, Azure Service Bus (ASB), API Management, Storage Accounts, Event Hub/Event Grid, Redis Cache
  • Fulltime
Read More
Arrow Right
New

Data Engineer

As a Data Engineer, you will join Genus and work within the Data Platform Engine...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
September 03, 2026
Flip Icon
Requirements
Requirements
  • Travel Occasional Travel will be required
  • Education Bachelors Degree or Equivilent in Computing related subject
  • Licenses/Certifications Desirable: Databricks Certified Professions
  • Dbt Certified
  • Microsoft Certified
  • SQL / T-SQL and analytical data warehouse development - 8+ years
  • Apache Spark experience using Python, PySpark and SparkSQL at scale - 5+ years
  • Expertise on performance optimisation and production stability of the data workloads - 3+ years
  • Expertise on building metadata-driven, reusable data pipelines using Databricks and ADF - 3+ years
  • Hands-on experience working with Delta-based datasets, multi-layer Lakehouse architectures, and evolving schemas - 3+ years
Job Responsibility
Job Responsibility
  • Design, develop, test, and deploy advanced data engineering solutions using Azure SQL Server, Python, PySpark, Databricks, and Azure Synapse
  • Optimize data models and queries for performance and resource efficiency
  • Build scalable data processing pipelines using Python and PySpark for large-scale data transformation and analytics
  • Implement and manage Databricks and Azure Synapse environments for big data processing and analytics
  • Develop and maintain data transformation models in dbt and Azure Data Factory
  • Ensure proper source control and deployment practices using Git and CI/CD tools (Azure DevOps, GitHub)
  • Collaborate with stakeholders to understand requirements and deliver technical solutions aligned with business needs
  • Guarantee data accuracy and integrity through data quality checks and cleansing processes
  • Performance tuning of SQL queries and data processing scripts
  • Manage and optimize the data platform for cost and performance, including proactive monitoring and self-healing design patterns
  • Fulltime
Read More
Arrow Right

Data Engineer

The Data Engineer position in Dallas, Texas, requires a Bachelor’s or Master’s d...
Location
Location
United States , Dallas
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Education: Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or related field
  • Experience: Minimum 3–5 years of hands-on coding experience
  • ability to troubleshoot SQL and basic scripting
  • 3+ Years of Professional proficiency in Python or Java
  • 3+ Years of experience with SDLC, CI/CD best practices and Kubernetes (K8s) deployment experience
Job Responsibility
Job Responsibility
  • Engineer will be part of the datastore-migration Factory team responsible for end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse
  • Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Executing the physical migration of underlying datasets while ensuring data integrity
  • Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners
  • Translating and optimizing legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg
  • Understanding usage patterns to deliver the required data products
  • Working on data reconciliation frameworks to ensure migrated data is functionally equivalent to production data
  • Working with internal data management platforms and learning new workflows and language constructs as necessary
  • Fulltime
Read More
Arrow Right

Data Engineer

Engineer will be part of the datastore-migration Factory team that will be respo...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to troubleshoot (SQL) and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices & K8s deployment experience
  • Sophisticated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, Architectural Theory
  • Experience with Kafka, ANSI SQL, FTP, Apache Spark
  • Data formats JSON, Avro, Parquet
  • Platforms Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ
Job Responsibility
Job Responsibility
  • Pipeline migration
  • Logic & Scheduling refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Data Transfer executing the physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement acting as a technical liaison to internal clients, facilitating handoff and sign-off with data owners
  • Code Conversion translating and optimizing legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg
  • Usage analysis to deliver required data products
  • Data Reconciliation & Quality working with reconciliation frameworks to build confidence that migrated data is functionally equivalent
  • Working with other internal data management platform
  • Fulltime
Read More
Arrow Right

Data Engineer

The Data Engineer will play a crucial role in migrating data from on-prem DataLa...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to troubleshoot SQL and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices and K8s deployment experience
  • Sophisticated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, Architectural Theory
  • Kafka, ANSI SQL, FTP, Apache Spark
  • JSON, Avro, Parquet
  • Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ
Job Responsibility
Job Responsibility
  • Perform end-to-end datastore migration from on-prem DataLake to AWS LakeHouse
  • Pipeline Migration including refactoring and migrating extraction logic and job scheduling
  • Data Transfer executing physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement acting as technical liaison to internal clients
  • Consumption Pattern Migration including code conversion translating and optimizing legacy SQL and Spark-based consumption patterns for Snowflake and Iceberg
  • Usage analysis to understand usage patterns and deliver required data products
  • Data Reconciliation and Quality working with reconciliation frameworks to ensure data equivalence
  • Work with internal data management platform and learn new workflows
  • Fulltime
Read More
Arrow Right

Data Engineer

Goldman Sachs is seeking a Data Engineer to join their datastore-migration Facto...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to trouble shoot (SQL) and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices & K8s deployment experience
  • Demonstrated understanding of Temporal Data Modeling (e.g., SCD Type 2)
  • Expertise in Schema Evolution (Ref: Iceberg Apache) and enforcement strategies
  • Advanced knowledge of data partitioning and clustering
  • Balancing Normalization vs. Denormalization and the strategic use of Natural vs. Surrogate Keys
  • Good conduct and ethical decision-making
Job Responsibility
Job Responsibility
  • Engineer will be part of the datastore-migration Factory team that will be responsible to perform for the end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse
  • Pipeline Migration - Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Data Transfer - Executing the physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement - Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners to ensure migrated assets meet business requirements
  • Consumption Pattern Migration - Code Conversion: Translating and optimizing legacy SQL and Spark-based consumption patterns (raw and modeled) for compatibility with Snowflake and Iceberg
  • Usage analysis: Understand usage patterns to deliver the required data products
  • Stakeholder Engagement - Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners to ensure migrated assets meet business requirements
  • Data Reconciliation & Quality - A rigorous approach to data validation is required
  • Candidates must work with reconciliation frameworks to build confidence that migrated data is functionally equivalent to that already used within production flows
  • Engineer will also need to work with our other internal data management platform, and must have an aptitude for learning new workflows and language constructs as necessary
  • Fulltime
Read More
Arrow Right

Data Engineer

We are looking for a Data Engineer with 8-12 years of experience to design and b...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8–12 years of experience in Data Engineering
  • Strong hands on expertise in Python and SQL (Snowflake preferred)
  • Experience with REST APIs / data service integrations
  • Working knowledge of AI/ML systems, Agentic AI frameworks
Job Responsibility
Job Responsibility
  • Build and maintain scalable data pipelines using Python and SQL
  • Enable AI/ML and GenAI use cases, including data preparation and feature pipelines
  • Ensure data quality, reliability, security, and observability
  • Fulltime
Read More
Arrow Right

Data Engineer

As our new Data Engineer, you will join a collaborative and skilled organization...
Location
Location
Poland , Katowice
Salary
Salary:
Not provided
vattenfall.com Logo
Vattenfall
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong Python and SQL skills
  • Practical experience with cloud data platforms (Azure, Databricks) and with different data storage solutions, including data lakes, lake houses and analytical databases
  • Hands-on experience with streaming systems (Kafka, Spark or Flink)
  • Familiarity with Airflow, CI/CD, DevOps practices
  • Experience with Docker, Kubernetes, and Infrastructure-as-Code
  • Understanding of data contracts, time-series data, and event-driven systems
  • Fluency in English and interest in the energy domain
Job Responsibility
Job Responsibility
  • Design, build, and operate batch and real-time data pipelines using Azure, Databricks, and Python
  • Ingest and process external and internal data such as weather data, market prices, grid telemetry, and event streams
  • Work with streaming technologies including Kafka, Spark Structured Streaming, and Flink
  • Ensure high data quality through validation, monitoring, and lineage
  • Develop and improve data workflows, standards, and reliability in production systems
What we offer
What we offer
  • Good remuneration
  • challenging and international work environment
  • possibility to work with some of the best in the field
  • opportunities for personal and professional development
  • Fulltime
Read More
Arrow Right