Data Engineer (Spark) Job at Addepto sp. z o.o. (Warsaw)

Job Description

Addepto is a leading AI consulting and data engineering company that builds scalable, ROI-focused AI solutions for some of the world’s largest enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. With our exclusive focus on Artificial Intelligence and Big Data, we help organizations unlock the full potential of their data through systems designed for measurable business impact and long-term growth. As a Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging projects across various industries, leveraging cutting-edge technologies.

Job Responsibility

Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability
Design and implement data pipelines that process large volumes of data in both streaming and batch modes
Optimize data workflows to ensure efficient data ingestion, processing, and storage using technologies such as Spark, Cloudera, and Airflow
Work with data lake technologies (e.g., Iceberg) to manage structured and unstructured data efficiently
Collaborate with cross-functional teams to understand data requirements and ensure seamless integration of data sources
Monitor and troubleshoot the platform, ensuring high availability, performance, and accuracy of data processing
Leverage cloud services (AWS) for infrastructure management and scaling of processing workloads
Write and maintain high-quality Python (or Java/Scala) code for data processing tasks and automation

Requirements

At least 4 years of commercial experience implementing, developing, or maintaining Big Data systems, data governance and data management processes
Strong programming skills in Python (or Java/Scala): writing a clean code, OOP design
Hands-on with Big Data technologies like Spark, Cloudera, Data Platform, Kafka, Airflow, NiFi, Docker, and Iceberg
Excellent understanding of dimensional data and data modeling techniques
Experience implementing and deploying solutions in cloud environments
Consulting experience with excellent communication and client management skills, including prior experience directly interacting with clients as a consultant
Ability to work independently and take ownership of project deliverables
Fluent English (at least C1 level)
Bachelor’s degree in technical or mathematical studies

Nice to have

Experience with an MLOps framework such as Kubeflow or MLFlow
Familiarity with Databricks and/or dbt

What we offer

Work in a supportive team of passionate enthusiasts of AI & Big Data
Engage with top-tier global enterprises and cutting-edge startups on international projects
Enjoy flexible work arrangements, allowing you to work remotely or from modern offices and coworking spaces
Accelerate your professional growth through career paths, knowledge-sharing initiatives, language classes, and sponsored training or conferences, including a partnership with Databricks, which offers industry-leading training materials and certifications
Choose your preferred form of cooperation: B2B or a contract of mandate, and enjoy 20 fully paid days off
Participate in team-building events and utilize the integration budget
Celebrate work anniversaries, birthdays, and milestones
Access medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching
Get full work equipment for optimal productivity, including a laptop and other necessary devices
With our backing, you can boost your personal brand by speaking at conferences, writing for our blog, or participating in meetups
Experience a smooth onboarding with a dedicated buddy, and start your journey in our friendly, supportive, and autonomous culture

Addepto sp. z o.o. - All Job Offers

Select Country

Data Engineer (Spark)

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?

Data Engineer (Spark)

Senior AWS Data Engineer / Data Platform Engineer

Lead Data Engineer Spark and SQL – Vice President

Lead Big Data Spark Engineer

Senior Data Engineer (ETL+Spark)

Data Engineer – Java & Spark

Real-Time Data Engineer (Flink / Spark Streaming)

Backend Data Engineer — Data Platform

Lead Data Engineer - Data Pipelines

Our AI answers in your language