This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
Data Platform Engineers at Adyen build the foundational layer of tooling and processes for our on-premise Big Data Platforms. These tools support 10s of products, 100s of developers, and 1000s of daily jobs that add to Adyen’s strong portfolio of capabilities. You’ll collaborate with data engineers and ML scientists and engineers to build and roll-out these tools. Along with DataOps Engineers, you’ll develop and operate multiple big data platforms, of which the biggest is 500+ nodes, with 25k+ CPU cores, 240+ TBs of RAM, and 23+ PBs of storage capacity, running tens of thousands of spark jobs a day.
Job Responsibility:
Develop and maintain scalable and high performance big data platforms
Work with distributed systems in all shapes and flavors (databases, filesystems, compute, etc.)
Identify opportunities to improve continuous release and deployment environments
Build or deploy tools to enhance data discoverability through the collection and presentation of metadata
Introduce and extend tools to enhance the quality of our data, platform-wide
Explore and introduce technologies and practices to reduce the time to insight for analysts and data scientists
Develop streaming processing applications and frameworks
Build the foundational layer of tooling and processes for on-premise Big Data Platforms
Collaborate with data engineers and ML scientists and engineers to build and roll-out tools
Develop and operate multiple big data platforms
Requirements:
Fluency in Python
Experience developing and maintaining distributed data and compute systems like Spark, Trino, Druid, etc
Experience developing and maintaining DevOps pipelines and development ecosystems
Experience developing and maintaining real-time and batch data pipelines (via Kafka, Spark streaming)
Welcome to CrawlJobs.com – Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
We use cookies to enhance your experience, analyze traffic, and serve personalized content. By clicking “Accept”, you agree to the use of cookies.