This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
PulsePoint Data Engineering team plays a key role in our technology company that’s experiencing exponential growth. Our data pipeline processes over 80 billion impressions a day (> 20 TB of data, 200 TB uncompressed). This data is used to generate reports, update budgets, and drive our optimization engines. We do all this while running against tight SLAs and provide stats and reports as close to real-time as possible.
Job Responsibility:
Design, build, and maintain reliable and scalable enterprise-level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives
Optimize jobs to utilize Kafka, Hadoop, Presto, Spark, and Kubernetes resources in the most efficient way
Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases)
Collaborate within a small team with diverse technology backgrounds
Provide mentorship and guidance to junior team members
Requirements:
8+ years of data engineering experience
Strong skills in and current experience with SQL and Python
Strong recent Spark experience (3+ years)
Experience working in on-prem environments
Hadoop and Hive experience
Proficiency in Linux
Strong understanding of RDBMS and query optimization
Passion for engineering and computer science around data
East Coast U.S. hours 9am-6pm EST
Notice period needs to be less than 2 months (or 2 months max)
Knowledge and exposure to distributed production systems i.e Hadoop
Nice to have:
Experience in Scala/Java is a plus (Polyglot programmer preferred!)
Knowledge and exposure to Cloud migration (AWS/GCP/Azure) is a plus
Welcome to CrawlJobs.com – Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
We use cookies to enhance your experience, analyze traffic, and serve personalized content. By clicking “Accept”, you agree to the use of cookies.