This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
We are seeking an experienced Backend Software Engineer to join our Crawler team. In this role, you will design, develop, and operate systems that ingest, process, and manage web-scale data in support of our next generation of advanced search technologies. This is a critical, high-impact engineering position, requiring expertise across both backend and frontend components of our data acquisition stack.
Job Responsibility:
Take ownership of and lead projects focused on developing large-scale web crawlers, ingestion pipelines, and data processing systems
Build, maintain, and optimize core backend and frontend components for crawler services, including storage, retrieval, and UI dashboards for data management
Collaborate closely with Search and Infrastructure teams to ensure the reliable, high-quality ingestion and processing of billions of web pages
Architect and implement fullstack features and scalable distributed systems that handle high-load and real-time data operations
Rapidly iterate, experiment with novel approaches, and continuously enhance system performance, usability, and reliability
Requirements:
Minimum of 5 years of software development experience
Strong knowledge of data structures and algorithms in at least one of the following languages: Python, C++, Rust, or Go
Experience with large-scale web crawlers is highly desirable
Proven experience building, deploying, and optimizing high-load, distributed, and hardware-adjacent services
Deep understanding of cloud infrastructure, with hands-on experience in Kubernetes (K8s) and AWS
Demonstrated passion for writing clean, efficient, and scalable systems