CrawlJobs Logo
Perplexity Logo Perplexity · IT - Software Development

Backend Software Engineer - Search, Crawler Team

· Job Posted February 21, 2026
Apply Position
Job Link Share

Job Description

We are seeking an experienced Backend Software Engineer to join our Crawler team. In this role, you will design, develop, and operate systems that ingest, process, and manage web-scale data in support of our next generation of advanced search technologies. This is a critical, high-impact engineering position, requiring expertise across both backend and frontend components of our data acquisition stack.

Job Responsibility

  • Take ownership of and lead projects focused on developing large-scale web crawlers, ingestion pipelines, and data processing systems
  • Build, maintain, and optimize core backend and frontend components for crawler services, including storage, retrieval, and UI dashboards for data management
  • Collaborate closely with Search and Infrastructure teams to ensure the reliable, high-quality ingestion and processing of billions of web pages
  • Architect and implement fullstack features and scalable distributed systems that handle high-load and real-time data operations
  • Rapidly iterate, experiment with novel approaches, and continuously enhance system performance, usability, and reliability

Requirements

  • Minimum of 5 years of software development experience
  • Strong knowledge of data structures and algorithms in at least one of the following languages: Python, C++, Rust, or Go
  • Experience with large-scale web crawlers is highly desirable
  • Proven experience building, deploying, and optimizing high-load, distributed, and hardware-adjacent services
  • Deep understanding of cloud infrastructure, with hands-on experience in Kubernetes (K8s) and AWS
  • Demonstrated passion for writing clean, efficient, and scalable systems
Perplexity - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Backend Software Engineer - Search, Crawler Team

8 matching positions

Senior Software Engineer, Data Acquisition

The Data Acquisition team within the Foundations organization at OpenAI is respo...
Location
Location
United States , San Francisco
Salary
Salary:
293000.00 - 385000.00 USD / Year
openai.com Logo
OpenAI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS/MS/PhD in Computer Science or a related field
  • 6+ years of industry experience in software development
  • Experience with large web crawlers a plus
  • Strong expertise in large stateful distributed systems and data processing
  • Proficiency in Kubernetes, and Infrastructure-as-Code concepts
  • Willingness and enthusiasm for trying new approaches and technologies
  • Ability to handle multiple tasks and adapt to changing priorities
  • Strong communication skills, both written and verbal
Job Responsibility
Job Responsibility
  • Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search
  • Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability
  • Work closely with the legal team to handle any compliance or data privacy-related matters
  • Develop and deploy highly scalable distributed systems capable of handling petabytes of data
  • Architect and implement algorithms for data indexing and search capabilities
  • Build and maintain backend services for data storage, including work with key-value databases and synchronization
  • Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks
  • Conduct and analyze experiments on data to provide insights into system performance
What we offer
What we offer
  • Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
  • Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
  • 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend to fuel your professional growth
  • Daily meals in our offices, and meal delivery credits as eligible
  • Fulltime
Read More
Arrow Right
New

Cleaner

Atlas FM are looking to recruit a Cleaner at our client’s premises. We are looki...
Location
Location
United Kingdom , Otley
Salary
Salary:
13.45 GBP / Hour
atlasfm.com Logo
Atlas Facilities Management Limited
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Attitude to thrive and cleaning experience (although we can fully train you from scratch if you match our positive and thoughtful approach)
  • Ability to carry out the physical aspects of the role
  • Basic level of spoken English
  • Good social skills to deliver service with a smile
  • Right to work in the UK
Job Responsibility
Job Responsibility
  • Deliver cleaning specification and use equipment
  • Provide high standard of customer care
  • Handle site-based equipment and cleaning materials
  • Replenish consumables such as paper towels, soap, toilet paper, bin liners
  • Empty litter bins into waste and recycling containers
  • Ensure cleaning completed to highest standards
  • Maintain Health and Safety standards
  • Work within a team that goes the extra mile
What we offer
What we offer
  • Full learning and support from Atlas
  • Stream financial wellbeing and benefits platform with real-time access to earned wages, financial health information, discounts at retailers and supermarkets
  • Career progression within Atlas
  • Learning and development support
  • Free uniform
  • Opportunity to work among people who value and support each other
  • Parttime
Read More
Arrow Right
New

Payroll Assistant

For August 2026, Cheltenham College seeks an organised and proactive Payroll Ass...
Location
Location
United Kingdom , Cheltenham
Salary
Salary:
22495.00 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
June 09, 2026
Flip Icon
Requirements
Requirements
  • Excellent organisational and administrative skills
  • Strong attention to detail and accuracy
  • Confident using Microsoft Excel and other IT systems
  • Strong communication and interpersonal skills
  • Ability to prioritise workload effectively and meet deadlines
  • Works well independently and collaboratively within a team
  • Demonstrates initiative and a willingness to learn new skills
  • Previous payroll or HR systems experience, although training may be provided for the right candidate
Job Responsibility
Job Responsibility
  • Preparing and processing monthly payroll information across a range of contract types
  • Inputting and maintaining payroll data for starters, leavers and contractual changes
  • Reviewing and processing timesheets, overtime and additional payments
  • Supporting statutory payments including sickness absence and family leave
  • Assisting with pension administration and monthly pension contributions
  • Responding to staff queries relating to pay and pensions
  • Producing payroll documentation including payslips, P45s and P60s
  • Supporting HMRC submissions including FPS and EPS returns
  • Assisting with payroll reporting, audit preparation and nominal ledger reports
  • Helping ensure compliance with payroll legislation, National Minimum Wage requirements and College policies
What we offer
What we offer
  • Lunch during term time
  • Executive Pension Scheme
  • Complimentary membership to College's gyms and pool
  • Discounts at local retailers
  • Health & wellbeing discounts
  • Hospitality venue discounts
  • Parttime
Read More
Arrow Right
New

Inclusive Learning Lecturer

Location
Location
United Kingdom , Swindon
Salary
Salary:
33000.00 - 43002.00 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A passionate educator committed to supporting learners with SEND or barriers to learning
  • Experience working with young people or adults in inclusive or SEND settings
  • A teaching qualification (or willingness to work towards one)
  • Strong communication, organisation and teamwork skills
  • A patient, supportive and student-centred approach to teaching
  • Enthusiasm for inclusive education and helping learners reach their full potential
Job Responsibility
Job Responsibility
  • Deliver engaging teaching across programmes including Personal and Social Development, Independent Living and Employability
  • Support learners with SEND and additional needs to develop confidence, independence and life skills
  • Plan and review individual learning programmes, ensuring EHCP support requirements are met
  • Track learner progress and provide feedback to support positive outcomes
  • Work collaboratively with parents, employers and external agencies to support learner progression
  • Deliver Functional Skills, life skills and employability-focused learning where required
  • Contribute to curriculum development and inclusive teaching practices
  • Support enrichment activities, trips and residential opportunities for learners
What we offer
What we offer
  • 11 weeks holiday inc bank holidays
  • 28.68% per
Read More
Arrow Right
New

Housekeeping Assistant

Job Opportunity: Housekeeping Assistant – JCA Condover Hall. Location: Condover ...
Location
Location
United Kingdom , Shrewsbury
Salary
Salary:
12.27 - 12.82 GBP / Hour
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Excellent attention to detail and pride in maintaining high standards
  • Able to work both independently and as part of a team
  • Reliable, responsible, and willing to learn new skills
  • Comfortable working in a fast-paced, physically active role
  • Previous experience in housekeeping or cleaning is desirable but not essential
Job Responsibility
Job Responsibility
  • Clean guest rooms and bathrooms to a high standard
  • Prepare beds and ensure fresh linen and supplies are ready for guest arrivals
  • Clean and maintain communal areas, including corridors, lounges, and the swimming pool area
  • Ensure storerooms are stocked with necessary linens, guest supplies, and cleaning products
  • Help decorate the centre for events and seasonal activities
  • Maintain health, safety, and cleanliness standards at all times
  • Ensure all products are used and stored correctly in accordance with COSHH
  • Work efficiently during guest arrival and departure days
  • Support other departments when required to ensure smooth operation across the site
  • Contribute to creating a welcoming and positive environment for all guests
What we offer
What we offer
  • Staff events and Friends & Family activity days
  • Employee discounts via Benefit Hub
  • Free DBS check
  • Parttime
Read More
Arrow Right
New

Service Engineer

Service Engineer – Renewables Sector £37,000 – £47,000 + Bonus + Overtime St Hel...
Location
Location
United Kingdom , St Helens
Salary
Salary:
37000.00 - 47000.00 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full UK driving licence required
  • Strong background in installation and/or service
  • Ability to read and interpret electrical schematics
  • Experience with testing and commissioning
  • Mechanically biased with solid electrical understanding
  • Comfortable working independently and taking ownership
  • ECS Gold Card (preferred)
  • 18th Edition
  • Level 3 qualification in Electrical or Mechanical preferred
  • Full UK driving licence (maximum 6 points)
Job Responsibility
Job Responsibility
  • Leading installation and commissioning activities on projects
  • Acting as the technical lead and key customer contact on-site
  • Taking ownership of planning (RAMS, delivery, coordination)
  • Making on-the-spot technical decisions and resolving issues
  • Supporting service and maintenance delivery
  • Maintaining strong customer relationships
  • Mentoring and supporting junior engineers
What we offer
What we offer
  • Bonus up to 10% monthly
  • Overtime (time and a half up to double time)
  • 24 days holiday + bank holidays
  • 6% pension employer contribution
  • Early finish Fridays
  • Enhanced family benefits
  • Company van, tools, discounts, wellbeing support
  • Hotels fully paid + daily allowances
  • Fulltime
Read More
Arrow Right
New

Assistant Management Accountant

We are seeking a capable and proactive Assistant Management Accountant to join o...
Location
Location
United Kingdom , Portsmouth
Salary
Salary:
33980.34 - 36035.26 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Part-qualified accountant (ACCA, CIMA, ACA, CIPFA or equivalent) or equivalent experience
  • Experience supporting the production of management accounts
  • Experience of working in a fast-paced finance environment with competing deadlines
  • Ability to work independently, use initiative, and quickly build confidence in your role
  • Good understanding of forecasting, budgeting, and financial analysis
  • Strong Excel skills and familiarity with financial systems
  • Good communication skills, with the ability to work effectively with non-finance stakeholders
  • Strong attention to detail and organisational skills
Job Responsibility
Job Responsibility
  • Play a key role in preparing monthly management accounts, including accruals, prepayments, and variance analysis
  • Support month-end processes, ensuring accuracy and adherence to deadlines
  • Contribute to financial forecasting and monitoring, including expected outturn reporting
  • Provide finance business partnering support to budget holders across pay, non-pay, income, and capital budgets
  • Assist with the annual budgeting process, working with stakeholders to ensure robust financial planning
  • Carry out financial analysis to support decision-making, including restructures and investment activity
  • Support cash flow monitoring and capital expenditure reporting
  • Assist with the preparation of financial statements, returns, and reporting to governors and senior leadership
  • Review monthly payroll and track against budgets and establishment
  • Maintain strong financial controls and compliance with procedures and audit requirements
What we offer
What we offer
  • Annual Leave: 25 days total and closure for winter shutdown period in line with Hampshire County Council school holidays & statutory bank holidays
  • Pension: Generous Local Government pension
  • Learning & Development: Programmes by The Community Adult Education Department
  • Staff Development: Access to the College staff development programmes and CPD Onsite
  • Discounts: City of Portsmouth College Hair Salon & 50% off at the City of Portsmouth College Sports Centre
  • Online Discounts: Unidays, Student Beans, and Totum with a City of Portsmouth College email address
  • Parking: Free car parking across all campus sites (apart from Arundel)
  • Blue Light Card: providing several discounts for clothing, electrical goods, food and days out, even concert tickets
  • Fulltime
Read More
Arrow Right
New

Customer Support Specialist

as a Customer Support Specialist EMEAI, you will be the go-to contact person for...
Location
Location
Netherlands , 's-Gravenhage
Salary
Salary:
2700.00 - 2900.00 EUR / Month
https://www.randstad.com Logo
Randstad
Expiration Date
May 26, 2027
Flip Icon
Requirements
Requirements
  • fluent in English and French OR English and German (both written and spoken)
  • previous experience in a customer service or office/B2B environment
  • familiarity with SAP and/or CRM tools (e.g., HubSpot) is a strong plus
  • detail-oriented
  • proactive
  • true team player
Job Responsibility
Job Responsibility
  • customer contact: handling daily customer inquiries via phone and email regarding product information, pricing, and orders
  • order management: entering orders accurately and maintaining up-to-date customer records in SAP
  • logistics & complaints: coordinating logistics and handling customer complaints professionally
  • team support: assisting regional sales managers and marketing specialists where needed
  • Parttime
Read More
Arrow Right