CrawlJobs Logo

Web Scraping / Data Acquisition Engineer

India, Mumbai · Job Posted May 03, 2026
Apply Position
Job Link Share

Job Description

Wissen Technology is hiring for Web Scraping / Data Acquisition Engineer. We are looking for a skilled Web Scraping / Data Acquisition Engineer with 3–7 years of experience to build robust data extraction pipelines for collecting legal data from public websites. The role involves designing crawlers to extract court judgments, tribunal orders, and regulatory decisions, storing structured metadata, and automating monitoring for new content. The ideal candidate has strong Python skills, hands-on web scraping experience, and the ability to handle large volumes of documents and structured data.

Job Responsibility

  • Design and develop web crawlers to extract data from public websites
  • Crawl listing pages and extract case metadata (case title, number, court, date, etc.)
  • Download judgments and maintain structured PDF/document storage
  • Build automated pipelines to monitor websites and detect new judgments
  • Extract structured data from documents and HTML pages
  • Store data in structured formats suitable for downstream processing or search
  • Handle pagination, anti-bot measures, and data cleaning workflows
  • Maintain scrapers for reliability, accuracy, and long-term scalability

Requirements

  • Strong hands-on experience with Python
  • Proven experience in web scraping and crawler development
  • Proficiency with browser automation tools: Playwright, Scrapy, or equivalent
  • Experience with PDF extraction tools (pdfplumber, PyMuPDF, Apache Tika, etc.)
  • Strong understanding of HTML parsing, pagination handling, and automated file downloads
  • Knowledge of anti-bot techniques (rate limiting, proxy handling, session rotation)
  • Experience processing structured and semi-structured documents

Nice to have

  • Experience with large-scale crawlers or distributed scraping
  • Working experience with document datasets and text-heavy systems
  • Familiarity with Apache Tika / advanced PDF extraction
  • Experience with AWS S3 for storing large volumes of raw documents
  • Exposure to Elasticsearch or search indexing systems
  • Experience with Kafka / AWS MSK for event-driven pipelines
  • Background in legal, regulatory, or compliance datasets (optional)

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Web Scraping / Data Acquisition Engineer

8 matching positions

Senior Software Engineer - Data Acquisition

Join TxODDS as a Senior Software Engineer and help build scalable, high-performa...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
txodds.net Logo
TXODDS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience with at least one core programming language (e.g. Python, Java, Scala)
  • Hands-on experience with Kubernetes, container orchestration, and Docker
  • Experience working with distributed systems and event‑driven technologies (e.g. Kafka)
  • Solid understanding of networking fundamentals (HTTP, APIs)
  • Experience with relational and NoSQL databases
  • Strong Git skills and familiarity with modern development practices (code reviews, testing, CI/CD)
  • Comfort working in a Linux/Unix command-line environment
  • Experience designing and debugging software from inception to deployment
  • Excellent problem‑solving skills and a proactive approach to improving systems and processes
  • Strong communication and collaboration skills, and the ability to work effectively across teams
Job Responsibility
Job Responsibility
  • Developing, testing, and deploying high‑quality software that processes data from diverse sources
  • Building, improving, and maintaining distributed systems and data pipelines (including Kafka-based services)
  • Deploying and supporting containerised workloads running in Kubernetes environments
  • Creating and maintaining clear, accurate documentation for the systems you build
  • Validating and monitoring data quality using internal tools and processes
  • Supporting data‑gathering workflows, including those involving web‑scraping or automated data acquisition
  • Investigating and resolving data‑related issues escalated from the Client Services team
  • Participating in an out‑of‑hours on‑call rotation to support critical data acquisition systems
  • Sharing knowledge widely and contributing to a positive, collaborative team culture
  • Mentoring junior engineers and helping raise the overall technical bar
What we offer
What we offer
  • Competitive benefits package tailored to your location
  • Fulltime
Read More
Arrow Right

Senior Technical Recruiter

As a key member of Anduril's software recruiting team, you will be responsible f...
Location
Location
United States , Costa Mesa; Atlanta
Salary
Salary:
70.00 - 80.00 USD / Hour
anduril.com Logo
Anduril Industries
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years experience recruiting and sourcing for technical roles and closing top tier engineering talent (startup/software experience preferred)
  • Experience hiring candidates within facets of robotics (computer vision, perception, motion planning, localization, mapping, etc.)
  • 4+ years of client interaction experience, including working with Hiring Managers and Directors, taking new requirements, reviewing profiles and updating pipeline progress
  • Experience acting in a consultative manner where your guidance has led to improved outcomes, and a positive candidate experience, while earning the respect of your peers and clients
  • Strong understanding of the technical skills and experience required for Engineering positions within the business
  • Clear and effective communication with candidates to understand motivation, drivers, and fit within the organization
  • Experience managing various funnels of candidates and keeping track of their progress throughout the recruitment process
  • Working knowledge of applicant tracking and HRIS systems
  • Working knowledge of interview techniques and applicant screening methods
  • Familiar with a wide variety of sourcing avenues
Job Responsibility
Job Responsibility
  • Move fast, with a process for consistently sourcing, tracking and maintaining a high volume of candidates while maintaining open lines of communication with stakeholders
  • Establish deep trust and partnership with business leaders to help influence talent strategy while executing on hiring deliverables
  • Build trust through consisteny and a strong operating rhythm in partnership with cross-functional stakeholders to help scale and manage company growth
  • Embed yourself into the business and into your teams to understand the product positioning and market fit, technical roadmap, and culture of the team
  • Develop a strong understanding of the mission while learning how to effectively pitch the team and opportunity, ultimately closing exceptional technical talent for the organization
  • Track and analyze pipeline and performance data to gain insights into areas of opportunity and translate that into a data-driven narrative that enables increased momentum
  • Build recruiting strategies that contribute to the long-range growth of the company, implementing best practices around referrals and process improvements where needed
  • Act as a subject matter expert in prospecting techniques and tools used for information retrieval, data extraction, web-scraping, continuous process improvement, process automation, and candidate management
  • Conduct interviews of potential candidates, demonstrating ability to anticipate hiring manager preferences through high interview-to-offer ratios
  • Engage passive candidates using Linkedin Recruiter, Gem, Boolean strings, referral and SOBO campaigns
What we offer
What we offer
  • Comprehensive medical, dental, and vision plans at little to no cost to you
  • Income Protection: Anduril covers life and disability insurance for all employees
  • Generous time off: Highly competitive PTO plans with a holiday hiatus in December. Caregiver & Wellness Leave is available to care for family members, bond with a new baby, or address your own medical needs
  • Family Planning & Parenting Support: Coverage for fertility treatments (e.g., IVF, preservation), adoption, and gestational carriers, along with resources to support you and your partner from planning to parenting
  • Mental Health Resources: Access free mental health resources 24/7, including therapy and life coaching. Additional work-life services, such as legal and financial support, are also available
  • Professional Development: Annual reimbursement for professional development
  • Commuter Benefits: Company-funded commuter benefits based on your region
  • Relocation Assistance: Available depending on role eligibility
  • Traditional 401(k), Roth, and after-tax (mega backdoor Roth) options
  • Fulltime
Read More
Arrow Right

Technical Recruiter, Production Engineering

As a key member of Anduril's Talent Acquisition team, you will be responsible fo...
Location
Location
United States , Ashville
Salary
Salary:
40.00 - 50.00 USD / Hour
anduril.com Logo
Anduril Industries
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of high-volume talent acquisition experience evaluating and hiring top tier talent
  • 1+ years of experience working with senior level leaders and stakeholders, reporting on progress, creating and advising on talent strategies
  • Experience recruiting at an agency and fast-paced startup, hiring exempt roles for Hardware, Production, or Manufacturing Engineering
  • Ability to be onsite/hybrid at Ashville, OH office
Job Responsibility
Job Responsibility
  • Act as a subject matter expert in prospecting techniques and tools used for information retrieval, data extraction, web-scraping, continuous process improvement, process automation, and candidate management
  • Own and drive hiring strategy full-cycle, providing effective, clear communication and reporting
  • Conduct a high volume of interviews, demonstrating ability to anticipate and influence hiring manager preferences through successful interview-to-offer conversion ratios
  • Engage and source passive candidates using LinkedIn Recruiter, Boolean strings, referrals and SOBO campaigns
  • Build talent maps to generate market insights to inform your engagement strategy
  • Drive diverse talent into the organization ensuring a positive candidate experience at every touchpoint
  • Represent the company's brand and recruiting team internally and externally at the highest caliber
  • Leverage internal resources, team mates, and cross functional partners to build strategy around selling our value proposition and impacting hiring practices
What we offer
What we offer
  • Comprehensive medical, dental, and vision plans at little to no cost to you
  • Income Protection: Anduril covers life and disability insurance for all employees
  • Generous time off: Highly competitive PTO plans with a holiday hiatus in December. Caregiver & Wellness Leave is available
  • Family Planning & Parenting Support: Coverage for fertility treatments, adoption, and gestational carriers
  • Mental Health Resources: Access free mental health resources 24/7
  • Professional Development: Annual reimbursement for professional development
  • Commuter Benefits: Company-funded commuter benefits based on your region
  • Relocation Assistance: Available depending on role eligibility
  • Retirement Savings Plan: Traditional 401(k), Roth, and after-tax (mega backdoor Roth) options
  • Fulltime
Read More
Arrow Right

Growth Engineer

You will be involved end-to-end in building tools and systems that enable Market...
Location
Location
Italy , Milan
Salary
Salary:
Not provided
satispay.com Logo
Satispay
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong Python expertise – Proven ability to develop, test, and maintain production-grade applications and services
  • Applied AI / automation mindset – Interest in AI/ML and automation with a focus on translating ideas into concrete, business-ready solutions
  • Web & front-end fundamentals – Experience building and maintaining websites, landing pages, or widgets
  • comfortable with basic web development concepts
  • Growth & marketing awareness – Basic understanding of digital marketing, funnels, and conversion rate optimization (CRO)
  • Impact-driven attitude – Strong curiosity and intrinsic motivation to work on projects that move real business metrics
  • Excellent problem-solving skills – Ability to navigate ambiguity, break down complex problems, and deliver pragmatic solutions
Job Responsibility
Job Responsibility
  • Build and scale growth tools – Design, develop, and maintain internal MarTech and growth tools (e.g. lead enrichment/prioritization, AI agents, website scraping, CRM tools, competitor monitoring and conversion enablement) to support Sales and Marketing efficiency and conversion
  • AI & automation for growth – Identify and implement AI solutions, automation, and advanced logic solutions that solve concrete business problems
  • Website development & optimization – Build and maintain websites, widgets, and landing pages, supporting experimentation and conversion rate optimization across acquisition funnels
  • Lead technical initiatives – Own technical projects end-to-end, from ideation and MVP to production, working closely with Marketing, Sales, RevOps, and Data teams
  • Innovate & explore – Continuously explore new tools, APIs, and technologies, making thoughtful build-vs-buy decisions to maximize impact and speed
What we offer
What we offer
  • Unlimited paid time off
  • Psychological support & mental health webinars with Serenis
  • Flexible hybrid working system
  • Extended parental leave
  • Childcare leave
  • Professional development programmes
  • Internal mobility program
  • Language classes with Preply
  • Internal workshops & training
  • Stock Option Plan (with additional grants often provided based on performance)
  • Fulltime
Read More
Arrow Right

Research Grants AI Internship

Join our AI team as a Research Grants AI Intern and contribute to building intel...
Location
Location
Switzerland , Basel
Salary
Salary:
Not provided
mdpi.com Logo
MDPI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Currently pursuing or recently completed a BSc or MSc in Data Science, Machine Learning, Software Engineering, or a related field
  • Initial practical experience in data acquisition, data processing, AI, or related areas
  • Basic knowledge of web scraping, API integration, and Python for data-intensive applications
  • Understanding of AI principles and interest in NLP or text mining
  • Familiarity with databases and data modeling concepts
  • Fluent in English (minimum B2)
  • Strong analytical and problem-solving skills
  • Ability to explain technical concepts to non-technical stakeholders
  • Comfortable working both independently and in an agile team environment
  • Curiosity about academic publishing and research ecosystems
Job Responsibility
Job Responsibility
  • Build workflows for crawling, extracting, and indexing grant information from public sources
  • Normalize and structure grant metadata (e.g., funding agency, grant IDs, investigators, timelines, keywords)
  • Develop and implement text-mining and metadata-matching methods to link grants with publications
  • Integrate grant data into internal databases and graph-based systems
  • Support the exploration of predictive models to identify research trends and funding opportunities
  • Contribute to potential integrations with internal analytics and editorial tools
What we offer
What we offer
  • The opportunity to contribute to the academic/scientific community
  • Flexible working hours
  • Team bond strengthening through team-building events
  • Professional growth opportunities with our global training system
  • Working in a collaborative and socially responsible team
  • Company retreat facility
  • Full-coverage insurance for accidents/daily sickness
  • Prime location near Basel train station and city center
  • Fulltime
Read More
Arrow Right

Lead Growth Engineer

Socure is on the search for a Growth Engineer—an entrepreneurial technologist wh...
Location
Location
United States
Salary
Salary:
150000.00 - 190000.00 USD / Year
socure.com Logo
Socure
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience with growth tools like Clay, Apollo, Clearbit, Instantly, or Hypergrowth, including multi-step outbound and enrichment flows
  • Full-stack development skills (e.g., JavaScript, Python, React) to build growth experiments and internal tools
  • Skilled in using AI copilots and agents to personalize messaging, automate outreach, and surface insights from CRM data
  • Comfortable working with APIs, webhooks, and automation platforms (Zapier, Retool, Make, etc.) to connect and scale GTM systems
  • Working knowledge of SQL and audience segmentation for performance measurement and optimization
  • Hands-on experience building or integrating AI agents to accelerate campaign creation, onboarding, and user research
  • Ability to generate dynamic, cross-vertical messaging at scale, personalized by industry, use case, or persona
  • Familiar with LLM tools (ChatGPT, Claude, custom embeddings) to create content and power conversion workflows
  • Track record of designing and shipping growth experiments across acquisition, onboarding, and retention
  • Confident optimizing funnels, running A/B tests, and driving data-led iteration cycles
Job Responsibility
Job Responsibility
  • Scale our sales and marketing functions through automation, AI assistance, and accessible insights
  • Own campaign automation with logic for lifecycle drips, and new GTM workflows
  • Automate prospecting with creative, never-seen-before outreach tactics
  • Build viral tools, shareable demos, and mini-products that drive organic traffic and referrals across industries like fintech, government, and eCommerce
  • Act as an internal multiplier, sharing tools, playbooks, and internal agents that help marketing and GTM teams move faster
  • Craft compelling copy and UI elements that effectively communicate our value proposition
  • Launch interactive landing pages, calculators, or integrations that showcase our AI/ML identity graph and verification capabilities
  • Partner in automating meaningful customer communications through community slack channels, as well as Substack and LinkedIn communities
  • Create AI agents that automate flows from product design to go-to-market, capturing the minds and hearts of new customers and driving cross-sell within existing ones
  • Change the game in competitive positioning by designing agents that continuously monitor the market, scraping for negative sentiment and/or market shifts that can be leveraged to out-position the competition
What we offer
What we offer
  • Offers Equity
  • Offers Bonus
  • Fulltime
Read More
Arrow Right
New

IT Training Lead

The IT Training Lead will drive technology learning and user adoption across the...
Location
Location
United States , Delray Beach
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience in IT training, instructional design, technical enablement, or learning and development
  • Strong knowledge of Microsoft 365
  • Excellent communication, facilitation, and content development skills
  • Ability to translate technical concepts into practical, user-friendly training.
Job Responsibility
Job Responsibility
  • Design, develop, and deliver IT training programs in instructor-led, virtual, and self-paced formats
  • Take lead in the Microsoft Copilot and AI training strategy, including onboarding, advanced use cases, responsible AI usage, and ongoing enablement
  • Partner with IT leadership to support new technology rollouts, system upgrades, and digital transformation initiatives
  • Create and maintain training content, including videos, guides, tutorials, and job aids
  • Identify skill gaps and develop targeted learning solutions to improve adoption and productivity
  • Gather feedback and measure training effectiveness to continuously improve programs.
Read More
Arrow Right
New

K Kitchen Representative

The position includes, but is not limited to, the following essential job duties...
Location
Location
United States , New Albany
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Excellent communication skills
  • Team player who can work well with others or independently
  • Acts with integrity
  • keeps commitments
  • Contagious positive attitude
  • Focuses on achieving results while having fun
  • Frequently bend, twist at waist, kneel, squat, stand, and walk
  • Occasionally climb and descend ladders
  • Tolerate extreme cold and hot temperatures and work in and around fryers, ovens, grills, coolers, freezers, sharp objects, and loud noises
  • Reach, grasp, and manipulate objects with hands for entire shift, including reaching for objects overhead
Job Responsibility
Job Responsibility
  • Provides excellent guest service in a fast and friendly manner
  • Maintains a clean restaurant environment by cleaning and performing general housekeeping duties
  • Prepares and serves food items in accordance with all Brand, Company, and health department regulations
  • Ensures product quality, food safety, and operational standards are met
  • Keeps accurate cash, sales, and inventory control records
  • Follows all government laws and safety codes
  • Completes reports on all incidents following our 5-minute rule policy
  • Lives our Company values: One Team, Do the Right Thing, Takes Ownership, Play to Win
What we offer
What we offer
  • Medical, Dental, Vision, Term Life and AD&D plans
  • Flexible spending and health savings accounts (FT)
  • Vacation paid time off
  • Company holidays paid at time and a half
  • Matching 401(k)
  • Tuition Reimbursement
  • Stock Purchase Plan
  • Employee Discount Program
  • Discount Meal Benefit
  • Wellness Plan
Read More
Arrow Right