Webscraper.io Chrome Plugin

Why choose a Chrome plugin?

Actually there are a lot of SaaS that offer web scraping that can map out data or even enrich the data using AI. I tried Instant Data, it's very straightforward when you're dealing with a single layer data structure. I also tried Claywhich is marketed as CRM/ lead generator with extensive AI and API support.

In the end, I have successfully scrape the data I needed with Webscraper.io which the Chrome plugin is totally free for local use. Webscraper do provide multi-layer data support, so I can crawl list pages and it browse single page accordingly. As I only need the data locally, I didn't use its cloud service.

Scrapping List of Items on single page

Web Scraper create a new tab under Inspect Elements (Alt + Cmd + C), Start by "Create a Sitemap" and enter site name and URL. Then click "Add new selector". Each selector can be considered items to scrape or items contain the link require checking it (e.g. Paginations). Assume you are scrapping a list of items on a single page, you can use "Selector" and click on the card / item.

Scrapping with pagination

There are few ways to deal with pagination and lazy loading. If the page query is exposed by url, use variables for start url. In the "Edit Metadata" or "Start URL". Change a single page number ?p=1 into ?p=[1-10] Then the plugin will browse from page 1 to 10.

If it doesn't fulfill your requirement, use Link selector to save all the links within pagination component. Create a pagination selector and select parent selector to both Root and pagination will allow the scraper to discover other pages on its own.

Learn more, check page number pagination & next pagination or watch this pagination tutorial directly.