Skip to main content
Editor’s note: If you want to crawl a website instead, we suggest going to sitemap or crawler

How many URLs can I add at a time?

You can paste up to 100 URLs at a time.

How do I add a URL to my knowledge base?

  1. Go to the Knowledge Base of the Library you want to add a knowledge to.
  2. Click Add Knowledge button.
Add Knowledge
  1. Select Scrape with links from the Add Knowledge window.
Scrape with links
  1. Paste the URLs you want to scrape in the field. Click Add more if you want to add more URLs.
Link
  1. Choose what scrape type you want to use by clicking on the button of the option you want to choose.

Regular Scraping

It will scrape all the content in the page including the unnecessary content like the header, footer, and other unnecessary content.
Regular Scraping

Regular Scraping Output

AI Scraping

Libraria’s AI will scrape and summarize the most important information. Output will not inlcude the unnecessary content.
AI Scraping

AI Scraping Output

  1. Click Submit button. Once submitted, a window will appear showing how much credits you will use to scrape.
The credits used for the Regular Scraping will depend on how much content is in the page. The more content, the more credits you will use.
Regular Scraping Credits

Regular Scraping Cost

The credits used for the AI Scraping also depends on content volume but it will be more costly than the regular scraping, five times more costly to be exact.
AI Scraping Credits

AI Scraping Cost

  1. Click on the Submit button in the window to confirm the scraping and add it to your library.

It’s scraping unnecessary content. How do I refine what I scrape?

Option 1: Advanced

You can open the Advanced accordion and use selectors to select the content you want to scrape in the URL.
  1. Open the Advanced accordion by clicking on it.
Open Advanced
  1. Input the selectors of the content you want to scrape in the text field. Thse selectors are used to select the content you want to scrape in the URL.
For example, you have a page that has the following code section:
Sample HTML Code
And you only want to scrape the text in Inclusive Design and Prior Art. You can use the following CSS selector to scrape those specific content.
CSS Selector
  1. Click on Submit button.
Submit
Once submitted, a window will appear showing how much credits you will use to scrape.
Submit Window
An error will appear in the window if the selector you inputted is invalid. Make sure that the selector you’re using is valid to proceed.
  1. Click on the Submit button in the window to confirm the scraping and add it to your library.
Confirm Submit

Advanced Scraping Output

Option 2: Use AI Scraping

Libraria’s AI will scrape and summarize the most important information. Refer to the AI Scraping section above for more information.