> ## Documentation Index
> Fetch the complete documentation index at: https://docs.libraria.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Sitemap XMLs

> Upload all the pages that matter in your site

## How do I add my sitemap to Libraria?

1. Go to `Knowledge Base` > `Add knowledge` > `Sitemap`
2. Paste your sitemap url

## Why should I use a sitemap instead of crawling a website?

1. **Efficiency**: A sitemap provides a clear roadmap of all the URLs on a website, saving time and resources as the scraper doesn't need to discover these URLs by following links.
2. **Completeness**: Sitemaps list all intended public pages, ensuring no important pages are missed, even if they aren't linked from other parts of the site.
3. **Prioritization**: Sitemaps can provide a `<priority>` tag, helping scrapers determine which pages to scrape first.
4. **Reduced Risk of Getting Blocked**: Using a sitemap makes scraping activities resemble the behavior of search engines, reducing the risk of being seen as suspicious and getting blocked.

## How do I find my sitemap?

Finding your website's sitemap depends on how your website was created and hosted. Here are some general steps and places to check:

<Note>
  TLDR: Most sitemaps are located in `https://<your_website>.com/sitemap.xml` or `https://<your_website>.com/sitemap_index.xml`. If your website does not have a sitemap, you can use a tool like <a href="https://www.xml-sitemaps.com/" target="_blank">XML-Sitemaps.com</a> to generate one or [crawl a website instead](/libraries/adding-documents/crawl).
</Note>

1. **Default Sitemap URL**: Most platforms and CMS systems have default locations for sitemaps.
   * Example: If your website is `https://example.com`, try accessing `https://example.com/sitemap.xml`.

2. **Content Management System (CMS)**:
   * **WordPress**: If you're using WordPress, and you've installed an SEO plugin like Yoast or All in One SEO, they often generate a sitemap for you. You can find it at `https://example.com/sitemap_index.xml` (for Yoast).
   * **Other CMS**: Check the documentation or settings of your CMS. There's often a section dedicated to SEO or sitemaps.

3. **Webmaster Tools**:
   * If you've submitted your sitemap to search engine webmaster tools (like Google Search Console or Bing Webmaster Tools), you can find the sitemap URL there.

4. **Robots.txt**:
   * Sitemaps are sometimes referenced in the `robots.txt` file. Try accessing `https://example.com/robots.txt` and see if there's a sitemap URL mentioned there.

5. **Website Footer or Header**:
   * Some websites link to their sitemap from the footer or header for user accessibility.

6. **Ask Your Web Developer or Hosting Provider**:
   * If you hired someone to develop your website or if you're using a hosting provider that offers website building tools, they might know where the sitemap is located.

7. **Search for It**:
   * If you're unsure of the exact URL, you can try doing a site-specific search on a search engine like Google. Type `site:example.com sitemap.xml` into the search bar and see if any results come up.

8. **Manual Generation**:
   * If you can't find your sitemap, it's possible you don't have one. In that case, there are many online tools and plugins available that can help you generate a sitemap for your website.

9. **Check Website Source Code**:
   * Sometimes, the sitemap link is included in the source code of the website. Right-click on the homepage and select "View Page Source" or a similar option, then use the browser's "Find" function (usually Ctrl+F or Cmd+F) and search for "sitemap".

10. **Check .htaccess or server configuration**:

* If you have access to the server or hosting environment, check the `.htaccess` file (for Apache servers) or other server configuration files to see if there's a rewrite rule or redirection related to the sitemap.

Remember, not all websites have a sitemap, if your website does not have one, you can use a tool like [XML-Sitemaps.com](https://www.xml-sitemaps.com/) to generate one or [crawl a website instead](/libraries/adding-documents/crawl).