Websites
Crawl public pages so your assistant stays aligned with your site.
Websites let you add URLs that Chatloop crawls on a schedule. Crawled text becomes retrievable knowledge for the selected agent.
Crawl depth
Choose whether to index a single page or follow links from the starting URL, depending on how much of the site you want in scope. Broader crawls take longer and use more resources.
Recrawl schedule
Options typically include once, daily, weekly, or monthly. Match frequency to how often your marketing or docs change. Static legal pages need fewer recrawls than pricing or changelog areas.
Status
You will see pending, processing, completed, or failed states. Failures often come from blocked pages, redirects, or robots rules—verify the URL in a browser and adjust scope.
Pages crawled
The UI shows how many pages were collected. If the count is unexpectedly low, check depth, filters, and whether the site requires authentication (public crawl cannot log in).