Open Brand Context, then Knowledge base. Scraping and crawling accept your own website only: your project's domain, with or without www, and its subdomains.

Scrape pages adds the pages you list. Choose Scrape pages, paste web addresses one per line (up to 50 at a time) and choose Scrape pages.

Crawl website adds many pages at once. Choose Crawl website, keep From the sitemap or choose From a start address, set the most pages to read (up to 300, 50 by default) and choose Start crawl. To limit a crawl, fill in Only these folders (for example /docs) or Never these folders (for example /blog). The crawl runs in the background.

Pages are read as plain text with no AI, and a website's robots.txt is respected.

Each project can start up to 5 crawls a month. To remove everything one crawl added, choose Delete N pages on that crawl in the Website crawls card.