Limits and freshness

How much an agent can hold#

Storage is counted per agent, not per workspace. There is no limit on the number of sources, only on their total size.

What counts is the size of what you add: the file as uploaded, the text of a page as read, the characters of a text or Q&A source. For a sense of scale, 1 MB of plain text is roughly 500 pages of a book. A few hundred web pages usually fit in 5 MB, because only the text is kept, not the images or the markup.

When an agent is full, a new source is refused and a crawl stops where it is and tells you so. Nothing already stored is removed.

Making room#

• Delete what the agent does not need. Sources you remove free their space at once. Their passages and the stored file go with them.
• Exclude paths when crawling. Archives, tag pages and job listings take space and make answers worse.
• Prefer text over PDF. A designed brochure of 4 MB often holds 20 KB of words.

Keeping it fresh#

ChatterLab does not re-read your website on a schedule. What the agent knows is what it read on the day you added it, until you change it.

• A page changed. Add it again as an Individual link. The agent already has that address, so the page is updated rather than added a second time.
• The site changed a lot. Crawl it again. Changed pages are updated, unchanged ones are left alone and new ones are added. Pages that have gone from the site stay until you delete them, because a crawl never removes anything.
• A document changed. Delete the old file and upload the new one. Do not keep both.

A sensible habit is to look at Overview once a month. Questions that suddenly score low often point at something that changed on your side.

Retrain agent#

The Retrain agent button re-processes everything the agent already has: every source is cut up and indexed again from the content we stored. It does not fetch your website again. You rarely need it. It is there for when indexing itself has improved.

What is removed automatically#