Fixing weak answers

An agent gets good by being corrected. The list Worth adding to your sources at the bottom of Overview is where you start: it shows the questions your sources answered worst, and the answers that went further than their sources. Open one to read the whole conversation.

Work through the steps in this order. Each one is cheaper than the next.

1. Read the question and the answer#

Is the answer actually wrong? A low confidence on "do you also sell cars?" at a bicycle shop is not a problem. The agent said it did not know, which is correct. Skip it.

Look at what the visitor really asked. People ask in their own words, and those are often not yours.

2. Is the information in your sources at all?#

This is the cause nine times out of ten. Search your own sources for the answer.

It is not there. Add it:

• one specific question: a Q&A pair, with the question worded the way the visitor worded it. The quickest way is Improve answer under the answer in Activity, which starts from the visitor's own question
• a whole subject: a text source, or the page on your site that should have existed

It is there, but the page was never read. Check Websites for why a crawler misses pages, and add the page as an Individual link.

It is there, in a scanned PDF. Then it is not there. See Files.

3. It is there, but it was not found#

Open the conversation in Activity and look at the sources under the answer. If the right source is missing from them:

• Wording. Your page says "returns procedure", the visitor says "send it back". The search works on meaning and usually bridges this, but a Q&A pair in the visitor's words always does.
• Buried. The answer is one sentence in a long page about something else. Lift it out into a Q&A pair or a short text of its own.
• Drowned out. Ten blog posts mention the subject in passing and outrank the one page that explains it. Delete or exclude what the agent does not need.
• Minimum similarity too high, if you raised it. See Retrieval settings.

4. It was found, but the answer went beyond it#

This is the Partly backed or Not backed badge: good passages, and an answer that added something.

• Lower the temperature to 0.3 or so.
• Say it in the instructions: "If the sources do not state a price, say you do not know the price. Never estimate."
• Fill the gap. Models embellish most where a source stops just short of what was asked. If the page gives the delivery time for Belgium and the visitor asked about France, add France.
• Check for a conflict. Two sources that disagree, such as an old and a new price list, produce answers that are half wrong. Delete the old one.
• If the verdict concerns a fact that is in your instructions, the answer may be fine. The checker does not read instructions.

5. It was found, and the answer is still poor#

• Too long, too stiff, wrong tone: that is an instructions matter.
• Mixes two products up: lower Top-K results so that it reads fewer passages at once.
• Misses half of a two-part question: raise Top-K results, or try a stronger model in the Playground.

6. Check that it worked#

Ask the same question in the Playground. Confidence should be up, and the right source should be listed under the answer. Then ask it once more in different words.

Make it a habit#

Ten minutes a week in the first month, then once a month. The list gets shorter quickly, because visitors ask the same things.

If you use an AI assistant such as Claude, it can do the reading for you. Over MCP it can fetch the weak answers, propose the missing Q&A pairs, and add them once you have approved them.