Confidence and grounding
The two measurements on every answer, what each one can and cannot tell you, and how to read them together.
An agent that sounds sure tells you nothing about whether it was right. So every answer is measured twice, by two methods that fail in different ways.
Confidence: did your content cover the question?
Confidence is a number from 0 to 100, measured on the search result, before the answer is written. It asks one thing: did we find something in your sources that really is about this question?
It looks at how good the best passage is, and at how far that passage stands out from the ones ranked below it. A question your content covers produces a clear winner. A question it does not cover produces a row of passages that are all equally beside the point.
| Confidence | Shown as | Meaning |
|---|---|---|
| 70 and up | Well covered | Your sources contain this. |
| 50 to 70 | Partly covered | Something related was found. The answer may be incomplete. |
| below 50 | Poorly covered | Your sources had little or nothing. A good agent will have said so. |
The scale is calibrated on real questions, not invented: 50 sits at the measured line between questions a knowledge base could and could not answer.
What it cannot see. Confidence is measured before the model writes, so it is blind to what the model then does. Perfect passages can still be followed by an answer that adds something of its own. That is what the second measurement is for.
Grounding: did the answer stay with your content?
For every answer with a confidence of 50 or more, a separate check is made after the answer is written. A model that sees only the passages and the answer judges whether the one is backed by the other:
- supported: every factual claim is backed by the passages
- partial: some claims are, at least one is not
- unsupported: the main claim is not in the passages, or contradicts them
On the Overview, partial appears as Partly backed and unsupported as Not backed.
Answers below 50 are not checked: they are already flagged as weak, and a second flag adds nothing. For those, grounding is simply empty. It is also empty in the rare case that the check itself failed. An empty verdict never means "fine", only "not checked".
The check costs you no credits, and in the widget it runs after the visitor has their answer, so it slows nothing down.
What it cannot see. The checker does not read your instructions. If an answer correctly uses a fact from your instructions, such as your phone number, the checker finds no passage for it and may call the answer partial. When you see a partial verdict, read the answer before you conclude anything. And it is a model's judgement, not a proof: it catches most unsupported claims, not every one.
Reading them together
| Confidence | Grounding | What happened | What to do |
|---|---|---|---|
| high | supported | The content was there and the agent used it. | Nothing. |
| high | partial or unsupported | The content was there, and the agent said more than it. | Tighten the instructions, lower the temperature, or add the missing detail as a source. |
| low | empty | The content was not there. | Add a source. |
The middle row is the important one. Without a second measurement these answers look like your best ones.
Where you see them
- Playground: confidence under each answer, with the sources used.
- Activity: confidence on every answer in a conversation, and the grounding verdict on the answers the check disagreed with.
- Overview: the share of well, partly and poorly covered answers, and the list Worth adding to your sources, where unsupported answers come first.
- API: every answer carries
confidenceandgrounding. See the API reference. - Visitors never see either number. The widget shows the answer only.