Grounded answers.

If it can't point to the page the answer came from, it should say it doesn't know.

01

The analogy

A good student answers with the page number: "it's on page 42". When the page they open only half covers the question, they say they aren't sure. A careless one points to any page and hopes nobody checks.

Here the student is an assistant that answers from a manual of sixteen passages. Each answer cites the passages it used. The slider decides how much of the question those passages must cover before the assistant may answer.

In the diagram below

the book
16 passages of a policy manual
the student
the assistant that answers
citing the page
linking each answer to its passages
"I'm not sure"
a refusal when coverage is too low
02

Try it

Twenty-four questions about the manual. Twenty-one have their answer in it; three ask about things the manual never mentions.

Your move / Predict, then test

Before you run it, place a bet: requiring 30% of the question to be covered, does every answer cite a page that really has it?

Choose your prediction, run the experiment, then reveal the result. You can also explore without predicting.

Each square is one question, in order. Green answered from the right page, blue said it wasn't sure, red answered from a page that doesn't have the answer.

Place your bet and press Run it.

03

With your bar or none

Same questions, same manual. One side uses your coverage bar; the other answers whenever it has any citation at all.

04

Where it fits

Worth it

When staff or customers act on what an assistant says and someone must be able to check where it came from. I picture a support desk answering from internal policies.

Not needed

For brainstorming or drafting, where nobody expects each sentence to come from a document.

05

What it proves

I treated a refusal as a good answer when the evidence is thin. The bar makes that trade visible: raise it and wrong pages disappear, raise it too far and good answers turn into refusals.

For engineers
  • Hybrid retrieval: BM25 and TF-IDF merged by reciprocal rank fusion, top 3 passages. Each claim cites a retrieved passage and is checked for grounding and support.
  • Coverage is the share of question terms (four or more letters or digits, accents kept) found in the retrieved passages. Accented words used to be cut in half; that is fixed.
  • Coverage per question is pinned in a fixture read by the TypeScript and Python suites. The retrievers and answerer run in TypeScript only.
  • Stack: Next.js 16, TypeScript, Python, Vitest, pytest.
Source code →