Ground everything: a practical guide to citation provenance
Citation isn't a UX layer on top of generation. It's the contract. Here's the architecture we use to make every generated sentence traceable.
Provenance as a contract
If a generated sentence cannot be traced to source material, it is not an answer. It is a hypothesis with good grammar. In consumer contexts that is often acceptable. In an enterprise context, where the output feeds a decision that someone signs, it is not.
Treating citation as a presentation concern — something added to the output after generation, as a UX affordance — produces systems that cite plausibly and incorrectly. The link points at a document that broadly concerns the topic, the reader assumes it supports the specific claim, and nobody checks until an auditor does.
The alternative is to treat provenance as a contract that the system is structurally incapable of violating: the generation step cannot emit a claim without also emitting the span it came from, because the output schema does not permit it.
Span-level retrieval, not document-level
Document-level retrieval returns the file. Chunk-level retrieval returns a paragraph. Neither is precise enough to defend a sentence, because both leave the reader to locate the supporting text themselves — and readers, reasonably, do not.
Span-level retrieval keeps character offsets into the source through the whole pipeline. When a passage is chunked, the chunk carries its origin: document identifier, version, and the exact byte or character range it occupies. When that chunk is passed to the model, the identifier travels with it.
The engineering cost is mostly bookkeeping, and it is paid at ingestion. The offsets have to survive text extraction, normalisation, and chunking, which means the extraction step cannot silently rewrite whitespace or strip headers without recording what it did. Getting this right at ingestion is far cheaper than reconstructing it at query time.
Putting citation in the output schema
The mechanism that makes provenance non-optional is structured output. Rather than asking a model to write prose and append references, you constrain it to emit a list of claims, each with the identifiers of the spans that support it.
Two properties follow. First, a claim with an empty support list is a schema-level detectable failure rather than a subtle one — you can reject it before a human ever sees it. Second, the rendering layer becomes trivial and honest: the interface displays what the model asserted was its evidence, not a post-hoc guess about it.
Validation runs on top. Each cited span is checked to exist, to belong to a document the requester is entitled to read, and to still match its recorded content — a version check that catches the case where the source changed after indexing. Failing any of those is an error, not a warning.
Reviewing at the citation level
The reviewer surface should let a human accept or reject at the level of a claim and its evidence, not at the level of an entire answer. Whole-answer approval forces a binary judgement on a paragraph that is usually mostly right, which trains reviewers to approve.
Claim-level review is faster, because the reviewer reads one assertion against one highlighted span rather than reconstructing the whole chain. It is also better telemetry: rejections tell you which kinds of claims the system gets wrong, which is directly actionable, whereas a rejected answer tells you almost nothing.
That telemetry is the input to the evaluation set. Every rejected claim is a case worth adding, with the correct answer recorded, and the set grows from real failures rather than from imagination.
What this does not solve
Grounding constrains a model to source material. It does not make the source material correct, current, or complete. A perfectly grounded answer drawn from a superseded policy document is precisely wrong, and it is wrong with a citation attached, which makes it more persuasive.
So provenance has to extend to freshness: the version of the document, when it was indexed, and whether a newer version exists. Surfacing that in the reviewer interface costs little and catches a failure mode that accuracy metrics do not measure.
Nor does grounding address the claims a model declines to make. A system that answers only what it can support will sometimes return nothing where a human expert would have known the answer. That is usually the correct trade, but it should be a decision someone made on purpose, not a property that emerged.
Cost, latency and the shape of the trade
Span-level provenance is not free. Keeping offsets through ingestion adds engineering at index time, structured output adds tokens and a constrained decoding path, and validating every cited span adds a lookup before the answer is returned.
In practice the retrieval and validation costs are small next to generation, and the structured-output overhead is modest. The real cost is the ingestion pipeline, which becomes something you have to maintain carefully rather than something you can rewrite casually. That is a reasonable price for a property the system cannot lose by accident.
The trade is worth stating to stakeholders in those terms. You are buying a guarantee that is enforced by the schema rather than by discipline, and you are paying for it in ingestion complexity.
Where to start if you already have a RAG system
Most teams arrive at this with a working retrieval system that cites at document level. The migration does not have to be a rewrite.
Start by adding offsets at ingestion for new documents while leaving the existing index alone; the two can coexist, with older documents degrading to chunk-level citation. Then change the output schema to require support identifiers, and let validation reject unsupported claims — at first as a logged warning, so you can see the rate before it starts blocking.
That rate is the interesting number. In systems that have never enforced grounding it is usually higher than the team expects, and seeing it is generally what settles any remaining argument about whether the work is worth doing.