Grounded Answers and Citations
Grounded answers and citations are the part of a retrieval-augmented generation system that turns retrieved passages into an auditable response. The desired outcome is narrower than simply answering a question: every material claim should be supported by evidence placed in the model context, citations should identify the exact source of that support, and the system should decline when retrieval does not contain enough evidence.
In this course, that matters because RAG is often adopted to reduce hallucination and expose source documents. Retrieval alone does not guarantee either result. A model can ignore retrieved text, overgeneralize from it, cite a document that does not support the sentence, or answer from pretraining when the index returns nothing useful. Grounding is the control loop around generation: retrieve candidates, select evidence, constrain the answer format, verify citation coverage, and decide whether to answer or abstain.
Purpose and Outcome
A grounded answer has three visible properties. First, its claims are entailed by supplied evidence, meaning the evidence directly supports the statement without requiring outside assumptions. Second, each claim carries a citation such as a source identifier, URL, document title, paragraph number, timestamp, or chunk identifier. Third, the answer separates supported content from uncertainty. If the evidence says full-time employees receive twenty days of leave, the answer should not add that contractors receive the same benefit unless a retrieved source says so.
A citation is a pointer back to the evidence record used at generation time. The evidence record should include stable metadata: document id, version or last indexed time where available, title, location inside the document, and the retrieved text span. Without that metadata, the application can show a source-looking label while making later verification impossible.
How Grounding Works Internally
A typical grounded-answer pipeline has five internal stages. The query stage rewrites or normalizes the user’s question while preserving intent. The retrieval stage searches a vector index, keyword index, graph, database, or hybrid retriever and returns candidate chunks with scores and metadata. The evidence selection stage removes duplicates, trims irrelevant spans, reranks candidates, and chooses context that fits inside the model window. The generation stage asks the model to answer only from selected evidence and cite supporting source ids. The verification stage checks whether the output follows the citation contract before returning it.
The hardest part is the relationship between a claim and a source span. A chunk can be relevant to a question without supporting the generated sentence. For example, a benefits document may discuss vacation eligibility but not the number of days. If the model says a number anyway and cites that chunk, the answer is cited but not grounded. Good systems track claim-level coverage, not only answer-level citation count. Teams often implement this by asking the model for structured claims, running a citation validator, or using a second model or rule-based checker to compare each claim with its cited spans.
Chunking affects citation quality. If chunks are too large, a citation may send the user to a long page where the supporting sentence is hard to find. If chunks are too small, retrieval may separate the condition from the fact, such as eligibility in one chunk and benefit amount in another. Overlap can help, but too much overlap creates duplicate citations and wastes context. A useful chunk preserves a coherent semantic unit: a policy paragraph, FAQ answer, table row group, API reference section, or transcript segment.
API and Data Anatomy
The core data types are simple. A source passage contains text and metadata. A generated answer contains one or more claims. A citation links a claim to source passages. An abstention is a first-class result, not an exception. Treating abstention as a normal response makes it easier to evaluate and safer to expose in product behavior.
from dataclasses import dataclass
@dataclass(frozen=True)
class SourcePassage:
source_id: str
title: str
text: str
@dataclass(frozen=True)
class Claim:
text: str
source_ids: tuple[str, ...]
sources = {
"doc-a": SourcePassage(
source_id="doc-a",
title="Leave Policy",
text="Full-time employees receive 20 days of annual leave.",
)
}
claim = Claim(
text="Full-time employees receive 20 days of annual leave.",
source_ids=("doc-a",),
)
print(f"{claim.text} [{', '.join(claim.source_ids)}]")This first example represents evidence separately from the claim. The expected output is Full-time employees receive 20 days of annual leave. [doc-a]. The important design choice is that the source id is not invented by the answer string. It comes from retrieved metadata and remains attached to the claim.
In a production API, the request usually includes the user question, retrieval filters, authorization context, and answer style. The response should include answer text, citations, abstention reason when applicable, and client-safe diagnostics such as retrieval count or policy category. Keep raw prompts, secrets, and private documents out of client-visible diagnostics.
Worked Example 1: One Claim, One Source
Suppose the retrieved source says, Full-time employees receive 20 days of annual leave. A grounded answer can repeat that fact and cite the source. It should not broaden the statement to all workers or add accrual rules unless those facts are also present. This is the base case: one source span supports one claim exactly.
The deterministic behavior is straightforward. If the source id is doc-a, the rendered sentence includes [doc-a]. If the same claim is copied into a summary, the citation should travel with it. This is why many implementations store claim objects internally and render prose only at the final step.
Worked Example 2: Detecting Missing Citation Coverage
The next example checks whether each claim has at least one citation and whether cited source ids exist in the retrieved evidence set. This does not prove semantic entailment, but it catches common formatting and plumbing failures before users see them.
from dataclasses import dataclass
@dataclass(frozen=True)
class SourcePassage:
source_id: str
text: str
def citation_coverage(answer_claims: list[tuple[str, list[str]]], sources: dict[str, SourcePassage]) -> dict[str, object]:
unsupported = []
unknown_sources = []
for claim, cited_ids in answer_claims:
if not cited_ids:
unsupported.append(claim)
continue
for source_id in cited_ids:
if source_id not in sources:
unknown_sources.append(source_id)
return {
"ok": not unsupported and not unknown_sources,
"unsupported_claims": unsupported,
"unknown_sources": sorted(set(unknown_sources)),
}
sources = {
"hr-1": SourcePassage("hr-1", "Full-time employees receive 20 days of annual leave."),
}
claims = [
("Full-time employees receive 20 days of annual leave.", ["hr-1"]),
("Contractors receive the same benefit.", []),
]
print(citation_coverage(claims, sources))The expected output reports ok as False and lists Contractors receive the same benefit. as unsupported. The cited first claim passes the structural check because hr-1 exists. The contractor claim fails because it has no source id. A stronger validator would also compare the contractor claim with source text and reject it even if the model attached an unrelated citation.
Worked Example 3: Abstaining on Weak Retrieval
Grounded systems need an answer policy. Retrieval scores are not universal truth, but they are useful signals when calibrated on your corpus. A policy can require at least one evidence passage and a minimum top score before generation proceeds.
def should_answer(top_score: float, evidence_count: int, min_score: float = 0.72) -> bool:
return evidence_count > 0 and top_score >= min_score
def response_policy(question: str, top_score: float, evidence_count: int) -> str:
if not should_answer(top_score, evidence_count):
return "I do not have enough retrieved evidence to answer that."
return f"Answer the question with citations: {question}"
print(response_policy("How many leave days do full-time employees get?", 0.84, 2))
print(response_policy("Do contractors receive leave?", 0.41, 1))The first call returns Answer the question with citations: How many leave days do full-time employees get? because the score and evidence count pass the threshold. The second call returns I do not have enough retrieved evidence to answer that. The trade-off is recall versus precision: a high threshold reduces unsupported answers but may refuse answerable questions when the retriever scores phrasing poorly.
Design Choices and Trade-offs
Citation granularity is the first major choice. Document-level citations are easy to produce but weak for verification. Paragraph-level or span-level citations are more useful, but require careful indexing and metadata. Table citations need row and column context, not just a page id, because a model can mix values across rows. Transcript citations often need timestamps. Codebase citations need file path and line range or symbol name.
Another choice is whether the model emits final prose with inline citations or structured JSON with claims and cited ids. Inline citations are easy for users to read but harder to validate. Structured output is easier to test and transform, but it may require a rendering layer. Many robust systems ask for structured claims first, validate them, and then render the final answer from the accepted claim set.
There is also a question of how much evidence to include. More passages can increase recall, but extra context introduces distractors and raises cost and latency. Reranking helps by ordering evidence according to likely support rather than raw lexical or vector similarity. Compression can help fit long documents into context, but it introduces another generation step whose summaries must themselves remain grounded.
Failure Modes and Troubleshooting
A common symptom is an answer with citations that do not support the claims. The cause is usually loose prompting, oversized chunks, or no post-generation validation. Diagnose by logging retrieved passages, generated claims, and cited ids for a failing example. Correct it by tightening the output schema, lowering chunk size around source sections, adding claim-level validation, and creating a regression test.
Another symptom is frequent abstention for questions humans know are answerable. The cause may be poor chunking, missing synonyms, overly strict filters, or a threshold calibrated on a different corpus. Diagnose by running the query directly against the retriever, inspecting top misses, and checking whether authorization filters removed the relevant document. Correct it with hybrid retrieval, query expansion, metadata fixes, or threshold tuning backed by an evaluation set.
A third symptom is citations pointing to stale or inaccessible documents. The cause is usually index drift: the vector index still contains chunks from an old document version, or the user can retrieve metadata for a source they cannot open. Diagnose by comparing citation ids with the current document store and permission system. Correct it by using stable source versions, deleting stale chunks during reindexing, and applying access control before evidence reaches the model.
Security, Performance, and Reliability
Grounded answers reduce unsupported claims, but they do not remove security risks. Retrieved documents are untrusted input and may contain prompt injection such as instructions to ignore system rules. The generator should treat passages as evidence, not authority over application behavior. Keep instructions outside retrieved text, label evidence clearly, and prevent document content from controlling tools, permissions, or hidden prompts.
Performance depends on retrieval latency, reranking cost, model context size, and validation passes. Span-level citations often require more metadata and sometimes more model calls. Cache stable retrieval results only when permissions and document versions are part of the cache key. Reliability improves when the system can degrade: if reranking fails, it may use conservative top-k retrieval; if validation fails, it should return an abstention or ask for clarification rather than presenting an uncited answer.
Hands-on Lab: Build a Tiny Grounded QA Loop
Prerequisites: Python 3, a terminal, and no external packages. The lab uses a tiny in-memory corpus so the behavior is deterministic and easy to inspect.
- Create a scratch Python file and paste the following code.
- Run it with
python3 grounded_lab.py. - Verify that the annual leave question returns a cited answer and the parental leave question abstains.
- Change the corpus by adding a parental leave passage, rerun the file, and observe that the current answer function still abstains until you explicitly teach it the supported claim. This shows retrieval and answer synthesis are separate responsibilities.
- Clean up by deleting the scratch file when finished.
from dataclasses import dataclass
@dataclass(frozen=True)
class Passage:
source_id: str
title: str
text: str
CORPUS = [
Passage("benefits-1", "Leave Policy", "Full-time employees receive 20 days of annual leave."),
Passage("benefits-2", "Sick Leave", "Employees may use up to 10 sick days per year."),
Passage("security-1", "Device Policy", "Company laptops must use disk encryption."),
]
def retrieve(query: str, limit: int = 2) -> list[Passage]:
query_terms = set(query.lower().replace("?", "").split())
scored = []
for passage in CORPUS:
passage_terms = set(passage.text.lower().replace(".", "").split())
score = len(query_terms & passage_terms)
if score:
scored.append((score, passage))
return [passage for score, passage in sorted(scored, key=lambda item: item[0], reverse=True)[:limit]]
def answer_leave_question(query: str) -> str:
passages = retrieve(query)
if not passages:
return "I do not have enough retrieved evidence to answer that."
for passage in passages:
if "20 days of annual leave" in passage.text:
return f"Full-time employees receive 20 days of annual leave. [{passage.source_id}]"
return "I do not have enough retrieved evidence to answer that."
print(answer_leave_question("How many annual leave days do full-time employees receive?"))
print(answer_leave_question("What is the parental leave policy?"))The expected output is two lines: Full-time employees receive 20 days of annual leave. [benefits-1] followed by I do not have enough retrieved evidence to answer that. Verification is not just whether the program prints something plausible. Check that the cited id exists in CORPUS, the sentence is present in the cited passage, and the unsupported question does not receive a guessed policy.
Assessment Exercises
- A model answers,
Employees receive 20 vacation days and 10 sick days.It cites only the annual leave policy. Explain why this is not fully grounded and how you would represent the two claims. - Your system returns many document-level citations, but users cannot find the supporting text. Propose a chunking and metadata strategy that improves citation usefulness without making retrieval brittle.
- Design an evaluation case for stale citations. What fixture data, expected behavior, and diagnostic signal would you include?
- Given a retriever with high recall but many irrelevant passages, decide whether to change top-k, add reranking, or tune the generation prompt. Justify the order you would try them.
- Write an abstention rule for a medical benefits assistant and explain which false positives and false negatives worry you most.
Summary
Grounded answers and citations make RAG inspectable. The mechanism is a chain of evidence retrieval, metadata preservation, constrained generation, citation validation, and abstention. The essential trade-off is between helpfulness and support: the system should answer when retrieved evidence supports the claim, cite the exact evidence used, and refuse when evidence is missing, stale, unauthorized, or too ambiguous.
