RAG Citation Accuracy: Verifying Answers Against Retrieved Evidence

RAG citation accuracy fails in a subtle way: an answer can include a real source link that does not support the sentence beside it. The retriever found a relevant document, the model produced fluent text, and the interface rendered a citation, yet the user still cannot verify the claim. A reliable system must preserve exact source spans, attach citations to atomic claims, test support and coverage separately, and refuse answers whose evidence cannot be resolved. This guide shows how to add that verification layer without replacing your existing retriever or generator.

Why a citation marker is not evidence

Most RAG pipelines treat citations as a formatting task. They pass retrieved chunks to a model, ask for bracketed source numbers, then replace those numbers with links. That proves only that the model emitted a valid marker. It does not prove that the referenced text entails the claim.

Several failures can hide behind a polished answer:

• A sentence contains two factual claims, but its citation supports only one.
• The source discusses the same subject without supporting the stated number, date, exception, or conclusion.
• The cited chunk is valid, but stale page offsets send the reader to a different page.
• The answer combines details from two chunks and cites only the first.
• A factual sentence has no citation because the model treated it as common knowledge.
• The citation points to a retrieved result that the final answer did not actually use.

Retrieval quality and answer support are separate checks. Microsoft's RAG evaluator guidance separates retrieval measures from groundedness and response completeness. A relevant chunk can still be attached to an unsupported answer. A grounded answer can also omit an important part of the requested information.

The ALCE citation-evaluation paper makes a similar separation for cited answers. It evaluates citation correctness and citation completeness as different dimensions. Correctness asks whether a cited source supports its claim. Completeness asks whether every claim that needs evidence has a citation.

Define the contract before changing prompts

Set the application's acceptance rules before changing the system prompt.

A practical contract has five rules:

1. Every externally verifiable factual claim must map to one or more retrieved source spans.
2. Every citation must resolve to a source that was available to the generator for that answer.
3. The cited span must support the whole attached claim, including qualifiers and values.
4. Source identity and location must come from ingestion metadata, not model-generated text.
5. An unresolved or unsupported claim must be removed, rewritten from available evidence, or sent to review.

The contract covers externally verifiable facts, not opinions, interface instructions, or clearly labeled recommendations. For example, "Select a review threshold based on your risk tolerance" is engineering advice. "The policy permits refunds for 30 days" is a factual claim and needs exact policy evidence.

Decide what "support" means for your domain. A support assistant may accept a direct statement from a current help article. A contract assistant may require the same clause, document version, jurisdiction, and effective date. A policy assistant may reject an older document even when its wording matches. The verifier needs those rules as data, not hidden assumptions inside a model prompt.

Preserve evidence coordinates during ingestion

Citation verification starts before retrieval. Each chunk needs enough immutable metadata to reconstruct the evidence a user should see.

Store at least these fields:

• document_id: stable identity for the source document
• document_version: immutable source version or content hash
• chunk_id: stable identity within that version
• page_number: reader-facing page when the format has pages
• source_start and source_end: offsets in canonical extracted text
• source_url: canonical reader-facing location
• section_path: heading hierarchy or equivalent structural path
• text: the exact indexed span

Keep the canonical extracted text for each immutable document version. Do not calculate reader links later from a newly parsed copy. If a parser update changes whitespace, reading order, or overlap behavior, old offsets no longer describe the new text.

The failure has occurred in practice. A reported FinSight-AI failure describes overlapping PDF chunks retaining stale character offsets and producing wrong citation pages. This is an author report, not a universal product defect. Its failure shape still belongs in the test suite because a correct answer can have broken attribution when chunk coordinates drift.

Validate coordinates as part of ingestion. Resolve every chunk back to the stored source version and assert that the referenced span matches the indexed text after your documented normalization. Quarantine chunks that fail. Rebuilding an index should create a new version rather than mutating the coordinates beneath prior answer records.

Generate structured claims, not citation-shaped prose

The generator should return a claim graph or another structured representation before the application renders prose. Each factual claim carries source references selected from the retrieved set.

{
  "claims": [
    {
      "claim_id": "c1",
      "text": "Expense reports must be submitted within 14 days.",
      "evidence": [
        {
          "chunk_id": "policy-v7-p12-c2",
          "quote": "Submit all expense reports within 14 calendar days",
          "source_start": 418,
          "source_end": 473
        }
      ]
    }
  ]
}

The model may propose the evidence mapping, but the application owns validation. Check that every chunk_id came from the retrieved context for this request. Resolve offsets against stored text. Verify that the quoted evidence is an exact or explicitly normalized slice. Never let the model supply a URL, page number, or document identity that the retrieval layer did not provide.

OpenAI file search returns file-citation annotations that an application can resolve into source labels. LlamaIndex's CitationQueryEngine example demonstrates generating granular citation sources from retrieved chunks and controlling citation chunk size. Both features produce useful references. The application must still test whether the referenced span supports the attached claim.

Keep claims atomic. "The plan renews annually and can be cancelled within 30 days" contains two assertions. Split them unless one evidence span clearly supports both. Atomic claims make failures repairable: the system can remove or regenerate one unsupported statement without discarding the entire answer.

Verify support and coverage separately

Run two passes after generation and before rendering.

The support pass evaluates each claim and its cited evidence. Start with deterministic checks:

• The source exists in the retrieved set.
• The document version still exists and is allowed for the user.
• The source span resolves and matches stored text.
• Required qualifiers such as date, region, product, or policy version agree.
• Numeric values and named entities in the claim appear in, or are directly entailed by, the evidence.

Then use a constrained entailment check for claims that pass structural validation. Give the evaluator one atomic claim and only its proposed evidence. Ask for a small result schema such as supported, unsupported, or insufficient, plus the unsupported fragment. Do not give it the full retrieved context, because a claim supported somewhere else should not rescue a wrong citation.

Ragas defines faithfulness by identifying answer claims and checking whether each can be inferred from the retrieved context. That pattern is useful, but citation verification narrows the context further: test each claim against the evidence attached to that citation, not against every chunk the retriever happened to return.

The coverage pass works in the other direction. Extract factual claims from the rendered answer and verify that each required claim has at least one accepted citation. This catches uncited facts even when every existing citation is correct.

Track both rates:

• Citation support rate: accepted citations divided by all rendered citations
• Claim coverage rate: supported cited claims divided by all claims requiring evidence

Do not merge them into one score at the release gate. A system can achieve perfect support by citing only one safe sentence while leaving the rest uncited. It can achieve complete coverage by attaching weak sources everywhere. Set separate thresholds and retain the raw failure reasons.

Choose a failure policy by consequence

A verifier needs a deterministic disposition. Otherwise the application logs a warning and publishes the same bad answer.

Use these decision rules:

• If a citation cannot resolve, block the claim. This is a data-integrity failure.
• If evidence contradicts the claim, block the whole answer and record the conflict.
• If evidence is insufficient, remove the claim or regenerate it using only accepted spans.
• If a required qualifier is missing, ask a clarifying question or state the narrower supported fact.
• If no source supports the user's request, abstain instead of producing an uncited answer.
• If the workflow can cause a business action, never let a generated citation authorize that action. Apply normal application policy and approval controls separately.

For low-risk internal search, partial answers may be acceptable when unsupported claims are removed and the result says what could not be verified. For legal, financial, medical, or operational decisions, route failed answers to a reviewer with the claim, proposed evidence, source version, and failure reason visible together.

Avoid an automatic retry loop that simply asks the same model to "fix citations." Retry once with the rejected claims and accepted evidence made explicit. If verification still fails, stop. Repeated generation can change wording without improving support and can hide a systematic retrieval or parsing defect.

Test the complete attribution path

Build fixtures from real failure shapes, not only ideal question and answer pairs. Each fixture should freeze the source versions, retrieved chunks, expected claims, acceptable evidence spans, and expected disposition.

Include at least these cases:

• One citation fully supports one atomic claim.
• One sentence contains two claims but evidence supports only one.
• A correct fact points to the wrong page after chunk overlap.
• Two source versions disagree and only one is current.
• The answer cites a relevant section that lacks the stated number.
• A claim needs two independent chunks for full support.
• Every citation is valid, but one factual sentence is uncited.
• Retrieval returns no supporting evidence and the system must abstain.
• A user lacks permission to open an otherwise supporting source.
• The citation target is deleted after generation but before display.

Run the fixtures after parser, chunking, embedding, retriever, prompt, model, or citation-renderer changes. Store verifier outcomes with the answer trace so an engineer can distinguish retrieval misses, offset corruption, unsupported generation, stale sources, and rendering defects.

Test the reader experience too. Clicking a citation should open the correct document version and place the supporting passage in view. If the original format cannot support a stable deep link, show a verified excerpt with document identity and page information rather than pretending a generic document link is precise.

Common mistakes that keep bad citations alive

Do not grade groundedness against all retrieved context. That can mark a claim as supported even though its visible citation points elsewhere. Evaluate the actual attachment. Page numbers also belong to ingestion metadata, not model output. Models should reference source handles, and application code should render locations.

Apply the same checks to every answer in the targeted workflow instead of validating only text that already looks suspicious. Keep support, coverage, resolution, version, and permission failures separate rather than hiding them in one "quality score." Each failure has a different owner and repair path.

A citation can inform a user, but it cannot authorize tool execution. Business actions still require server-side authorization, validation, and any required approval.

Put the verifier in the release gate

Start with a shadow pass on current traffic. Record claims, source mappings, support decisions, and coverage without changing answers. Review failures by category and fix coordinate corruption before tuning model prompts. Then enforce blocking for unresolved citations and unsupported high-risk claims. Add partial-answer behavior only after the failure reason is visible to the reader.

RAG citation accuracy becomes dependable when evidence identity survives ingestion, every factual claim has an explicit source attachment, and the application rejects mappings it cannot prove. The next action is concrete: take 20 recent answers, split them into atomic claims, resolve every citation to a stored span, and convert each failure you find into a regression fixture before your next parser or model release.

References

1. Microsoft RAG evaluators supports the separation of retrieval, groundedness, and response-completeness checks.
2. Ragas faithfulness metric supports claim extraction and evidence-based faithfulness evaluation.
3. LlamaIndex CitationQueryEngine supports granular citation generation from retrieved chunks.
4. OpenAI file search supports the use of application-resolved file-citation annotations.
5. ALCE citation-evaluation paper supports separate citation-correctness and citation-completeness evaluation.
6. FinSight-AI issue 1108 provides the practitioner-reported wrong-page failure caused by chunk and offset misalignment.

---