RAG Grounding Verification Protocol (QA)

RAG Grounding Verification Protocol (QA)

1. 01

   Confirm the retrieval architecture before trusting any output

   Ask the vendor or your engineering lead whether the system uses RAG or a fixed training cutoff. Forrester's Q3 2026 Wave analysis flags this as a procurement criterion, so document the answer in the client's file.

2. 02

   Compile a test set of 20 client-specific questions with known answers

   Pull from real client FAQs, product documentation, and past support tickets. Each question must have a verifiable source document in the indexed corpus.

3. 03

   Run the test set through the RAG pipeline and capture raw responses

   Log the retrieved chunks alongside the generated answer. This lets you separate retrieval failures from generation errors.

4. 04

   Score each answer for citation accuracy and completeness

   Mark whether every factual claim maps to a cited source and whether the source actually supports the claim. GPTZero's detection of fabricated sources in PwC reports shows how costly hallucinated citations can be.

5. 05

   Identify patterns in retrieval misses and rank them by client impact

   Group failures by document type, query phrasing, or chunking strategy. Prioritize fixes that affect the client's most common use cases.

6. 06

   Adjust chunking, embedding, or reranking settings and re-run the test set

   Change one variable at a time, such as chunk size or similarity threshold, to isolate what improves accuracy. Document the before and after scores.

7. 07

   Publish a grounding scorecard for the client

   Share the pass rate, top failure modes, and planned fixes. This transparency builds trust, which Forrester now treats as a competitive differentiator in AI engagements.