Extraction Accuracy Gate (QA)

Extraction Accuracy Gate (QA)

1. 01

   Freeze a 50-document golden set before the first production run

   Pull a stratified sample across every document type the client sends (invoices, contracts, intake forms) and have the client's own ops lead confirm the correct field values by hand. This becomes the only benchmark that matters.

2. 02

   Score field-level accuracy, not document-level pass rates

   A 94% document pass rate can hide a 60% miss rate on a single high-value field like payment terms. Report precision and recall per field so the client sees where the risk actually sits.

3. 03

   Set a confidence threshold and route everything below it to human review

   Platforms such as Instabase return confidence scores per extracted value; pick the cutoff where the cost of a human check is cheaper than the cost of a downstream error, and document that math.

4. 04

   Re-run the golden set after every template, model, or prompt change

   Treat any configuration edit as a release. A vendor-side model update can silently shift extraction behavior on edge cases the client cares about.

5. 05

   Log every human correction and review the log weekly

   Corrections are free training data for the next threshold adjustment and the clearest evidence for a retainer renewal conversation.

6. 06

   Publish a one-page accuracy scorecard to the client each month

   Show documents processed, straight-through rate, exception rate, and hours saved against the pre-automation baseline. Numbers the client can forward internally are what keep the engagement funded.