Extraction Accuracy Gate (QA)

Extraction Accuracy Gate (QA)

1. 01

   Freeze a 50-document gold set before the first production run

   Pull a stratified sample across every document type the client sends (invoices, contracts, intake forms) and hand-key the correct field values once. This becomes the only defensible baseline for every accuracy claim you make afterward.

2. 02

   Score field-level precision and recall, not document-level pass rates

   A 92% document pass rate can hide a 40% miss rate on a single high-value field like total amount or renewal date. Report per-field numbers so the client sees which fields carry risk.

3. 03

   Set a confidence threshold and route everything below it to human review

   Platforms such as Instabase expose confidence scoring per extracted field; pick a cutoff that keeps straight-through processing above 70% while capping silent errors near zero, then document the tradeoff in writing.

4. 04

   Test the exception paths with deliberately damaged documents

   Feed the pipeline rotated scans, password-protected PDFs, and multi-page contracts with signature pages out of order. Note where the workflow stalls and who gets notified when it does.

5. 05

   Reconcile extracted totals against the client's system of record weekly

   Compare a random 20-document batch against the ERP or accounting ledger. Discrepancies above 0.5% of line items trigger a retraining or rule review before the next billing cycle.

6. 06

   Log every correction a human makes and feed it back into the model or ruleset

   Correction logs are the cheapest training signal available. Review them monthly with tools like Rossum or Reducto in the loop, and retire any rule that has not fired in 60 days.

7. 07

   Publish a one-page accuracy scorecard to the client each month

   Include documents processed, straight-through rate, human touches, and estimated hours saved. This is the artifact that keeps the engagement positioned as a capability rather than a commodity subscription.