Screening Model Review Before Client Delivery (QA)

Screening Model Review Before Client Delivery (QA)

1. 01

   Freeze a 50-candidate regression set per open role before scoring goes live

   Pull real applicants from the last two hiring cycles, label the outcomes your client would call correct, and store the set outside the ATS so vendor model updates cannot overwrite it.

2. 02

   Re-score the frozen set after every model or workflow change and diff the results

   A 5-point swing in pass-through rate on the same 50 candidates is the signal to pause delivery, not a rounding error. Log the diff with a date and the vendor release note that triggered it.

3. 03

   Sample 20 AI-screened rejections per week and have a recruiter re-read them blind

   Track the overturn rate. Anything above 10 percent means the screening logic is filtering on proxies the client did not ask for, and the client will find out before you do.

4. 04

   Check that automated candidate messaging carries the client's name, not the platform's

   Read the last 30 outbound sequences end to end. Generic rejection copy sent under a client brand is a retainer risk that surfaces in review calls, not in dashboards.

5. 05

   Confirm every agent-generated shortlist has a named human reviewer on record

   Fewer than half of agent-generated insights at surveyed organizations lead to meaningful action, so an unreviewed shortlist is an unverified claim in a client deliverable.

6. 06

   Document who controls scoring thresholds, feature access, and dispute resolution for the platform

   Forrester flags vendors that set usage limits, gate features, and arbitrate their own disputes. Write the answer into the client file so a pricing change does not become a delivery surprise.

7. 07

   Run a bias spot-check by segment before the monthly client report ships

   Compare selection rates across at least three demographic or geography segments where data allows. Report the numbers even when they look clean; the audit trail is the deliverable.