Evaluation Coverage Handoff to Client Operations (Handoff)

Evaluation Coverage Handoff to Client Operations (Handoff)

1. 01

   Freeze the evaluation asset inventory at the moment of handoff

   Export the full list of scorers, judge prompts, golden datasets, and trace sampling rules with version numbers and last-edited dates. Anything still living in a personal workspace gets promoted to a shared project before the call.

2. 02

   Map every scorer to the client outcome it protects

   Pair each quality metric with the business failure it catches, such as hallucinated pricing in a support agent or tone drift in a voice bot. Scorers with no named owner on the client side get flagged for retirement rather than inherited.

3. 03

   Document the alert thresholds and who receives them

   Record the numeric trigger for each monitor (latency ceiling, cost per session, judge score floor) and the named client contact plus agency escalation path. A threshold without a recipient is a dashboard, not a control.

4. 04

   Replay three historical failures against the client's own access

   Pull traces from real incidents, rerun them inside the client's environment, and confirm the same failure surfaces. This proves the instrumentation travels with the account rather than staying in agency tooling.

5. 05

   Transfer judge calibration notes and known blind spots

   Hand over the disagreements found during judge tuning, including the cases where automated scoring diverged from human review. Clients who inherit a scorer without its known failure modes will over-trust it within a month.

6. 06

   Set the review cadence and the drift re-baseline date

   Agree a monthly quality review and a quarterly re-baseline of golden datasets, since model upgrades and prompt edits quietly invalidate old scoring sets. Put both dates on the client calendar before the handoff call ends.

7. 07

   Run a joint readout of the first post-handoff week

   Schedule a 30-minute session seven days after transfer where the client walks through their own traces and the agency observes only. If the client cannot explain a failing trace unaided, the handoff is incomplete.