Agent Failure Containment Drill (QA)

Agent Failure Containment Drill (QA)

1. 01

   Freeze the failing bot and snapshot its last 50 run logs before anyone edits it

   Pull timestamps, input payloads, and the exact application state at failure. Editing first destroys the evidence you need to prove to the client what went wrong and what changed.

2. 02

   Classify the failure as selector drift, permission overreach, or model hallucination

   Selector drift shows as element-not-found errors after a UI release. Permission overreach shows as actions on records outside the defined scope. Hallucination shows as plausible but wrong field values in otherwise successful runs.

3. 03

   Replay the failed run in a sandboxed copy of the client environment

   Never re-run against production to reproduce. Warmwind-style visual workers and browser-extension automations like Robin both need an isolated profile with scrubbed credentials before replay.

4. 04

   Cap the blast radius by revoking credentials the bot no longer needs

   Scope each automation to the minimum access its defined task requires. Agents that broke out of controlled test environments and reached external systems did so with standing permissions nobody had trimmed.

5. 05

   Rewrite the failing step with an explicit success assertion

   A bot that clicks a button and moves on has no way to detect a silent failure. Add a check that the expected record, field value, or confirmation state actually exists before the next step fires.

6. 06

   Log the incident in the client-facing automation register with root cause and fix date

   The register becomes the evidence base for retainer renewals. Clients accept failures that are documented and closed; they churn on failures they discover themselves.

7. 07

   Re-run the full workflow end to end and compare output against the pre-incident baseline

   Diff record counts and key field values. A fix that repairs the failing step but shifts downstream output is a new incident, not a resolution.