config-drift-checker Baseline Pinning and Canary Setup (Onboarding)

config-drift-checker Baseline Pinning and Canary Setup (Onboarding)

1. 01

   Initialize the tool in your agency's repository

   Run the setup command to install config-drift-checker, ensuring your GitHub Actions workflow file is created with the required permissions for creating PRs.

2. 02

   Configure the baseline model and Claude Code version

   In the configuration file, set the exact model identifier and Claude Code version to pin your baseline. This ensures reproducibility across client environments.

3. 03

   Generate initial test cases from CLAUDE.md, skills, and hooks

   Use the built-in generator to automatically create test cases from your existing setup. Review the generated cases to confirm they cover critical workflows.

4. 04

   Set up the GitHub Actions schedule for canary runs

   Edit the workflow YAML to define a cron schedule (e.g., daily) that triggers canary checks for new Claude Code releases. Adjust the schedule based on client release cycles.

5. 05

   Run a manual canary to validate the pipeline

   Trigger a manual workflow run to ensure the canary process correctly opens a PR when two green canaries pass on a new model. Verify the PR contains the drift index and grader verdicts.

6. 06

   Document the baseline scores and drift index for the client

   Record the initial baseline scores and drift index in a handoff report. Explain how to interpret the judge explanations and grader verdicts to the client.

7. 07

   Establish alerting and review procedures

   Configure notifications for failed canaries or drift alerts. Define a review process for your agency team to triage and remediate any regressions before they impact client delivery.