Agnost AI Client Agent Improvement Cycle (Delivery)

Agnost AI Client Agent Improvement Cycle (Delivery)

1. 01

   Open the Agnost AI dashboard and select the client's workspace from the project switcher.

   Ensure the workspace corresponds to the client's production agent and that the OpenTelemetry integration is active, otherwise event ingestion will not populate.

2. 02

   Navigate to the 'Conversations' tab and apply a date range filter for the last 7 days.

   This aligns with the Starter plan's 30-day retention, but a weekly window keeps the review focused on recent friction points.

3. 03

   Use the natural language query bar to ask: 'Show me conversations with repeated retries or user frustration'.

   Agnost AI's intent and sentiment extraction will surface the highest-impact failure patterns that standard evals miss.

4. 04

   Review the 'Failure Detection' panel and note the top 5 intents with the highest failure counts.

   Prioritize intents that correlate with user drop-offs or negative sentiment, as these represent the most urgent client-facing issues.

5. 05

   Click on a specific failure intent to drill into the conversation transcripts and identify the root cause.

   Look for broken workflows or ambiguous agent responses that trigger repeated retries, which are common friction signals.

6. 06

   Trigger the 'Generate Fix' action for the selected intent to create a reviewed pull request.

   Agnost AI will propose a code change; review the PR for correctness and alignment with the client's agent behavior before merging.

7. 07

   After merging the PR, monitor the same intent in the next weekly cycle to confirm the failure rate drops.

   Document the improvement in the client's monthly report, tying the fix to a measurable reduction in friction events.