Operating ProcedureExecution layer

Provider Concentration Risk Review (Onboarding)

A checklist with 7 steps: Inventory every model provider and inference host behind the client's existing AI features.

By InnovaAI ResearchPublished

What are the steps?

checklist

Provider Concentration Risk Review (Onboarding)

  1. 01

    Inventory every model provider and inference host behind the client's existing AI features

    List each endpoint in production, including gateway layers such as Helicone or Portkey, plus any direct calls to Anthropic, OpenAI, or Google Vertex AI. Record which client deliverable depends on each one.

  2. 02

    Score each dependency on two axes: switching cost and blast radius

    Switching cost covers prompt rewrites, fine-tuned weights, and eval re-runs. Blast radius covers how many client-facing workflows stop if that provider changes pricing, deprecates a model, or tightens rate limits.

  3. 03

    Confirm a second provider can serve the same workload before signing the retainer

    Run a sample of 20 to 50 representative prompts through an alternate model and compare output quality, latency, and cost per call. A fallback that has never been tested is not a fallback.

  4. 04

    Set a spend ceiling and alert threshold per client account

    Token-billed agentic workflows can erode margin quietly. Define the monthly ceiling in the statement of work and wire alerts at 70 and 90 percent of that figure.

  5. 05

    Document data residency and retention terms for each provider in the client file

    Note where prompts and outputs are stored, whether the provider trains on submitted data, and what the deletion window is. EU-facing clients need this on record before delivery starts.

  6. 06

    Map agent permissions to the minimum scope each task requires

    Restrict every agent connected to client systems to the narrowest credential set that completes its job, and log the credential owner. Unscoped access is the most common source of incident exposure.

  7. 07

    Write the fallback runbook and assign an owner

    One page per critical dependency: who is paged, which alternate model takes over, and what the client is told. Store it where delivery staff can reach it without asking.