Multi-Model Routing Decision Record (Delivery)

Multi-Model Routing Decision Record (Delivery)

1. 01

   Name the routing decision and the client deliverable it governs

   One record per client AI feature, not per vendor. Tie it to the retainer line item or project milestone the routing choice affects so the record has an owner and a budget.

2. 02

   List every model endpoint in scope with its contracted price basis

   Record input and output token rates, minimum commit, and overage terms for each provider in the pool. Include self-hosted options such as Ollama or vLLM-backed deployments where the agency carries the compute cost.

3. 03

   State the routing rule in plain language before writing config

   Example: default to the primary frontier model, fall back to a cheaper model when prompt length exceeds a threshold, escalate to a reasoning model only for tasks tagged as analysis. Ambiguity here is what turns into silent cost drift.

4. 04

   Assign a quality floor per task class and the test set that proves it

   Define the pass threshold on the agency's own client content, including technical, multilingual, and edge-case inputs. Routing changes that clear a generic benchmark can still fail a client's actual corpus.

5. 05

   Document the gateway and observability layer that enforces the rule

   Name the proxy or gateway in use (for instance Helicone for caching and request logs, or Portkey for policy-based routing) and confirm which metrics it captures: latency percentiles, error rates, and spend per client.

6. 06

   Record the failover path and the client-facing degradation message

   Specify the secondary provider, the trigger condition, and what the client sees when the fallback is active. A fallback that silently returns lower-quality output is worse than a visible delay.

7. 07

   Set the review cadence and the re-decision triggers

   Review monthly against actual spend and latency data. Force a re-decision when a provider changes pricing, deprecates a model, or shifts safety guardrails that affect client work.

8. 08

   File the record where delivery and finance both read it

   Attach it to the client workspace alongside the project stage label so scoping conversations and invoices reference the same routing assumptions.