Failure PatternDecision layer
Why Agencies Fail With IQ Routing in Multi-Step Agent Workflows
Symptom: Client invoices show LLM spend dropping 40% in week one, then creeping back up by week three as agent loops start hitting frontier models for steps that previously routed to cheap models. Root cause: IQ Routing routes per request, not per workflow step, so agencies that set a single quality threshold for an entire agent loop end up sending every step to the frontier model, negating the 40-80% savings the tool advertises.
By InnovaAI ResearchPublished
Symptoms
- •Client invoices show LLM spend dropping 40% in week one, then creeping back up by week three as agent loops start hitting frontier models for steps that previously routed to cheap models.
- •The IQ Routing dashboard shows per-team budgets being exhausted by a single client's background jobs, while interactive chat requests get throttled at the 240 requests/min Free tier limit.
- •Support tickets from clients report inconsistent response quality: the same prompt returns a concise answer from a cheap model in one session and a verbose, expensive response from a frontier model in another.
- •Your delivery team spends hours debugging why semantic cache hit rates stay below 10% despite repeated user queries, because cache keys include timestamps or user IDs from the client's application code.
Root Causes
- •IQ Routing routes per request, not per workflow step, so agencies that set a single quality threshold for an entire agent loop end up sending every step to the frontier model, negating the 40-80% savings the tool advertises.
- •The Free plan's 240 requests/min limit forces agencies to upgrade to Team at $70/month per client, but per-team budgets are only available on Team, so agencies on Free cannot isolate spend by client and miss cost overruns until the invoice arrives.
- •Semantic caching requires tuning similarity thresholds, and agencies that enable it without adjusting for their client's dynamic query patterns (e.g., including user-specific context) see low cache hit rates, so the cache provides little cost relief.
- •Agencies assume IQ Routing's unified endpoint handles all providers automatically, but they must manage separate API keys for OpenAI, Anthropic, and Google upfront, and misconfigured keys cause fallback routing to the most expensive provider.
Fast Fixes
- •In the IQ Routing dashboard, create custom band maps per client workflow and assign each step a quality tier (auto, cheap, frontier) so agent loops use cheap models for retrieval steps and frontier only for final generation.
- •Upgrade to the Team plan and set per-team budgets with alerting thresholds for each client, then review the audit log weekly to catch spend anomalies before they hit the client's invoice.
- •Review the semantic cache configuration and adjust the similarity threshold downward for chat-heavy workloads, and strip dynamic parameters like timestamps from cache keys in your client's SDK integration.
- •Verify that each provider key (OpenAI, Anthropic, Google) is active and correctly mapped in the gateway settings, and test routing with a sample request to confirm cheap models are selected for non-critical steps.
More for AI Infrastructure
- Failure PatternsThe Algolia Metered Usage Trap: Why Agencies Fail With Algolia in High-Traffic Client Deployments
- Failure PatternsThe Single-Provider Margin Trap in AI Infrastructure
- Failure PatternsThe Blind Cost-Accrual Trap in AI Infrastructure
- Failure PatternsWhy Agencies Fail With DigitalOcean in AI Infrastructure Delivery