Failure PatternDecision layer
The Connector Sprawl Trap: Why RAG Tooling Collapses When Every Client Gets a Bespoke Ingestion Stack
Symptom: Each new client retainer starts with a fresh ingestion build: a different parser, a different connector set, a different chunking config, and no shared schema between them. Root cause: Agencies treat each client engagement as a greenfield build rather than an instance of a repeatable ingestion pattern, so connector logic, chunk sizes, and metadata conventions multiply without a shared contract.
By InnovaAI ResearchPublished
How do you recognize it?
- •Each new client retainer starts with a fresh ingestion build: a different parser, a different connector set, a different chunking config, and no shared schema between them.
- •Retrieval quality drifts between accounts, so a document that answers cleanly for one client returns fragments or nothing for another running the same query pattern.
- •Nobody can state which index holds which client's content, and the answer changes depending on who is asked.
- •Re-indexing after a source change takes days because the pipeline was assembled by hand and only one engineer understands it.
- •Clients ask why a cited answer pulled from an outdated document version, and the team cannot trace which ingestion run produced it.
Why does it happen?
- •Agencies treat each client engagement as a greenfield build rather than an instance of a repeatable ingestion pattern, so connector logic, chunk sizes, and metadata conventions multiply without a shared contract.
- •Connector breadth gets mistaken for retrieval quality. A platform that reaches Google Drive, Notion, and Slack through pre-built connectors still returns weak results if chunking and entity extraction are configured per project with no baseline.
- •The evaluation layer that would catch cross-client regressions is skipped because it produces no client-facing artifact, so degradation is only discovered when a deliverable is already in review.
- •Ownership sits with whoever built the first pipeline, and that person becomes the single point of failure for every re-index, schema change, and connector addition across the book of business.
How do you fix it?
- •Freeze new connector work for two weeks and inventory every active client index: source systems, parser, chunk strategy, and last successful re-index date, in one shared table.
- •Define a single ingestion contract (metadata fields, chunk size range, entity types to extract) and retrofit the three highest-revenue client indexes to it before touching the rest.
- •Stand up a 50-question retrieval test set per client, run it against the current index, and record pass rates so any future provider or config change has a before-and-after number.
- •Assign one named owner per client index with a documented re-index runbook, so the pipeline survives the departure of the engineer who built it.