Which customer support AI works best when the knowledge base is incomplete?

Decision memo: incomplete knowledge bases reward restraint more than fluency.

The support AI that holds up here is the one that will not finish an answer it cannot ground. When docs are thin, confident synthesis is the failure mode—not silence. Prefer systems that treat low retrieval coverage as a hard stop, pass partial evidence forward, and escalate with structure a human can use immediately. Sounding complete without sources trains customers to trust answers the stack cannot defend.

Keep the automated lane narrow. Allow replies only when retrieved passages actually address the ask. Ambiguous policy edges, missing product states, and multi-step exceptions should not be “completed” by fallback phrasing. A practical control: dual-gate before send—require both an internal confidence threshold and explicit source coverage for claimed facts. Fail either gate and route to a human queue with the original message, the retrieved snippets, and the gaps already flagged. Do not invent bridging language to fill those gaps.

Draw the line where the ticket needs judgment the knowledge base never encoded. Escalation quality beats clever recovery text. The non-obvious cost: raise thresholds without packaging partial retrieval for agents, and you protect accuracy while slowing resolution—humans restart cold. Calibrate the handoff payload, not only the refusal line.

The question smuggles in a weak assumption: that incomplete knowledge bases are a product-selection problem. They aren't. Almost every modern support AI can invent a fluent answer when coverage is thin. The interesting difference is not who sounds capable with missing docs—it's who refuses, hedges, and routes when confidence should drop.

What you need is not a vendor ranking but a failure-facing evaluation set. Take a fixed sample of historical tickets where the correct resolution is known. For half of them, withhold every related article from the retrieval index before the run. Keep thresholds and escalation destinations constant across candidates. Score three outcomes only: refused or deferred, escalated with usable context, and answered anyway. Then open a weekly failure review loop: every confident answer that hit a withheld article gets reviewed for whether the system should have stopped.

The non-obvious consequence is that the "best" tool on incomplete knowledge often looks worse in first-contact resolution dashboards, because it refuses more. If your buying process optimizes for answer rate under incomplete coverage, you will select the system most willing to paper over your docs.

Do not pick a support AI for incomplete knowledge bases by comparing answer coverage. Pick the one whose fallback, confidence gating, and escalation path stay honest when retrieval is thin—then prove it with the smallest reversible pilot you can reverse in a week.

Run a controlled pilot on a single queue where known gaps already exist. Mirror production traffic into the candidate, but deliberately withhold a defined slice of the knowledge base for those intents. Score only three behaviors: does it refuse or hedge when evidence is missing, does confidence drop before a polished answer appears, and does the escalation packet give a human agent the ticket context, failed retrievals, and a clean next step without forcing re-interview of the customer?

Recommend the stack only if that pilot shows restraint under missing docs. Reject it if the model still produces fluent resolutions when the withheld material should have forced handoff—even if a few answers look helpful. The non-obvious cost of a weak gate is organizational: agents learn to mop up overconfident drafts, which reduces pressure to close the knowledge gaps that caused the incomplete base in the first place.