How to Reduce AI Chatbot Hallucinations

Why chatbots invent things

A language model produces the most plausible continuation of a conversation. That is the whole job. When a visitor asks what your callout fee is and the model has never seen your price list, the most plausible continuation is still a number, a reasonable-sounding one, delivered with the same confidence as a fact.

This gets called hallucination, which makes it sound like a malfunction. It is closer to the opposite: the system is doing exactly what it was built to do, in a situation where nobody gave it the material to do it correctly.

That reframing matters, because it points at the fix. You cannot instruct fluency out of a model. You can control what it is asked and what it has to work with.

Control 1: gate on retrieval

The strongest control in the Grounded AI Agent & Knowledge Base is Strict mode. It checks for relevant knowledge before answering, and when it finds none it can return your configured refusal without calling the answer model at all.

That distinction is the whole point. A prompt saying “only use the provided context” is an instruction the model can drift from. A gate that never sends the question is not something the model can drift from, because the model is never involved.

Use Strict wherever a wrong answer costs money or trust: prices, coverage, eligibility, warranty terms, availability. Balanced is fine for softer questions where tone matters more than precision.

Control 2: tune retrieval strictness

Gating only helps if “relevant” means what you think it means. Retrieval strictness controls how closely a passage must match before it counts.

Set too loose, a passage about one service gets used to answer about another, technically grounded, practically wrong. Set too tight, valid questions get refused because your source phrased something differently from the customer.

Read a week of transcripts and adjust. Loosely related answers mean tighten. Refusals on questions you clearly documented mean loosen, or rewrite the source in customer language.

Control 3: write a refusal you would send yourself

Most refusal messages are written in thirty seconds and read like an error page. That is a wasted opportunity, because the refusal is the moment a careful visitor decides whether you are trustworthy or useless.

A good one does three things: says what it cannot confirm, avoids apologising four times, and offers the next step. “I don’t have that detail confirmed, I can pass you to someone who does. Want me to?” is worth writing properly, because visitors will see it more often than you expect.

Control 4: escalate before it becomes a pattern

Set the number of refusals that triggers a handoff. Two is a reasonable default: one refusal is a gap, three in a row is a bad experience.

Turn limits, timeouts and provider errors escalate the same way. When any of them fire, the conversation moves into the Shared Team Inbox with its history intact and bot ownership stops, so a person owns the reply from that point rather than the visitor being handed back and forth.

Control 5: turn on source attribution

Optional source attribution shows where an answer came from. Its most valuable use is not the visitor. It is you, reading transcripts and spotting that a run of odd answers all trace back to one outdated document. Without attribution that is guesswork; with it, it is a five-minute fix.

Control 6: fix the knowledge, not the prompt

When the chat gets something wrong, the instinct is to rewrite the instructions. Usually the real cause is upstream: the fact was never added, was added in language nobody uses, or was added twice with two different answers.

Contradictory sources are the quiet killer. If your terms document says 14 days and your FAQ says 30, retrieval will find one of them, more or less at random. Delete or correct the loser.

What this adds up to

Together these controls change the failure mode from “confident wrong answer” to “honest refusal and a human”. That is a genuine, meaningful improvement in what your customers experience.

It is not a promise of perfection, and any vendor offering one is describing something that does not exist. Grounding controls guide behaviour. They reduce fabricated answers substantially. They do not certify every response, and human availability still shapes what happens after a handoff. We would rather you configure the escalation path properly than believe you will never need it.

A weekly review that catches drift

Configuration gets you a good starting point. Reading conversations is what keeps it good.

Fifteen minutes a week is enough. Open the recent conversations and sort what you find into three piles.

Answered well. Nothing to do, but note the questions, the frequent ones are candidates for a structured flow step, which is faster and cheaper than an AI turn.

Refused. Each refusal names a missing knowledge source. Add it, or accept that the question is genuinely outside what you want the chat handling.

Answered wrongly. The most important pile and usually the smallest. Trace it back: if source attribution is on, you will often find the same outdated document behind several of them.

Do this for a month after launch, then monthly. The failure mode this prevents is the slow one, where the knowledge base ages while the configuration stays perfect.

Where the remaining risk actually sits

With grounding gated, retrieval tuned, refusals written and escalation configured, the residual risk is concentrated in two places, and both are content problems rather than model problems.

Outdated sources that still retrieve well. A confidently retrieved wrong price is worse than a refusal, because it looks correct. This is why deleting stale content matters as much as adding new content.

Contradictory sources. Two documents disagreeing means retrieval picks one more or less at random. Audit for this whenever you update anything commercial.

Neither is fixed by a setting. Both are fixed by someone reading the knowledge base occasionally, which is the least glamorous and most effective control available.

Read next: what happens when the AI can’t answer, or Strict, Balanced and Open grounding modes compared.