Hallucination

The behaviour follows from how the models work. They are trained to produce likely continuations, and a fluent, specific-sounding answer is more likely than an admission of ignorance unless something in the training or the prompt makes the admission preferable. The output is not a lie in any intentional sense; it is a high-probability sequence that happens not to correspond to anything real.

Voice raises the cost of every instance. There is no hyperlink to check, no visible hedging, and the delivery is as confident for a fabrication as for a fact. Callers reasonably treat a spoken statement from a business as authoritative, and a hallucinated commitment — a discount, a booking, an eligibility decision — can create real obligations for the operator.

Speech pipelines can also hallucinate before the model does. Some recognisers, when fed silence, noise or music, emit plausible text rather than nothing, and the agent then responds to something no one said. Filtering low-confidence and non-speech segments before they reach the model closes a failure mode that prompt-level defences cannot touch.

The practical mitigations are consistent: ground answers in retrieved or function-returned data, instruct the model explicitly to refuse rather than guess, keep the scope narrow, and escalate anything outside it. Reviewing transcripts for confident statements that no source supports is the only reliable way to find what slipped through.