RexaThe short answer
A voice agent is the whole assembly: recognition, a language model with instructions and tools, synthesis, turn-taking, and a telephony path to the caller. What distinguishes it from an IVR is that the caller states their problem in their own words. What distinguishes it from a chatbot is that everything must happen inside a conversational pause.
Read the full guide
This entry is the short definition. Definition of a voice AI agent, how the speech-to-text, language model, and text-to-speech loop works, and where it fits against an IVR.
What is a voice AI agent?In detail
The loop is straightforward to describe and hard to make feel natural. Audio streams in and is transcribed. Turn logic decides the caller has finished. The transcript, the system prompt, the conversation history and any tool results go to a language model. The reply is synthesised and played, while the system simultaneously listens for the caller interrupting. Each cycle is one turn.
Real deployments need more than the loop. They need tools, so the agent can look up an order or book a slot rather than guess. They need an escalation path to a human that carries context. They need a record of what happened in a form other systems can consume. And they need guardrails, because a model that will say anything is unusable in a regulated conversation.
The honest limitations are worth stating on the same page as the capabilities. Voice agents are poor at judgement calls, at negotiation, and at anything where being confidently wrong is expensive. They struggle with heavy accents on poor lines, with callers who are distressed, and with requests that fall outside what they were given tools for. A clean handoff matters more than breadth of capability.
Related terms
Conversational AI
Conversational AI is the field of systems that interact with people through natural-language dialogue across text or speech.
IVR
An IVR is an automated phone system that presents callers with a predefined menu and routes them according to keypad or spoken selections.
Function calling
Function calling is the mechanism by which a language model requests execution of a defined external operation and incorporates its result into the conversation.
Latency budget
A latency budget is an allocation of the maximum acceptable end-to-end response delay across each component that contributes to it.