The short answer
The system prompt is where a voice agent’s persona, scope and refusals are defined. It is sent with every turn, so its length is paid for on every response, and its content is the primary control surface for behaviour. For voice specifically it must also constrain output length and format, because everything it produces will be spoken aloud.
In detail
A workable structure covers role and scope, the specific task, style constraints, escalation rules, and explicit boundaries. Style constraints matter unusually much in voice: without an instruction to keep responses to a sentence or two, models produce paragraphs that take an uncomfortably long time to hear, and the caller starts talking over the agent.
Formatting instructions need to be voice-aware because the output is synthesised, not rendered. Markdown, bullet characters and emoji either get read aloud or silently dropped. Numbers, dates and currency should be written the way they are to be spoken. Long enumerations should be broken into two or three options at a time with a check-in, because a caller cannot rewind.
Prompts should state what the agent must not do as explicitly as what it should. Do not quote prices that are not in the provided data. Do not confirm anything a function has not returned. Transfer to a human when the caller asks, when they become distressed, or when the request falls outside scope. These negative instructions are what keep a plausible-sounding model from inventing commitments.
Treat the prompt as versioned configuration rather than a text box. It changes agent behaviour globally and immediately, so changes deserve review, a record of who made them and why, and the ability to roll back — the same discipline applied to any other production configuration.
Related terms
Voice agent
A voice agent is software that conducts a spoken conversation with a person over a phone line or a browser connection, understanding natural speech and responding in speech.
Grounding
Grounding is the practice of constraining a model’s responses to information supplied from a verified source rather than its parametric knowledge.
Hallucination
A hallucination is model output that is fluent and confident but not supported by the model’s inputs or by fact.
Function calling
Function calling is the mechanism by which a language model requests execution of a defined external operation and incorporates its result into the conversation.