By Hakim Engineering9 min read

Function calling in voice agents: wiring tool use without dead air

Acknowledgment phrases, parallel function calls, timeout fallbacks, and interruption policy for side-effecting tool calls, the patterns that keep a voice agent talking while a backend API is still thinking.

EngineeringProduct

A voice agent that can only talk is a chatbot with a microphone. The moment it needs to check an order status, book an appointment, or look up an account balance, it needs function calling, the same tool-use pattern text-based LLM agents use, but with an extra constraint text agents don't have: the caller is on the line, in real time, waiting through however long the function call takes. This is a practical guide to wiring function calling into a voice agent without the silence (or worse, a stalled TTS stream) that makes callers hang up.

The core pattern

Function calling in a voice agent follows the same shape as any LLM tool-use flow: the LLM's response includes a structured call to a named function with arguments, your code executes that function (hits an API, queries a database, whatever the tool does), and the result gets fed back into the LLM's context so it can generate the actual spoken response. The voice-specific complexity is entirely about what happens to the audio channel during the gap between "LLM decided to call a function" and "function result is back."

The naive implementation fails silently, literally

The default behavior in most frameworks: the LLM pauses generating tokens while it waits for the function result, so the TTS layer has nothing new to synthesize, so the caller hears silence. If the function call takes 200 ms (a fast API), this is barely noticeable. If it takes 1-2 seconds (a slower backend, a third-party API with its own latency, a database query under load), the caller experiences dead air and, in our testing across GCC support-line deployments, dead air past about 800 ms reliably produces "hello? are you there?" or a hangup.

Fix 1: acknowledgment phrases

The simplest fix: before executing the function call, have the agent say something ("let me check that for you", "واحدة لحظة، خليني أتأكد") while the call is in flight. This requires the LLM prompt to be structured so the model emits an acknowledgment utterance as a separate turn before the function call, not bundled with the eventual answer. Otherwise you're back to waiting for the full response (acknowledgment + answer) before TTS has anything to speak.

The practical implementation: split the LLM's response into two turns via prompt instruction ("when calling a tool, first emit a short acknowledgment sentence, then call the tool"), synthesize and play the acknowledgment immediately, and pipe the function result into a second LLM turn that generates the actual answer once the result is available. Both LiveKit Agents and Pipecat support this as a standard pattern: the tool-call event fires, you trigger the acknowledgment TTS call independently of waiting on the tool's return value.

Fix 2: parallel function calls

If your agent's turn might need multiple pieces of information (checking both order status and estimated delivery date, for example), fire the function calls in parallel rather than sequentially. Most LLM providers' function-calling APIs support multiple tool calls in a single turn; awaiting them with asyncio.gather (or your framework's equivalent) rather than one after another cuts the total wait roughly to the slowest single call instead of the sum of all of them.

Fix 3: timeout and fallback framing

Function calls to third-party APIs will occasionally be slow or fail. Set an explicit timeout (2-3 seconds is a reasonable ceiling for a voice interaction) and have a fallback utterance ready ("I'm having trouble pulling that up right now, let me transfer you" or "give me one more moment") rather than letting the caller sit through an unbounded wait. This is also where you decide whether a failed function call should fall back to a human handoff. For anything transactional (payments, account changes), failing loud and handing off is almost always the right call over the agent guessing or retrying silently.

Fix 4: streaming the acknowledgment, not just the answer

If your TTS is genuinely streaming (audio starts before the full acknowledgment text is generated), the time-to-first-audio on the acknowledgment phrase itself matters. A 500 ms delay before "let me check that" starts playing defeats the purpose of having an acknowledgment at all. This is one of the places where a slow TTS provider costs you twice: once on the eventual answer, and once on the acknowledgment meant to cover for the function-call latency. We cover what makes TTS genuinely low-latency (as opposed to labeled "streaming" but not really) in our streaming architecture post.

Function calling and interruptions

If the caller interrupts the agent while a function call is in flight (they've changed their mind mid-lookup, or they're providing additional information unprompted), your pipeline needs to decide whether to cancel the in-flight function call or let it complete and discard the result. For idempotent read-only calls (looking something up), cancelling and letting the caller's new input take priority is usually right. For calls with side effects (initiating a payment, sending a confirmation email), you generally want to let the call complete even if the caller interrupts, then handle the interruption's content in the next turn. Cancelling a side-effecting call mid-flight because of an unrelated interruption is how you get duplicate charges or half-completed transactions. This is worth deciding explicitly per function rather than defaulting to one behavior globally.

Worked example (LiveKit)

from livekit.agents import function_tool

@function_tool
async def check_order_status(order_id: str) -> str:
    """Look up the current status of an order."""
    # Acknowledgment is spoken by the LLM's own turn before this
    # executes, per the prompt instruction pattern above.
    status = await orders_api.get_status(order_id)
    return f"Order {order_id} is {status.state}, expected {status.eta}."

LiveKit's AgentSession handles the acknowledgment-then-tool-call flow automatically once the LLM is prompted to split turns; the framework fires the TTS for the acknowledgment turn without waiting on check_order_status to return.

Checklist before shipping

  1. Every function call that can plausibly take over ~500 ms gets an acknowledgment utterance, tested with the acknowledgment audio measured for its own TTFB.
  2. Independent function calls within a turn fire in parallel, not sequentially.
  3. Every external call has an explicit timeout and a scripted fallback utterance, no unbounded waits.
  4. Side-effecting calls (payments, bookings, anything with real-world consequences) have an explicit interruption policy: complete-then-handle, not cancel-on-interrupt.
  5. Test with a slow/flaky mock of your backend API, not just the happy path. The acknowledgment flow is exactly the thing that's invisible in a fast-network demo and painfully obvious in production.

If you're building on LiveKit or Pipecat, both frameworks' tool-calling APIs are covered in more depth in our LiveKit and Pipecat integration tutorials.

Read next