What is turn taking?
Turn taking is the management of who speaks when during a live exchange. It covers the decision that a caller has finished speaking and the handling of an interruption while the agent is mid-sentence. Human conversation negotiates all of this in a few hundred milliseconds, which is the standard a voice agent is measured against whether or not that is fair.
Also called: Turn management
What a half second of hesitation signals to a caller
Ask anyone who has used a voice agent what was wrong with it and the answer is about timing rather than about understanding. The agent interrupted them, or it waited so long they started again, and both failures come from one mechanism deciding when a turn has ended. That decision rests on partial information, since a pause of six hundred milliseconds is a finished sentence in one caller and a moment of thought in another. Tuning it trades an agent that talks over people against an agent that feels slow, and the right setting differs by language.
- Interruption and hesitation are both failures of the same endpointing decision.
- Endpoint thresholds that suit English are wrong for several Indic languages.
- A caller who is interrupted repeats themselves, which degrades recognition further.
- Barge-in handling determines whether a caller can correct the agent at all.
How turn taking works
- Voice activity detection marks where caller speech starts and stops.
- An endpointing policy decides when a pause has become the end of a turn.
- Partial transcripts are examined for syntactic completeness alongside the pause length.
- The agent begins generating only once the turn is judged to have ended.
- Caller speech during playback triggers barge-in, which stops the agent immediately.
- Interrupted agent audio is recorded as truncated, so the transcript matches what was heard.
How turn taking is evaluated
Turn taking is assessed on the two failure directions separately, because tuning trades one against the other. Measure the following on live calls per language:
- rate at which the agent speaks over a caller mid-utterance
- distribution of pause length before the agent replies
- false endpoint rate on callers who pause to think
- barge-in detection latency during agent playback
- caller repetition rate as a proxy for perceived interruption
Limitations, risks and common failure modes
- A single endpoint threshold is applied across every language in a deployment.
- Background speech on a shared line is treated as the caller taking a turn.
- Barge-in stops playback while the agent's logic continues from the full utterance.
- Latency work shortens the pause until the agent interrupts slower speakers.