Latency

Perceived latency is not one HTTP round-trip. It is the full pipeline:

local turn detection → upload → model TTFT / first PCM → player queueing

Typical composite audio-turn backend:

1. User speaks → Session tracks local microphone volume
2. Live chunks hit ASR; asr_partial keeps updating
3. Sustained local silence closes the turn → user_audio_end
4. The gateway sends the first complete LLM clause to TTS while later tokens keep arriving
5. The first PCM packet is enqueued → assistant_audio_start

The current Web example also has one unified audio-turn path: after local silence, it uploads one completed turn. A native Audio LLM or server-side ASR → LLM → TTS can return input transcript, assistant text, and audio over the same SSE response; the cascade plays sentence-sized MP3 segments. It does not run rolling browser ASR.

Practical levers

1. Use streaming ASR with partials when captions must appear during speech. If post-turn authoritative captions are enough, let the audio-turn provider return the input transcript and avoid a second request. Local VAD owns the turn boundary in both cases.
2. Tune silenceTimeoutMs on target devices. The web example starts at 500 ms; noisy rooms or deliberate speakers may need a longer value.
3. A composite backend should stream LLM output and start clause-sized TTS as early as possible. This is an AudioLLMProvider implementation detail; Core consumes only the unified text/audio stream.
4. Reuse connections and playback; cap reply length and maxTokens.
5. Use stricter interruptionDetection during playback so echo does not false-trigger, without delaying real barge-in.

How to measure

Interval	Events	Meaning
Turn end → first audio	user_audio_end → assistant_audio_start	Perceived first-audio latency (primary KPI)
Turn end → first text	user_audio_end → first assistant_text_delta	Text TTFT
Live captions	First asr_partial	Input-side feel

Sample on target devices, speaker volume, and real noise — not only a quiet lab.

Compatibility path

Rolling partials (batch ASR) re-upload accumulated audio — fine for demos. Prefer native WebSocket streaming ASR (Deepgram / ElevenLabs) in production. ASR adapters expose provisional and final transcript segments only. Turn closure stays independent of provider segmentation and network inference.

See TurnDetectionConfig, policy, and the events guide.