Emergency dispatch triage
A caller in distress will talk over a slow bot. The agent has to yield instantly, keep the address and the symptom in memory, and never replay a half-sentence the caller already corrected.
Case study ยท NerdWare Tech
Talk over it.
It stops.
A streaming voice agent built to answer inside 500 ms and go silent the instant you cut in.
Channel scope
simulated signal
CALLER
AGENT
TTS buffer
State: LISTENING
VAD armed. Waiting for an agent turn.
Where the 500 ms goes.
Each stage starts streaming before the last one finishes. The legacy cascade waits for whole files at every hop.
Target budget / illustrative. Cascade figures are a representative range for a record, upload, batch-transcribe, complete, synthesize bot (about 1.5 to 2.5 s), not a benchmark of any product.
One stream, no files, no polling.
Audio frames move through memory from the caller's mic to the caller's speaker. Nothing is written to disk and nothing waits on a poll.
- CallerBrowser or phone mic, Opus framesin
- LiveKit SFUWebRTC transport<100 ms
- Agent workerLiveKit Agents SDK. Local Silero VAD runs here.<5 ms VAD
- Deepgram Nova-3Streaming STT, interim then final<150 ms
- Llama 3.3 70BOn Groq or Cerebras, streamed tokens, memory retrieved in parallelTTFT first
- Cartesia Sonic-3Streaming TTS, first byte out<90 ms
The cancel path
When the VAD hears the caller during agent speech, the worker does four things in one tick, in this order.
- Mark the turn as interrupted.
- Abort the in-flight LLM stream.
- Close the TTS stream and flush every pending audio buffer, so no stale words play after the cut.
- Keep the partial reply in memory with an interrupted flag, then hand the floor to STT.
Stateful memory sits beside the LLM: the last turns plus retrieved facts for this caller, fetched while STT is still finalizing.
Every turn reports its own latency.
The agent emits four timings per turn, so a slow stage is visible the same hour it regresses. The rows below are simulated by the scope above.
| Turn | Transport | STT | LLM TTFT | TTS TTFB | Round trip | Barge-in |
|---|
Built for people who are already stressed.
Insurance claims intake
Long, interrupted narratives. Barge-in lets the caller fix a date mid-sentence without restarting the form.
Technical interviews
Natural turn-taking matters more than polish. A reply that starts under half a second feels like a person, not a queue.
The stack, by job.
- Transport
- LiveKit Agents SDK over a WebRTC SFU
- Hearing
- Deepgram Nova-3 streaming STT, local Silero VAD
- Thinking
- Llama 3.3 70B on Groq or Cerebras, with stateful memory retrieval
- Speaking
- Cartesia Sonic-3 streaming TTS
- Observing
- Per-turn telemetry: STT, TTFT, TTFB, round trip
NerdWare Tech
Want an agent that answers the phone?
Tell us the calls you want handled. The waveform above is a simulation and the latency figures are target budgets.