4 posts tagged with "voice-agents"

Pick a voice stack and you are really picking two things: who owns the audio transport, and what you can see when a turn goes wrong. The vendor list barely matters — LiveKit Agents and Pipecat both put the same speech-to-text, language model and text-to-speech providers behind the same three seams, and both let you swap any of them in a line.

LangSmith shipped tracing for both in July, alongside OpenAI Realtime and Gemini Live. That is a useful forcing function for this comparison, because a trace shows you where a framework's abstraction actually sits — and the two answers are genuinely different.

Here is a fact that should make telephony simple. A Twilio call carries μ-law audio at 8000 Hz, and ElevenLabs' text-to-speech API will produce exactly that if you ask for the ulaw_8000 output format. One vendor's output is the other vendor's wire format. No resampling, no transcoding.

And yet none of the four common ways to put an ElevenLabs voice on a Twilio call just passes those bytes through. Each one converts somewhere, and where it converts tells you who is in charge of the audio.

"Add Twilio" sounds like one integration. It is three, and they disagree about the most basic question in a voice agent: who holds the audio. LiveKit takes the call as a SIP trunk and turns the caller into a participant in a room. Pipecat takes it as a WebSocket carrying raw audio frames. And Twilio's own ConversationRelay keeps the audio entirely, and hands your server text.

The previous post compared LiveKit and Pipecat on the seam where a voice agent breaks: a caller talking over the bot. Put a phone line in front of either one and that seam moves, because now there is a third system buffering audio that neither framework controls.

In July, LangSmith launched voice tracing for four frameworks: Pipecat, LiveKit, OpenAI Realtime and Gemini Live. The launch post lists what a trace captures, including full conversation audio overlaid on the trace, speech-to-text and text-to-speech latency, voice activity detection events, and interruptions and overlapping speech.

It is not the only way to trace a voice agent, and the alternatives are more alike than their marketing suggests. Most of them read the same spans from the same place. The real differences are further down: which audio each one records, how much of each is actually open source, and what happens when there are no spans to read.