An ElevenLabs voice on a Twilio call: four routes, one 8 kHz wire
Here is a fact that should make telephony simple. A Twilio call carries μ-law audio at 8000 Hz, and ElevenLabs' text-to-speech API will produce exactly that if you ask for the ulaw_8000 output format. One vendor's output is the other vendor's wire format. No resampling, no transcoding.
And yet none of the four common ways to put an ElevenLabs voice on a Twilio call just passes those bytes through. Each one converts somewhere, and where it converts tells you who is in charge of the audio.
This is the third post in a short series. The first compared LiveKit and Pipecat on interruptions; the second compared the three ways to connect Twilio. This one follows the voice itself from ElevenLabs to the caller's ear, and then asks which of those paths you can still see in a trace.
⚠️ What follows is read, not measured. I have not compared the audio quality or the latency of these routes on real calls, and I would distrust a comparison that did not name the carrier, the handset and the network. Everything below comes from the documentation and the published source, with links to each claim.
Route 1: Pipecat asks for 8 kHz PCM and encodes at the edge
Pipecat chooses ElevenLabs' output format from your pipeline's sample rate. The mapping in its source is a plain match statement: 8000 becomes pcm_8000, 16000 becomes pcm_16000, and so on up to 48000. An unsupported rate logs a warning and falls back to pcm_24000.
So when you follow Pipecat's Twilio guide and set the pipeline to 8000 Hz, ElevenLabs sends 8 kHz linear PCM. The Twilio serializer then encodes it to μ-law with pcm_to_ulaw at the moment it leaves for Twilio. The sample rate already matches, so no resampling happens, and μ-law encoding is a cheap per-sample operation.
Why not ask ElevenLabs for ulaw_8000 and skip the encode? Because the pipeline wants PCM. Everything between the TTS service and the transport — anything that inspects, mixes or measures audio frames — works on linear samples, and a compressed format in the middle of the pipeline would make every one of those processors decode it first. Encoding at the edge is the right design for a framework whose whole point is that you can put your own processor anywhere.
The default model, in the service source, is eleven_flash_v2_5.
Route 2: LiveKit asks for MP3 and lets SIP transcode
LiveKit's ElevenLabs plugin defaults to a different format entirely. Its source sets mp3_22050_32 as the default encoding, derives the sample rate by splitting the format name, and supports MP3, Opus and PCM encodings, but not μ-law.
That makes sense once you remember how LiveKit reaches Twilio: over a SIP trunk, with the caller joining a room as a participant. The agent does not know the caller is on a phone. It publishes audio to the room like it would for a browser, and the conversion to telephone audio happens at LiveKit's SIP edge. That is the same bargain as everywhere else in LiveKit: the transport is the framework's business, including the last codec.
The default model here is eleven_turbo_v2_5, which the plugin's documentation names too. So the two frameworks disagree on the default ElevenLabs model, and on a phone call you will hear one or the other unless you set it yourself.
Route 3: ConversationRelay picks the voice for you, as an attribute
With Twilio's ConversationRelay, you never touch ElevenLabs' API at all. ElevenLabs is the default text-to-speech provider, and Twilio runs the synthesis, so the audio never crosses your server.
Choosing a voice becomes string formatting. The voice configuration guide says ElevenLabs voices use the Flash 2.5 model by default, and you pick another model — Flash 2, Turbo 2.5 or Turbo 2 — by appending it to the voice ID with a hyphen. Speed, stability and similarity go on the end the same way, as an underscore-separated triple, with speed limited to 0.7 to 1.2 and the other two to 0.0 to 1.0.
That is less control than either framework gives you: no WebSocket-versus-HTTP choice, no flush control, no word timestamps to handle yourself. In exchange, the ElevenLabs part of your system is one TwiML attribute.
Route 4: ElevenLabs owns the whole agent, and Twilio is just the number
The fourth route removes the frameworks. ElevenLabs' agents platform runs speech recognition, the conversation and the voice itself, and connects to Twilio three ways:
- Native integration. You import a Twilio number with its SID and auth token, and ElevenLabs configures the number automatically. Purchased numbers work inbound and outbound; verified caller IDs work outbound only. The docs recommend a restricted Twilio API key rather than account-wide credentials.
- Register call. Your own Twilio webhook calls ElevenLabs'
register_call, gets back TwiML to hand to Twilio, and returns it. You keep your call routing; the docs are explicit that call transfers are not supported this way. - SIP trunk. ElevenLabs' SIP deployment sends and receives G.711 at 8 kHz or G.722 at 16 kHz, and reads Twilio's own call-ID header when standard headers are absent, so a Twilio Elastic SIP trunk works without extra configuration.
That G.722 detail matters: it is wideband audio at 16 kHz, twice the telephone rate, available only where every hop on the path supports that codec. LiveKit lists HD Voice among its SIP features for the same reason.
What the four routes have in common
Put the conversion points side by side:
| Route | ElevenLabs sends | Converted to telephone audio by |
|---|---|---|
| Pipecat + Media Streams | 8 kHz PCM | Pipecat's Twilio serializer |
| LiveKit + SIP trunk | MP3 by default | LiveKit's SIP edge |
| ConversationRelay | never reaches you | Twilio |
| ElevenLabs agents | never reaches you | ElevenLabs or Twilio |
Nobody uses ulaw_8000 on the path you control. The routes where you hold the audio want linear PCM or a streaming codec, because audio you can still process is worth more to them than the encode they would save. The routes where you do not hold the audio do the conversion themselves.
Which routes a LangSmith trace can see
LangSmith's voice tracing, published in July, ships native integrations for four things: Pipecat, LiveKit, OpenAI Realtime and Gemini Live. A trace carries the full conversation audio overlaid on the spans, speech-to-text and text-to-speech latency, voice activity detection events, interruptions and overlapping speech, and timing across each stage of the pipeline.
Map that onto the four routes and the line falls exactly where the audio ownership does:
- Pipecat over Media Streams is covered.
configure_pipecat()plus the tracing flags on the pipeline task gives you one trace per conversation with a span per pipeline stage (STT, LLM, TTS), so the ElevenLabs stage is its own span. The integration is marked beta, so expect its API to move. - LiveKit over a SIP trunk should be covered: you call
configure_livekit()before the server starts, as for any LiveKit agent. LangSmith's page never mentions SIP or phone calls, and it should not need to: per LiveKit's SIP docs, a phone caller is a room participant like any other, so the agent session being traced is the same one a browser caller would get. That is my reading of the two docs together, not something either one states. - ConversationRelay is not in the list, and it could not be in the same way. Speech recognition and ElevenLabs synthesis run inside Twilio; your server only ever sees the caller's words and sends back text. You can still trace your own model call, as for any text agent, but the stages that make a voice agent feel slow are not on your side of the WebSocket.
- ElevenLabs' agents platform is not in the list either. By default the model turn runs on ElevenLabs' side too, but you can connect your own LLM server — and then you are where ConversationRelay leaves you: the model call is yours to trace, the speech stages are not.
So the route decision is also a visibility decision. The two routes that keep the audio on your side are the two whose voice pipeline a LangSmith trace can open up, and the two that take the audio away also take away the spans you would use to find a slow turn or a bad interruption.
How I would actually choose
Choose based on who should own the voice, not on the voice itself, because every route here can reach the same ElevenLabs voices.
If the voice is part of a pipeline you are building, take Pipecat, set the pipeline to 8000 Hz so ElevenLabs sends audio at the rate the phone will play, and choose the model deliberately rather than inheriting a default. Turn the LangSmith tracing on from the first call.
If the phone is one channel among several, take LiveKit, let the SIP edge handle the codec, and check which model you are running, because it is not the same default as Pipecat.
If you only need the agent to talk, ConversationRelay gets you an ElevenLabs voice for the price of a TwiML attribute, and ElevenLabs' own platform gets you the whole agent for the price of a phone-number import. Both cost you the per-stage trace, so decide up front how you will debug a slow turn.
Whichever you choose, listen to the result from an actual phone. A voice chosen by listening to a 44.1 kHz preview will not sound the same through 8 kHz μ-law, and every route on this page ends with the caller hearing it through that codec.