LangSmith's voice tracing against the rest: same spans, different audio
In July, LangSmith launched voice tracing for four frameworks: Pipecat, LiveKit, OpenAI Realtime and Gemini Live. The launch post lists what a trace captures, including full conversation audio overlaid on the trace, speech-to-text and text-to-speech latency, voice activity detection events, and interruptions and overlapping speech.
It is not the only way to trace a voice agent, and the alternatives are more alike than their marketing suggests. Most of them read the same spans from the same place. The real differences are further down: which audio each one records, how much of each is actually open source, and what happens when there are no spans to read.
This is the fourth post in a series. The first compared LiveKit and Pipecat on interruptions, the second compared three ways to connect Twilio, and the third followed an ElevenLabs voice onto a phone call and found two routes a LangSmith trace cannot see.
⚠️ What follows is read, not measured. I have not run these tools against one another, and nothing here is a benchmark. Everything comes from each vendor's documentation, with links to each claim. Where a document is silent, I say it is silent rather than guess.
Most of them read the same spans
Start with where the data comes from, because it is usually not the tracing tool.
Pipecat has built-in OpenTelemetry tracing: a conversation span, turn spans inside it, and a span per STT, LLM and TTS call, enriched with attributes such as the provider and time to first byte. Its own documentation lists the destinations for those spans: Jaeger and Grafana over gRPC, and Langfuse, LangSmith, MLflow and SigNoz over HTTP. LiveKit Agents emits OpenTelemetry too, and its data docs describe exporting each session's spans to any compatible backend, such as Langfuse.
The third-party integrations are built on that:
- Langfuse receives Pipecat's spans on its OpenTelemetry endpoint, and for LiveKit the Langfuse SDK registers as a span processor on the tracer provider LiveKit uses.
- Braintrust attaches to LiveKit's built-in OpenTelemetry pipeline the same way, with its own span processor.
- Arize converts Pipecat's traces to OpenInference with openinference-instrumentation-pipecat, for viewing in Arize or Phoenix.
- LangSmith's Pipecat integration, per its own docs, hooks into the spans Pipecat already emits and maps them onto LangSmith's format, one trace per conversation with a span per stage.
So the span tree is the framework's, and it largely comes out the same wherever you send it. That is useful to know before you choose: the latency breakdown per stage, which the LangSmith post rightly calls the core of voice observability, is not what separates these tools.
The integration details still differ in ways that bite. Langfuse notes that one Pipecat trace is one whole conversation, unlike its usual one-trace-per-interaction pattern, and tells you how to patch trace-level input and output. Braintrust's LiveKit guide is TypeScript and warns that LiveKit's JavaScript SDK is built against OpenTelemetry JS 1.x, so every OpenTelemetry package must stay on 1.x. Langfuse's LiveKit guide works around the 2.x side of the same split. Framework telemetry is shared; framework OpenTelemetry versions are not.
What actually differs: which audio
A voice trace without audio is a chat trace with timestamps. Every serious tool here therefore adds audio, and the interesting question is which audio, recorded where.
LangSmith records what the user heard. Its Pipecat integration places its recorder after the output transport, with the comment that this "records what was heard". For OpenAI Realtime the instruction is explicit: record the agent's audio from the speaker, so the recording reflects only what was played, and "audio a barge-in discards before playback is never recorded". The Gemini Live integration says the same. For LiveKit it uses LiveKit's session recordings by default, or an Egress recording to your own storage, and waits a bounded time for the recording before exporting the trace without audio.
That choice matters because of the seam the first post was about. When a caller interrupts, the agent generated audio the caller never heard. A recording of what was heard is the ground truth for what the conversation actually was; a recording of what was generated would contain speech that, from the caller's side, never happened.
LiveKit Agent Insights records what the model heard. LiveKit Cloud's own observability puts transcripts, traces, logs and audio on one timeline, with recordings of both the agent and the user. With noise cancellation on, the user's recording is taken after it, and the docs say what that means: the recording "reflects what the STT or realtime model heard". That is the right audio for debugging a transcription error, and a different artifact from what the caller's microphone picked up.
Braintrust attaches synthesized audio to spans, if you ask. Its Pipecat integration stores no audio by default. Setting two environment variables attaches user speech to user-speaking spans and TTS audio to TTS response spans. That is audio per span rather than one recording of the call, which suits inspecting a single synthesis and says nothing on its own about what a barge-in cut off.
Langfuse and Arize's Pipecat instrumentation document no audio capture in their integrations. Their integration pages above describe spans, timing and token or character counts, and none of them mentions recording or audio attachments. For Langfuse that is a gap in the integration, not the product: its general multi-modality docs accept MP3, WAV and OGG attachments on a trace. Nothing in either integration puts the call recording there for you.
Where the data can live
This is where the first-party option has conditions worth reading closely. LiveKit's Agent Insights works for agents on LiveKit Cloud and for self-hosted agents that connect to LiveKit Cloud media servers, but not with self-hosted media servers or fully self-hosted deployments. It needs Python Agents SDK 1.3.0 or Node 1.0.18 or later. The data is deleted after 30 days on every plan, the storage region is fixed when the project is created, and PII redaction is bundled at no extra cost but off by default.
At the other end, Langfuse describes itself as an open-source platform and documents self-hosting, and Pipecat's plain OpenTelemetry path can go to Jaeger or any collector you run yourself. If the audio of your calls cannot leave your infrastructure, that path exists; you give up the audio-aligned view unless you build it.
How open is the tracing, layer by layer?
"Open source" gets used loosely in this market. A voice trace passes through five layers before you see it, and each has its own answer: the code that produces the spans, the integration that captures them and the audio, the wire format, the schema of what is on the wire, and the server that stores and shows it.
The spans are produced by open code. Pipecat is BSD 2-Clause; LiveKit Agents is Apache 2.0, and so are the LiveKit server and its Egress recorder; the OpenTelemetry SDK is Apache 2.0.
The integrations are open — including the closed products'. LangSmith's voice integrations, audio handling included, live in its MIT-licensed SDK, in folders for Pipecat, LiveKit, OpenAI Realtime, Gemini Live and Google ADK Live. Braintrust's Pipecat and LiveKit integrations are in its Apache 2.0 Python SDK. Arize's OpenInference and Roark's analytics SDK are Apache 2.0. You can read exactly what each vendor captures, and where in the pipeline, which is how this post knows where each one records audio. What you cannot do with an open SDK is change where it sends the data.
The wire is open; what is on it is not shared. Every framework here speaks OTLP, which is why Pipecat can list LangSmith, Langfuse and Jaeger side by side. But the span attributes are each framework's own: Braintrust's LiveKit guide warns that LiveKit's voice spans use lk.* attributes and that filtering to standard AI spans would drop the session, speaking and turn spans. OpenInference exists precisely to convert Pipecat's spans into its own conventions. Moving spans between backends is easy; moving their meaning takes a mapping.
The audio is not portable at all. OTLP carries spans, not recordings. Each tool attaches audio its own way: LangSmith through its attachment API, Braintrust through span attachments behind two environment variables, Langfuse through media references to files in its object storage. The part that makes a voice trace a voice trace is the part that locks you in.
The server is where they split:
- LangSmith is a hosted product. Running it yourself is possible, but self-hosted LangSmith is "an add-on to the Enterprise plan", with a license key.
- Braintrust self-hosts only the data plane: you run the storage, and Braintrust provides "the managed UI, authentication, and platform updates".
- LiveKit Agent Insights runs only in LiveKit Cloud, as covered above, even though everything underneath it is Apache 2.0.
- Arize Phoenix publishes its source under the Elastic License 2.0, which forbids offering it to third parties as a hosted service. That is source-available, not an OSI open-source license.
- Langfuse is MIT outside its enterprise folders. Its own open-source page says every product capability is MIT-licensed without usage limits, and only modules such as SCIM, audit logging and data retention policies need a commercial license when you self-host.
- Jaeger (Apache 2.0) and MLflow (Apache 2.0), both on Pipecat's list of exporters, are fully open, and general-purpose: neither documents anything voice-specific.
ElevenLabs and Twilio are closed services with very different exits. ElevenLabs' agents platform is closed, but it exports OpenTelemetry: OTLP JSON from a post-call webhook, from the conversation API, or live from a monitoring WebSocket, all with the same attribute names, for Datadog, Grafana Tempo, Honeycomb "or any backend that ingests OTLP". It does not push to your collector — you receive the JSON and forward it — and long tool parameters are cut at 4 KB per span attribute. The OpenTelemetry webhooks carry no audio; the recording comes from a separate audio webhook as MP3, and the OpenTelemetry page notes audio webhooks are never retried. So a closed voice platform can still feed a fully open tracing backend, minus the audio unless you catch that webhook. Twilio's ConversationRelay documents nothing comparable: its observability is Conversation Intelligence (classic), switched on per call with an intelligenceService attribute, which stores the transcript and runs Twilio's Language Operators on it. By default, ConversationRelay transcripts are not stored by Twilio at all, and the Intelligence features are billed separately. That is analysis of a transcript inside Twilio, not spans you can take elsewhere.
So a fully open-source voice-tracing stack exists, but you assemble it. Pipecat, or LiveKit Agents on your own LiveKit server, emits the spans; OpenTelemetry carries them; a self-hosted Langfuse — or Jaeger, if you do not need an LLM-aware UI — stores and shows them. The audio is the part you wire yourself: Egress can record the room, and Langfuse can store audio attachments (MP3, WAV and OGG) on a trace, but neither Langfuse integration page connects the two for you. And the one retention control you would want for call recordings, data retention policies, is on the commercial side of Langfuse's line.
When the pipeline is someone else's
The third post ended on two routes — Twilio's ConversationRelay and ElevenLabs' own agents platform — where the speech pipeline runs on someone else's servers, so none of the framework integrations above has anything to hook into. The section above changes half of that: ElevenLabs exports its own OpenTelemetry traces after (or during) each conversation, so its route can reach the same backends as everything else. ConversationRelay remains the route with no spans to export.
That gap is what call-level platforms fill. Roark is the clearest example in its integration matrix: it pulls production calls in from the platforms themselves rather than from your code. For ElevenLabs agents, its integration pulls conversation audio, transcripts and metadata on a schedule, with a condition that echoes the retention theme: agents with Zero Retention Mode or recording disabled cannot sync calls. For anything without an integration — a ConversationRelay call you recorded yourself, say — its API accepts a recording URL and does the transcription and analysis itself. It also instruments code-first agents directly, placing its Pipecat observer after the output transport to capture the bot's post-TTS audio, the same "what was heard" position LangSmith chose.
What you get from that is the call, scored against your metrics, not the pipeline: no span per STT request when the STT belongs to the platform.
A table, with the gaps left in
| Tool | Frameworks (per the linked docs) | Audio | Span source | Data location | Server license |
|---|---|---|---|---|---|
| LangSmith | Pipecat, LiveKit, OpenAI Realtime, Gemini Live | recording of what was heard | framework spans, mapped | hosted; self-hosting is an Enterprise add-on | closed (SDK MIT) |
| LiveKit Agent Insights | LiveKit Agents | agent and user recording, user after noise cancellation | LiveKit's own | LiveKit Cloud, 30 days | cloud feature (framework and server Apache 2.0) |
| Langfuse | Pipecat, LiveKit | not in the integrations; MP3/WAV/OGG attachments supported | framework OpenTelemetry | cloud or self-hosted | MIT, except enterprise modules |
| Arize (OpenInference) | Pipecat | not documented | converted to OpenInference | not covered here | Phoenix: Elastic 2.0, source-available |
| Braintrust | Pipecat, LiveKit | opt-in, attached per span | framework OpenTelemetry | hosted, or your own data plane | closed (SDK Apache 2.0) |
| Roark | platforms and code-first agents | call recordings | call-level, not pipeline | not covered here | hosted (SDK Apache 2.0) |
| ElevenLabs agents | its own platform | separate MP3 audio webhook | its own OTLP JSON export, forwarded by you | ElevenLabs | closed service |
| Twilio ConversationRelay | its own platform | not documented | none documented; transcripts via Conversation Intelligence | Twilio | closed service |
"Not covered here" means I did not read that vendor's retention or hosting docs for this post, not that the answer is bad.
How I would actually choose
If you run on LiveKit Cloud, turn on Agent Insights first. It is already there, the audio is aligned with the traces, and the conditions — LiveKit Cloud media servers, 30 days, redaction off until you turn it on — are the things to check against your requirements.
If your text agents already live in LangSmith, its voice integrations put both kinds of agent under one set of review and evaluation workflows, which is the launch post's own argument, and the recording-what-was-heard design is the right one for debugging interruptions.
If you need it fully open source, the stack is Pipecat or LiveKit Agents, OpenTelemetry, and a self-hosted Langfuse, with the call recording attached to the trace by your own code. Budget for that wiring, and read Langfuse's enterprise line before you assume retention policies come with it. Phoenix is self-hostable too, but under a source-available license, not an open-source one.
If your speech pipeline is ElevenLabs', forward its OpenTelemetry export to whichever backend you chose above, and catch the audio webhook if you want the recording, because the OpenTelemetry payloads carry none.
If it is Twilio's ConversationRelay, there are no spans to forward: turn on Conversation Intelligence for transcripts, record the call yourself if you need audio, and use a call-level platform like Roark to score it.
And whichever you choose, check which audio it keeps before you trust it in an incident. The recording of what the caller heard and the recording of what your agent generated disagree exactly when something went wrong.