跳到主内容
@wquguru
精选86Avi Chawla产品发布/更新

Deepgram Flux TTS:解决语音AI长对话连贯性与中断处理难题

This is the ChatGPT moment for Voice AI.

原文
发到 X
推荐理由

Agent开发者的痛点,Flux解决了语音交互中最难搞的“断片”和“打断”问题,有具体数据支撑,值得接入测试。

This is the ChatGPT moment for Voice AI.

这是语音AI的ChatGPT时刻。

A ten-second sample can show that a synthetic voice sounds natural. But it tells little about whether that voice can stay coherent across an entire conversation.

十秒的样本可以展示合成声音听起来是否自然,但它无法说明该声音能否在整个对话中保持连贯性。

Real callers ask follow-up questions, change their requirements, and interrupt before the agent finishes speaking. The voice must respond quickly without resetting its tone and pacing after every turn.

真实来电者会提出后续问题、更改需求,并在代理说完之前打断。语音必须在每次交互后快速响应,而无需重置其语气和节奏。

This creates two memory problems.

这带来了两个记忆问题。

  • The LLM needs to remember what the caller said. - The speech model needs to retain how the agent has been speaking.
  • LLM需要记住来电者说了什么。 - 语音模型需要保留代理一直以来的说话方式。

A transcript handles the first problem, but not the second. It preserves the words, not their delivery or how much of the previous response the caller actually heard.

转录文本解决了第一个问题,但无法解决第二个。它保留了字词,却没有保留其表达方式或来电者实际听到了多少之前的回复内容。

Deepgram's Flux TTS handles this by maintaining acoustic state inside a persistent streaming session. Each response begins with the speech history from earlier turns, and the first audio arrives in under 200 ms.

Deepgram的Flux TTS通过在持久化的流式会话中维护声学状态来解决这一问题。每个回复都以之前交互中的语音历史开头,且首段音频在不到200毫秒内到达。

Flux also handles interruptions through playback tracking. The application sends the point where playback stopped, and the API reports which words the caller heard.

Flux还通过播放跟踪来处理中断。应用程序发送播放停止的位置,API报告来电者听到了哪些字词。

That gives the LLM an accurate conversation history for its next response.

这让LLM能够为其下一次回复提供准确的对话历史。

In a blind listening evaluation across roughly 14,400 paired comparisons, Flux's best voice recorded a 73.4% naturalness win rate. Cartesia Sonic 3.5 recorded 72.0%.

在对大约14,400组配对比较进行的盲听评估中,Flux的最佳声音获得了73.4%的自然度胜率。Cartesia Sonic 3.5为72.0%。

Build with Flux TTS here: https://fandf.co/3UJLAsY

在此使用Flux TTS进行构建:https://fandf.co/3UJLAsY

I recorded a conversation with a car dealership agent about hybrid and electric cars. I changed my requirements and asked follow-up questions to hear how the voice behaved across the full exchange.

我录制了一段与汽车经销商代理关于混合动力车和电动车的对话。我更改了需求并提出了后续问题,以观察语音在整个交流过程中的表现。

The recording is below, and I worked with the Deepgram team on this post to put it together.

录音如下,我与Deepgram团队合作完成了这篇帖子。

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

关联信息,但可能不是同一事件