Add Respeecher TTS plugin#1646
Conversation
🦋 Changeset detectedLatest commit: 45c4539 The changes in this PR will be included in the next version bump. This PR includes changesets to release 34 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| } finally { | ||
| timeout.cleanup(); | ||
| this.queue.close(); | ||
| } |
There was a problem hiding this comment.
🔴 ChunkedStream.run() closes queue in finally block, breaking retry behavior
The finally block at line 380 unconditionally calls this.queue.close(). The base class ChunkedStream (agents/src/tts/tts.ts:567) wraps run() inside mainTask().finally(() => this.queue.close()) and retries run() on retryable errors (like APITimeoutError or APIConnectionError, which are retryable by default per agents/src/_exceptions.ts:119,136). When a retry occurs, sendLastFrame checks !queue.closed and silently drops all audio frames, so the retry "succeeds" but produces zero output. The MiniMax plugin (plugins/minimax/src/tts.ts:429-432) explicitly documents this pitfall: "Do NOT close this.queue here. The base class wraps run() in mainTask().finally(() => this.queue.close()) and retries run() on retryable errors."
| } finally { | |
| timeout.cleanup(); | |
| this.queue.close(); | |
| } | |
| } finally { | |
| timeout.cleanup(); | |
| } | |
Was this helpful? React with 👍 or 👎 to provide feedback.
| const markInputEnded = sendTask().then(() => { | ||
| inputEnded = true; | ||
| }); |
There was a problem hiding this comment.
🟡 Unhandled promise rejection from markInputEnded when WebSocket closes unexpectedly
In recvTask, markInputEnded = sendTask().then(...) is created at line 460. If the inner await new Promise<void>() at line 464 rejects first (e.g., due to an unexpected WebSocket close at line 523-529), the function exits before reaching await markInputEnded at line 537. Meanwhile, sendTask() will eventually call sendWsJson on the closed WebSocket, which throws APIConnectionError (since signal.aborted is false in the non-abort close scenario). This makes markInputEnded a rejected promise with no handler, triggering Node.js UnhandledPromiseRejection warnings or process crashes depending on configuration.
Prompt for agents
In SynthesizeStream.run()'s recvTask (plugins/respeecher/src/tts.ts), the markInputEnded promise created from sendTask().then(...) at line 460 can become an unhandled rejected promise if the inner await new Promise<void>() rejects before await markInputEnded is reached at line 537. To fix this, attach a no-op .catch() to markInputEnded to suppress the unhandled rejection warning, or restructure to ensure markInputEnded is always awaited (e.g., using Promise.allSettled or try/finally around the inner await). For example: const markInputEnded = sendTask().then(() => { inputEnded = true; }).catch(() => {}); -- or use a finally block that always awaits markInputEnded regardless of whether the inner promise rejected.
Was this helpful? React with 👍 or 👎 to provide feedback.
Description
This PR adds a new TTS plugin for LiveKit Agents, integrating Respeecher’s synthesis and streaming API. It currently supports the English voice model and the Ukrainian voice models.
Why Respeecher?
Respeecher’s technology is notable not just for voice quality but also for its ethical design and safeguards. This integration offers: