Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/concepts/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ Used with `agent.with_tts()`. Each TTS vendor produces audio at a specific sampl
| `MiniMaxTTS` | MiniMax | `model` for supported Agora-managed global models; `key`, `group_id`, `model`, `voice_id`, `url` for BYOK | — |
| `GenericTTS` | Generic OpenAI-compatible TTS over HTTP(S) | `url` | Configurable |
| `DeepgramTTS` | Deepgram | `api_key`, `model` | Configurable |
| `GradiumTTS` | Gradium | `api_key` | Configurable |
| `MistralTTS` | Mistral | `api_key` | — |
| `SarvamTTS` | Sarvam | `api_key` | — |
| `XaiTTS` | xAI | `api_key`, `language` | Configurable |

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/regional-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ If you omit `with_stt()`, AgentKit uses `FengmingSTT` by default for `Area.CN` c

| Client area | STT classes | LLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |

Global client example:
Expand Down
24 changes: 23 additions & 1 deletion docs/reference/vendors.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Construct vendors directly from `agora_agent`, then bind a client with `Agent(cl

| Area | STT classes | LLM classes | TTS classes | Avatar classes |
|---|---|---|---|---|
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |

Global example:
Expand Down Expand Up @@ -303,6 +303,18 @@ The SDK also includes named helpers for the remaining Agora-supported LLM provid
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Deepgram TTS parameters, flattened into `params` |
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |

### `GradiumTTS`

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | `str` | Yes | — | Gradium API key |
| `url` | `str` | No | `None` | WebSocket endpoint for streaming TTS output |
| `model_name` | `str` | No | `None` | Gradium TTS model name (e.g., `default`) |
| `voice_id` | `str` | No | `None` | Gradium voice identifier |
| `sample_rate` | `int` | No | `None` | Audio sample rate in Hz |
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Gradium TTS parameters, flattened into `params` |
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |

### `HumeAITTS`

| Parameter | Type | Required | Default | Description |
Expand Down Expand Up @@ -389,6 +401,16 @@ AgentKit serializes `credential_mode` at the top level of the Rime TTS configura

`key` and `group_id` are required together for BYOK. `url` is optional. In both BYOK and managed modes, exactly one of `voice_id` or `timber_weights` must be provided. Without `key`, `model` must be one of the supported Agora-managed MiniMax models.

### `MistralTTS`

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | `str` | Yes | — | Mistral API key |
| `model` | `str` | No | `None` | Mistral TTS model name (e.g., `voxtral-mini-tts-2603`) |
| `voice` | `str` | No | `None` | Mistral voice identifier |
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Mistral TTS parameters, flattened into `params` |
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |

### `MurfTTS`

| Parameter | Type | Required | Default | Description |
Expand Down
4 changes: 4 additions & 0 deletions src/agora_agent/agentkit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@
GenericAvatar,
GoogleSTT,
GoogleTTS,
GradiumTTS,
Groq,
HeyGenAvatar,
HumeAITTS,
Expand All @@ -185,6 +186,7 @@
MicrosoftSTT,
MicrosoftTTS,
MiniMaxTTS,
MistralTTS,
MurfTTS,
OpenAI,
OpenAIRealtime,
Expand Down Expand Up @@ -393,12 +395,14 @@
"CartesiaTTS",
"DeepgramTTS",
"GoogleTTS",
"GradiumTTS",
"AmazonTTS",
"HumeAITTS",
"RimeTTS",
"FishAudioTTS",
"MiniMaxTTS",
"MiniMaxCNTTS",
"MistralTTS",
"TencentTTS",
"BytedanceTTS",
"CosyVoiceTTS",
Expand Down
4 changes: 4 additions & 0 deletions src/agora_agent/agentkit/regional_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,11 @@
ElevenLabsTTS,
FishAudioTTS,
GoogleTTS,
GradiumTTS,
HumeAITTS,
MicrosoftTTS,
MiniMaxTTS,
MistralTTS,
MurfTTS,
OpenAITTS,
GenericTTS,
Expand Down Expand Up @@ -99,6 +101,8 @@
GenericTTS,
XaiTTS,
DeepgramTTS,
GradiumTTS,
MistralTTS,
]
GlobalLLM = typing.Union[
OpenAI,
Expand Down
4 changes: 4 additions & 0 deletions src/agora_agent/agentkit/vendors/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,11 @@
ElevenLabsTTS,
FishAudioTTS,
GoogleTTS,
GradiumTTS,
HumeAITTS,
MicrosoftTTS,
MiniMaxTTS,
MistralTTS,
MurfTTS,
OpenAITTS,
GenericTTS,
Expand Down Expand Up @@ -92,12 +94,14 @@
"CartesiaTTS",
"DeepgramTTS",
"GoogleTTS",
"GradiumTTS",
"AmazonTTS",
"HumeAITTS",
"RimeTTS",
"FishAudioTTS",
"MiniMaxTTS",
"MiniMaxCNTTS",
"MistralTTS",
"MurfTTS",
"SarvamTTS",
"GenericTTS",
Expand Down
4 changes: 4 additions & 0 deletions src/agora_agent/agentkit/vendors/catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,11 @@
ElevenLabsTTS,
FishAudioTTS,
GoogleTTS,
GradiumTTS,
HumeAITTS,
MicrosoftTTS,
MiniMaxTTS,
MistralTTS,
MurfTTS,
OpenAITTS,
GenericTTS,
Expand Down Expand Up @@ -102,6 +104,8 @@ def __init__(
"generic": GenericTTS,
"xai": XaiTTS,
"deepgram": DeepgramTTS,
"gradium": GradiumTTS,
"mistral": MistralTTS,
},
avatar={
"akool": AkoolAvatar,
Expand Down
4 changes: 4 additions & 0 deletions src/agora_agent/agentkit/vendors/namespaces.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,11 @@
ElevenLabsTTS,
FishAudioTTS,
GoogleTTS,
GradiumTTS,
HumeAITTS,
MicrosoftTTS,
MiniMaxTTS,
MistralTTS,
MurfTTS,
OpenAITTS,
GenericTTS,
Expand Down Expand Up @@ -101,6 +103,8 @@ class GlobalTTSVendors:
generic = GenericTTS
xai = XaiTTS
deepgram = DeepgramTTS
gradium = GradiumTTS
mistral = MistralTTS


class CNTTSVendors:
Expand Down
2 changes: 2 additions & 0 deletions src/agora_agent/agentkit/vendors/region.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@
"generic",
"xai",
"deepgram",
"gradium",
"mistral",
)

CN_LLM_VENDORS: typing.Tuple[str, ...] = (
Expand Down
52 changes: 52 additions & 0 deletions src/agora_agent/agentkit/vendors/tts.py
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,35 @@ def to_config(self) -> Dict[str, Any]:
return result


class GradiumTTS(BaseTTS):
model_config = ConfigDict(extra="forbid")

api_key: str = Field(..., description="Gradium API key")
url: Optional[str] = Field(default=None, description="WebSocket endpoint for streaming TTS output")
model_name: Optional[str] = Field(default=None, description="Gradium TTS model name")
voice_id: Optional[str] = Field(default=None, description="Gradium voice identifier")
sample_rate: Optional[int] = Field(default=None, description="Audio sample rate in Hz")
additional_params: Optional[Dict[str, Any]] = Field(default=None, description="Additional Gradium TTS parameters")
skip_patterns: Optional[List[int]] = Field(default=None)

def to_config(self) -> Dict[str, Any]:
params: Dict[str, Any] = dict(self.additional_params or {})
params["api_key"] = self.api_key
if self.url is not None:
params["url"] = self.url
if self.model_name is not None:
params["model_name"] = self.model_name
if self.voice_id is not None:
params["voice_id"] = self.voice_id
if self.sample_rate is not None:
params["sample_rate"] = self.sample_rate

result: Dict[str, Any] = {"vendor": "gradium", "params": params}
if self.skip_patterns is not None:
result["skip_patterns"] = self.skip_patterns
return result


class HumeAITTS(BaseTTS):
model_config = ConfigDict(extra="forbid")

Expand Down Expand Up @@ -455,6 +484,29 @@ def to_config(self) -> Dict[str, Any]:
return result


class MistralTTS(BaseTTS):
model_config = ConfigDict(extra="forbid")

api_key: str = Field(..., description="Mistral API key")
model: Optional[str] = Field(default=None, description="Mistral TTS model name")
voice: Optional[str] = Field(default=None, description="Mistral voice identifier")
additional_params: Optional[Dict[str, Any]] = Field(default=None, description="Additional Mistral TTS parameters")
skip_patterns: Optional[List[int]] = Field(default=None)

def to_config(self) -> Dict[str, Any]:
params: Dict[str, Any] = dict(self.additional_params or {})
params["api_key"] = self.api_key
if self.model is not None:
params["model"] = self.model
if self.voice is not None:
params["voice"] = self.voice

result: Dict[str, Any] = {"vendor": "mistral", "params": params}
if self.skip_patterns is not None:
result["skip_patterns"] = self.skip_patterns
return result


class SarvamTTS(BaseTTS):
model_config = ConfigDict(extra="forbid")

Expand Down
23 changes: 23 additions & 0 deletions tests/custom/test_regional_vendors.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,19 @@
Area,
DeepgramSTT,
GenericTTS,
GradiumTTS,
MiniMaxCNTTS,
MiniMaxTTS,
MistralTTS,
OpenAI,
SpatiusAvatar,
TencentSTT,
XaiSTT,
XaiTTS,
XaiGrok,
)
from agora_agent.agentkit.vendors.catalog import GLOBAL_VENDOR_NAMESPACE
from agora_agent.agentkit.vendors.namespaces import GlobalTTSVendors
from agora_agent.agentkit.vendors.region import (
CN_ASR_VENDORS,
CN_AVATAR_VENDORS,
Expand Down Expand Up @@ -194,6 +198,25 @@ def test_xai_asr_and_tts_are_classified_as_global_vendors() -> None:
assert global_agent.tts is not None and global_agent.tts["vendor"] == "xai"


@pytest.mark.parametrize(
("vendor_name", "tts"),
[
("gradium", GradiumTTS(api_key="gradium-key")),
("mistral", MistralTTS(api_key="mistral-key")),
],
)
def test_new_tts_are_classified_as_global_vendors(vendor_name: str, tts) -> None:
assert vendor_name not in CN_TTS_VENDORS
assert vendor_name in GLOBAL_TTS_VENDORS
assert GLOBAL_VENDOR_NAMESPACE.tts[vendor_name] is type(tts)
assert getattr(GlobalTTSVendors, vendor_name) is type(tts)

global_agent = Agent(client=_client(Area.US)).with_tts(tts)

assert global_agent.__class__.__name__ == "GlobalAgent"
assert global_agent.tts is not None and global_agent.tts["vendor"] == vendor_name


def test_xai_grok_remains_mllm_vendor() -> None:
agent = Agent(client=_client(Area.US)).with_mllm(XaiGrok(api_key="xai-key"))

Expand Down
6 changes: 6 additions & 0 deletions tests/custom/test_root_exports.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ def test_root_exports_match_agentkit_for_common_symbols() -> None:
"GlobalAgent",
"RegionalAgent",
"DeepgramSTT",
"GradiumTTS",
"MistralTTS",
"MiniMaxCNTTS",
"TencentSTT",
"OpenAI",
Expand Down Expand Up @@ -42,6 +44,8 @@ def test_unknown_root_export_raises_attribute_error() -> None:

def test_dir_includes_agentkit_vendor_exports() -> None:
assert "DeepgramSTT" in dir(agora_agent)
assert "GradiumTTS" in dir(agora_agent)
assert "MistralTTS" in dir(agora_agent)
assert "MiniMaxCNTTS" in dir(agora_agent)
assert "TencentSTT" in dir(agora_agent)
assert "CNAgent" in dir(agora_agent)
Expand All @@ -51,6 +55,8 @@ def test_dir_includes_agentkit_vendor_exports() -> None:
def test_all_includes_agentkit_vendor_exports() -> None:
assert "CredentialMode" in agora_agent.__all__
assert "DeepgramSTT" in agora_agent.__all__
assert "GradiumTTS" in agora_agent.__all__
assert "MistralTTS" in agora_agent.__all__
assert "MiniMaxCNTTS" in agora_agent.__all__
assert "TencentSTT" in agora_agent.__all__
assert "OpenAI" in agora_agent.__all__
Expand Down
Loading
Loading