Skip to content

feat: add funasr_asr_python local FunASR (SenseVoice) ASR extension#2191

Open
LauraGPT wants to merge 4 commits into
TEN-framework:mainfrom
LauraGPT:add-funasr-asr-extension
Open

feat: add funasr_asr_python local FunASR (SenseVoice) ASR extension#2191
LauraGPT wants to merge 4 commits into
TEN-framework:mainfrom
LauraGPT:add-funasr-asr-extension

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Jun 22, 2026

Copy link
Copy Markdown

Summary

Adds a local funasr_asr_python ASR extension powered by FunASR. It follows TEN's AsyncASRBaseExtension contract, defaults to iic/SenseVoiceSmall, runs on CPU or CUDA without an API key, and accepts another FunASR-compatible model through configuration.

The extension converts 16 kHz signed 16-bit PCM into normalized float audio, runs AutoModel.generate(...) outside the event loop, post-processes SenseVoice output, and emits TEN ASRResult messages.

Reliability

  • Reports connected/disconnected lifecycle state, including model-load failures.
  • Preserves PCM samples beyond the 30-second processing window instead of dropping the remainder.
  • Emits cumulative start offsets across processed chunks.
  • Extracts SenseVoice's detected language tag when language is configured as auto and normalizes it for TEN results.
  • Releases locked audio-frame buffers in finally and always reports finalize completion.
  • Adds package metadata, a repository-compatible test runner, and focused client/extension regression coverage.

Runtime dependencies are declared consistently in pyproject.toml: funasr>=1.1.0, numpy>=1.24.0, pydantic>=2.0.0, and typing-extensions>=4.5.0.

Validation

Validated at exact head 7a468a0795729bfc702c5d9c0b86a2353508f633, rebased onto current TEN main 0c914f7ba3437f6eb28c0a00a103f7b458e25b6c:

  • repository-native tests/bin/start: 13 passed
  • repository Pylint configuration: 10.00/10
  • Black check with the repository's 80-character setting: passed on all 11 Python files
  • tman check manifest-json and tman check property-json: both conform to schema
  • all 11 Python files parse with the Python 3.10 grammar and compile successfully
  • git diff --check: passed

A real CPU smoke used this extension's FunASRClient, FunASR 1.3.14, the official 5.592-second Mandarin sample, and iic/SenseVoiceSmall. The callback returned one final result with detected language zh and transcript:

开饭时间早上9点至下午5点。

@LauraGPT LauraGPT changed the title Add funasr_asr_python: local FunASR (SenseVoice) ASR extension feat: add funasr_asr_python local FunASR (SenseVoice) ASR extension Jun 22, 2026
@LauraGPT
LauraGPT force-pushed the add-funasr-asr-extension branch from ede91cf to 43cbedc Compare June 22, 2026 21:40
@LauraGPT

Copy link
Copy Markdown
Author

CI note for reviewers: the only failing check is not a code/test failure.

I inspected claude-review (Claude Code Review) run 27985905295, job 82827178776. The action exits before reviewing the diff because this PR is from my fork and the actor only has read permission on TEN-framework/ten-framework:

Checking permissions for actor: LauraGPT
Permission level retrieved: read
Actor has insufficient permissions: read
Action failed with error: Actor does not have write permissions to the repository

So this check is blocked by the review action's permission policy for external contributors, not by the FunASR extension code. The PR remains MERGEABLE; it likely needs a maintainer-triggered review/run or a maintainer-side review path to clear that status.

@LauraGPT

LauraGPT commented Jul 7, 2026

Copy link
Copy Markdown
Author

Fresh FunASR/SenseVoice-side validation recheck (2026-07-07 UTC):

  • PR is open, non-draft, and mergeable; mergeable_state=blocked is currently the review/check gate. Current head is 43cbedce.
  • Review-thread audit shows 0 current unresolved review threads.
  • The only GitHub check failure I see is claude-review, which exits before reviewing because the fork actor has only read permission. I do not see a code/test failure attached to this head.
  • Local focused validation in a fresh checkout:
    • git diff --check origin/main...HEAD passed after fetching the merge base.
    • manifest.json and property.json parse as JSON.
    • python3 -m py_compile passed for the new funasr_asr_python Python files and tests/test_config.py.
    • The focused config tests passed with minimal local stubs for the TEN runtime packages that are normally materialized by the TEN package environment: PYTHONPATH=<stub>:ai_agents/agents/ten_packages/extension python3 -m pytest ai_agents/agents/ten_packages/extension/funasr_asr_python/tests/test_config.py -q -> 3 passed.

Caveat: my bare checkout does not include installed/materialized ten_runtime / ten_ai_base, so I did not run a full TEN runtime integration test locally. From the FunASR extension side, this looks ready for human maintainer review.

@LauraGPT

LauraGPT commented Jul 7, 2026

Copy link
Copy Markdown
Author

I checked the only failing check on this PR (claude-review). It is not failing on the FunASR extension code or tests.

The job exits before doing a review because the workflow sees the PR actor with read-only repository permission:

Checking permissions for actor: LauraGPT
Permission level retrieved: read
Actor has insufficient permissions: read
Action failed with error: Actor does not have write permissions to the repository

So the current red check is a Claude review workflow permission issue on pull_request_target, not a regression in the funasr_asr_python extension. A maintainer can either re-run/trigger that review workflow with trusted permissions or treat this check as non-actionable for the PR diff.

@LauraGPT
LauraGPT force-pushed the add-funasr-asr-extension branch from 43cbedc to 7a468a0 Compare July 17, 2026 00:23
@LauraGPT

Copy link
Copy Markdown
Author

Synced this PR to current main; exact head f1c61703027345d9b53972c2e207235af89793b0 is 0 commits behind and mergeable. The upstream merge changed none of the 17 FunASR extension files.

Fresh verification passes the repository-native extension suite (13/13), Pylint at 10.00/10, Black on all 11 Python files, both tman manifest/property schema checks, Python 3.10 grammar and compilation, and git diff --check. There are no unresolved review threads.

The only red check still exits before reading the diff: the review action sees the fork actor with read permission and requires write. @halajohn @plutoless, could one of you review the new local FunASR/SenseVoice ASR extension when convenient?

@LauraGPT

Copy link
Copy Markdown
Author

Synced this FunASR/SenseVoice ASR extension PR with the latest TEN-framework/main after upstream advanced by 2 commits.

Head: b53ed14702582c3d9349d966b42e0de9b4eb1401

Fresh validation on ind-gpu8:

PYTHONPATH=".:.ten/app/ten_packages/system/ten_runtime_python/interface:.ten/app/ten_packages/system/ten_ai_base/interface" \
  /cpfs_speech/user/zhifu.gzf/.cache/funasr-ops/ten-2191-venv-20260717/bin/python -m pytest -q tests

PYTHONPATH="..." \
  /cpfs_speech/user/zhifu.gzf/.cache/funasr-ops/ten-2191-venv-20260717/bin/python -m compileall -q \
  addon.py config.py const.py extension.py funasr_client.py reconnect_manager.py tests

./agents/scripts/pylint.sh funasr_asr_python
black --check --line-length 80 agents/ten_packages/extension/funasr_asr_python/*.py agents/ten_packages/extension/funasr_asr_python/tests
tman -y install --standalone
git diff --check

Results:

  • funasr_asr_python tests: 13/13 passed
  • Python compile passed
  • manifest.json and property.json parse cleanly
  • Pylint: 10.00/10
  • Black check: 10 files unchanged
  • tman -y install --standalone passed with TEN Framework 0.11.68
  • diff whitespace check passed

The branch is now 0 commits behind the target branch. GitHub reports it mergeable; the remaining gate is maintainer review. The claude-review check restarted after the push and was still in progress when I checked.

@LauraGPT

Copy link
Copy Markdown
Author

Follow-up on the restarted claude-review check for head b53ed1470258: it failed before reviewing the diff because the workflow sees the PR actor as read-only.

Relevant log lines from run https://github.com/TEN-framework/ten-framework/actions/runs/29664747134/job/88133177090:

Checking permissions for actor: LauraGPT
Permission level retrieved: read
Actor has insufficient permissions: read
Action failed with error: Actor does not have write permissions to the repository

So the remaining red check is a repository permission gate for the Claude review action, not a FunASR extension test failure. The local extension validation above remains current for this exact head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant