Skip to content

feat: Add ExApp / dedicated-device classification backend#1517

Open
szaimen wants to merge 2 commits into
mainfrom
feat/exapp-classification-backend
Open

feat: Add ExApp / dedicated-device classification backend#1517
szaimen wants to merge 2 commits into
mainfrom
feat/exapp-classification-backend

Conversation

@szaimen

@szaimen szaimen commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Fixes #73

Summary

Adds an option to offload Recognize's machine-learning classification to an External App (ExApp) container instead of running Node.js locally — so inference can run on a different, more powerful machine, optionally with a GPU. This is the long-requested "dedicated devices / Tensorflow-serving / ExApp backend" feature.

The design follows marcelklehr's plan in the issue: keep the classifiers running the exact same classifier_<model>.js scripts, but add a setting that routes their execution to the Recognize ExApp via AppAPI rather than to a local process. Results are compatible because the same scripts are reused.

The ExApp is installable from the Nextcloud App Store / External Apps page (the default, recommended deployment), exactly like the other AI ExApps (llm2, context_chat_backend).

How it works

┌───────────────────────┐      AppAPI (authenticated HTTP)        ┌──────────────────────────┐
│ Nextcloud + recognize  │ ── POST /classify (model + image bytes) ─▶│ recognize_exapp container │
│ (classifier.backend =  │                                          │  Node.js + Tensorflow.js  │
│        "exapp")        │ ◀── newline-delimited JSON results ────── │  classifier_<model>.js    │
└───────────────────────┘                                          └──────────────────────────┘

The PHP app prepares the same downscaled preview images it would feed to the local process, uploads them to the ExApp as multipart, and the container runs the unmodified classifier scripts and streams JSON results back. Image bytes are sent (not paths), so the ExApp can run on a remote machine that doesn't share the Nextcloud filesystem.

Changes

Recognize PHP app

  • lib/Classifiers/Classifier.phpclassifyFiles() keeps file preparation + result parsing, but dispatches execution to runLocally() (the original Symfony\Process path, behavior unchanged) or runExApp() (uploads files to the ExApp via AppAPI) based on the new classifier.backend setting.
  • lib/Service/ExAppService.php (new) — defensive wrapper around OCA\AppAPI\PublicFunctions. AppAPI is an optional dependency: every method degrades gracefully when it isn't installed, so the local backend keeps working.
  • lib/Service/SettingsService.php — two new lazy settings: classifier.backend (local/exapp, default local) and exapp.id (default recognize_exapp).
  • lib/Controller/AdminController.php + appinfo/routes.php — new GET /admin/exapp endpoint that tests ExApp reachability (mirrors the existing nodejs()/ffmpeg() checks).
  • src/components/ViewAdmin.vue — new "Classification backend" admin section: local/ExApp radio selector, App-ID field, live status feedback, App Store install links.

ExApp container (exapp/)

  • server.js — minimal HTTP server exposing /heartbeat, /init, /enabled, /classify; receives uploads, spawns the existing classifier scripts, streams results. AppAPI shared-secret auth.
  • Dockerfile — self-contained, multi-stage; BUILD_TYPE=cpu (default) / gpu (CUDA base). Fetches the matching Recognize sources at build time.
  • appinfo/info.xml — App Store ExApp manifest (registry image, env vars, AppAPI requirement).
  • Makefile, krankerl.toml, .nextcloudignore — packaging / dev registration.
  • README.md — architecture, install, build, release docs.
  • test/server.test.js — standalone functional test for the server (no TensorFlow needed).

CI (.github/workflows/)

  • exapp-publish-docker.yml — on v* tag, builds & pushes the image to ghcr.io/nextcloud/recognize_exapp.
  • exapp-appstore-build-publish.yml — on release, packages, signs, and uploads the manifest to the App Store.

Testing done locally

  • ExApp server: 13/13 functional tests pass (auth, heartbeat, multipart parse, file ordering incl. out-of-order reassembly, model validation, error paths, lifecycle hooks) — node exapp/test/server.test.js.
  • PHP: php -l clean on all 5 changed files; php-cs-fixer clean against the repo's pinned nextcloud/coding-standard v1.1.0.
  • Docker image: builds successfully (CPU); container runs and serves /heartbeat200 {"status":"ok"}.
  • Manifests/workflows: XML well-formed (xmllint), workflow YAML parses, Dockerfile passes hadolint (advisory warnings only).

Not covered / follow-ups

  • Psalm and the Vue build/eslint weren't run locally (no full composer install / node_modules) — they run in CI.
  • Only the CPU image was built (no local NVIDIA runtime for the GPU variant).
  • The manifest pins version/image-tag 12.1.0; the matching v12.1.0 Recognize tag must exist at release time (CI keeps RECOGNIZE_REF in sync via the image-tag).
  • The ExApp ships in-tree under exapp/; it may eventually warrant its own nextcloud-releases repo, but keeping it here makes the whole feature reviewable together.

🤖 Generated with Claude Code

Adds an option to offload Recognize's machine-learning classification to an
External App (ExApp) container instead of running Node.js locally, so inference
can run on a different, more powerful machine, optionally with a GPU.

The classifiers keep running the same classifier_<model>.js scripts; a new
`classifier.backend` setting routes their execution to the Recognize ExApp via
AppAPI rather than to a local process, keeping results compatible.

- Classifier::classifyFiles dispatches to runLocally (unchanged) or runExApp
- ExAppService wraps OCA\AppAPI\PublicFunctions (optional dependency, degrades
  gracefully when AppAPI is not installed)
- New settings classifier.backend + exapp.id, admin UI backend selector, and a
  /admin/exapp reachability test endpoint
- exapp/ ships the App Store-installable ExApp: self-contained Dockerfile
  (CPU/GPU), HTTP server reusing the classifier scripts, manifest, packaging
  and a functional test
- CI workflows to build/push the image and publish the manifest to the App Store

Fixes #73

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
@szaimen szaimen force-pushed the feat/exapp-classification-backend branch from 4e09120 to dfef072 Compare June 9, 2026 15:15
@szaimen szaimen added enhancement New feature or request 3. to review labels Jun 9, 2026
- Classifier: generator return type void -> null (bare return yields null)
- AdminController: avoid RiskyTruthyFalsyComparison via explicit (bool) cast
- ExAppService: mark final; suppress Mixed* from the optional AppAPI
  PublicFunctions calls (UndefinedClass dependency) and cast the error
  message operand to string

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
@szaimen szaimen force-pushed the feat/exapp-classification-backend branch from b2b572e to da6dba8 Compare June 9, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set up dedicated devices / Tensorflow serving / Use an ExApp as backend

1 participant