Skip to content

feat: Add external chat gateways for Telegram, WhatsApp, Slack, and Discord#1444

Merged
MODSetter merged 65 commits into
MODSetter:devfrom
AnishSarkar22:feat/whatsapp-gateway-integration
Jun 1, 2026
Merged

feat: Add external chat gateways for Telegram, WhatsApp, Slack, and Discord#1444
MODSetter merged 65 commits into
MODSetter:devfrom
AnishSarkar22:feat/whatsapp-gateway-integration

Conversation

@AnishSarkar22
Copy link
Copy Markdown
Contributor

@AnishSarkar22 AnishSarkar22 commented May 29, 2026

Description

  • Added the external chat gateway persistence layer for platform accounts, conversation bindings, inbound inbox events, and canonical chat thread/message metadata.
  • Implemented the Telegram gateway path, including pairing, webhook intake, long-poll/BYO intake modes, command handling, stream translation, inbox processing, worker maintenance, and settings UI.
  • Refactored gateway internals into provider-neutral abstractions for commands, formatting, platform bundles, inbox processing, request context, metrics, and external chat routing.
  • Added WhatsApp support through both WhatsApp Cloud API and self-hosted Baileys bridge flows, including Cloud webhooks, bridge service, pairing endpoints, Docker/runtime wiring, and messaging channel controls.
  • Added Slack gateway support with workspace-scoped accounts, app mention parsing, threaded replies, OAuth/webhook flow, Slack command handling, maintenance integration, and settings UI.
  • Added Discord gateway support with guild-scoped accounts, bot mention intake supervisor, REST send/update support, install flow, channel replies, maintenance integration, and settings UI.
  • Added gateway observability and operational safeguards, including queue/inbox metrics, webhook parse metrics, rate-limit handling, HITL fallbacks, event stream cleanup, and maintenance health checks.
  • Added docs for all 4 gateways.

Motivation and Context

FIX #

Screenshots

API Changes

  • This PR includes API changes

Change Type

  • Bug fix
  • New feature
  • Performance improvement
  • Refactoring
  • Documentation
  • Dependency/Build system
  • Breaking change
  • Other (specify):

Testing Performed

  • Tested locally
  • Manual/QA verification

Checklist

  • Follows project coding standards and conventions
  • Documentation updated as needed
  • Dependencies updated as needed
  • No lint/build errors or new warnings
  • All relevant tests are passing

High-level PR Summary

This PR integrates external messaging channels (Telegram and WhatsApp) into SurfSense. The implementation creates a gateway infrastructure that routes messages from external platforms through the same canonical chat agent used by the web UI. It adds database tables for managing external chat accounts, bindings, and inbound event queues, platform-specific adapters for Telegram and WhatsApp (both Meta Cloud API and self-hosted Baileys bridge), and an in-process FastAPI inbox worker that processes external messages and invokes the chat agent. The architecture supports webhook-based intake for production (Telegram, WhatsApp Cloud) and long-polling fallback for self-hosted deployments. The PR also includes a WhatsApp bridge Node.js service using Baileys for self-hosted WhatsApp integration, web UI components for pairing/managing channels, comprehensive observability metrics, and unit tests for core gateway components.

⏱️ Estimated Review Time: 1-3 hours

💡 Review Order Suggestion
Order File Path
1 surfsense_backend/app/config/__init__.py
2 surfsense_backend/app/db.py
3 surfsense_backend/alembic/versions/144_add_gateway_tables.py
4 surfsense_backend/app/gateway/base/adapter.py
5 surfsense_backend/app/gateway/base/translator.py
6 surfsense_backend/app/gateway/base/commands.py
7 surfsense_backend/app/gateway/telegram/adapter.py
8 surfsense_backend/app/gateway/telegram/translator.py
9 surfsense_backend/app/gateway/telegram/commands.py
10 surfsense_backend/app/gateway/whatsapp/adapter_cloud.py
11 surfsense_backend/app/gateway/whatsapp/adapter_baileys.py
12 surfsense_backend/app/gateway/whatsapp/translator.py
13 surfsense_backend/app/gateway/whatsapp/translator_baileys.py
14 surfsense_backend/app/gateway/whatsapp/commands.py
15 surfsense_backend/app/gateway/registry.py
16 surfsense_backend/app/gateway/inbox.py
17 surfsense_backend/app/gateway/inbox_processor.py
18 surfsense_backend/app/gateway/inbox_worker.py
19 surfsense_backend/app/gateway/agent_invoke.py
20 surfsense_backend/app/gateway/pairing.py
21 surfsense_backend/app/gateway/bindings.py
22 surfsense_backend/app/gateway/accounts.py
23 surfsense_backend/app/gateway/runner.py
24 surfsense_backend/app/gateway/byo_long_poll.py
25 surfsense_backend/app/routes/gateway_webhook_routes.py
26 surfsense_backend/app/routes/gateway_whatsapp_webhook_routes.py
27 surfsense_backend/app/routes/gateway_whatsapp_baileys_routes.py
28 surfsense_backend/app/routes/__init__.py
29 surfsense_backend/app/tasks/celery_tasks/gateway_tasks.py
30 surfsense_backend/app/celery_app.py
31 surfsense_backend/app/app.py
32 surfsense_backend/app/observability/metrics.py
33 surfsense_backend/scripts/whatsapp-bridge/bridge.js
34 surfsense_backend/scripts/whatsapp-bridge/Dockerfile
35 surfsense_backend/scripts/whatsapp-bridge/package.json
36 docker/docker-compose.yml
37 surfsense_backend/.env.example
38 surfsense_backend/pyproject.toml
39 surfsense_backend/scripts/docker/entrypoint.sh
40 surfsense_backend/scripts/register_webhook.py
41 surfsense_web/app/dashboard/[search_space_id]/user-settings/components/MessagingChannelsContent.tsx
42 surfsense_web/app/dashboard/[search_space_id]/user-settings/layout-shell.tsx
43 surfsense_web/app/dashboard/[search_space_id]/user-settings/messaging-channels/page.tsx
44 surfsense_web/.env.example
45 surfsense_web/zero/schema/chat.ts
46 surfsense_web/content/docs/manual-installation.mdx
47 surfsense_backend/tests/unit/gateway/test_formatting.py
48 surfsense_backend/tests/unit/gateway/test_hitl_filter.py
49 surfsense_backend/tests/unit/gateway/test_pairing.py
50 surfsense_backend/tests/unit/gateway/test_webhook_routes.py
51 surfsense_backend/tests/unit/gateway/test_inbox_worker.py
52 surfsense_backend/tests/unit/gateway/test_byo_long_poll_lifespan.py
53 surfsense_backend/tests/unit/gateway/test_enqueue_received_sweep.py
54 surfsense_backend/tests/unit/gateway/test_process_inbound_event_task.py

Need help? Join our Discord

@AnishSarkar22 AnishSarkar22 changed the title feat: messaging gateway integration feat: Add external chat gateways for Telegram, WhatsApp, Slack, and Discord Jun 1, 2026
@AnishSarkar22 AnishSarkar22 marked this pull request as ready for review June 1, 2026 19:00
@AnishSarkar22 AnishSarkar22 marked this pull request as draft June 1, 2026 19:00
@AnishSarkar22 AnishSarkar22 marked this pull request as ready for review June 1, 2026 19:49
@MODSetter MODSetter merged commit a80a9cb into MODSetter:dev Jun 1, 2026
3 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants