Skip to content

Add comprehensive FlatBuffers documentation page #1827

@oberstet

Description

@oberstet

Summary

Create a dedicated documentation page explaining autobahn-python's FlatBuffers features, bundled flatc compiler, and
integration with the WAMP ecosystem.

Background

autobahn-python now includes significant FlatBuffers functionality that is not well-documented:

  1. Bundled flatc compiler in binary wheels (since v25.12.x)
  2. Vendored FlatBuffers runtime for serialization
  3. WAMP FlatBuffers schemas for efficient binary serialization
  4. Cross-project version synchronization with zlmdb

Currently, FlatBuffers information is scattered across:

Proposed Documentation Content

1. Overview Section

  • What is FlatBuffers and why autobahn-python uses it
  • Benefits: zero-copy access, efficient serialization, cross-language compatibility
  • Use cases: WAMP binary serialization, high-performance messaging

2. Bundled flatc Compiler

  • Feature: Wheels include a pre-compiled flatc executable; source installs compile it from deps/flatbuffers
  • Usage: After pip install autobahn, run flatc --version
  • Platform support: Linux (x86_64, aarch64), macOS (arm64), Windows (x86_64)
  • ISA requirements: Need GLIBC 2.28+ (e.g., Ubuntu 20.04+, Debian 11+, RHEL 8+)
    • Linux x86_64: manylinux_2_28 (GLIBC 2.28+)
    • Linux aarch64: manylinux_2_28 (GLIBC 2.28+)
  • Benefits: No need to install system-wide flatc, version matches vendored runtime
  • Source builds: Require cmake to compile flatc from deps/flatbuffers during install

3. Vendored FlatBuffers Runtime

  • Version tracking: autobahn.flatbuffers.version() returns exact version tuple
  • Location: autobahn/flatbuffers/ (vendored from google/flatbuffers)
  • Includes reflection schema (reflection.fbs, reflection.bfbs)

4. WAMP FlatBuffers Schemas

  • Schema files location: autobahn/wamp/flatbuffers/*.fbs
  • Available schemas: wamp.fbs, session.fbs, auth.fbs, pubsub.fbs, rpc.fbs, roles.fbs, types.fbs
  • Binary schemas (.bfbs) generated at wheel build time
  • Link to existing schema reference
    documentation

5. Cross-Project Version Synchronization

  • autobahn-python and zlmdb share the same FlatBuffers version
  • Verification function: autobahn.check_zlmdb_flatbuffers_version_in_sync()
  • Why this matters: cfxdb and Crossbar.io depend on both libraries
  • Example usage:
    import autobahn
    version = autobahn.check_zlmdb_flatbuffers_version_in_sync()
    print(f"FlatBuffers version: {version}")  # e.g., (25, 9, 23, None, None)
  1. Development & Build Information
  • FlatBuffers source: deps/flatbuffers git submodule (tracks google/flatbuffers)
  • Build process: hatch_build.py compiles flatc during both wheel and source builds
  • Updating vendored FlatBuffers: just bump-flatbuffers
  • Testing bundled flatc: just test-bundled-flatc
  1. Troubleshooting
  • "flatc not found after source install": Ensure cmake is installed before pip install
  • Version mismatch with zlmdb: Update both packages to latest versions
  • GLIBC errors on old Linux: Need GLIBC 2.28+ (e.g., Ubuntu 20.04+, Debian 11+, RHEL 8+)

Documentation Location

Suggested location: docs/wamp/flatbuffers.rst (or docs/flatbuffers.rst if scope extends beyond WAMP)

Should be linked from:

  • Main documentation index
  • WAMP serialization documentation
  • Installation guide (mention bundled flatc)

Related

Checklist

  • I have searched existing issues to avoid duplicates
  • I have described the problem clearly
  • I have provided use cases
  • I have considered alternatives
  • I have assessed impact and breaking changes

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions