Add ros2 nodl describe: rosgraph_msgs/Node -> NoDL document (#53)#88
Draft
luke-alloy wants to merge 18 commits into
Draft
Add ros2 nodl describe: rosgraph_msgs/Node -> NoDL document (#53)#88luke-alloy wants to merge 18 commits into
ros2 nodl describe: rosgraph_msgs/Node -> NoDL document (#53)#88luke-alloy wants to merge 18 commits into
Conversation
Stage one of the Observe -> Describe pipeline (ros-tooling#68): observe_node() fills a rosgraph_msgs/Node from the live graph - topics with actual QoS and RIHS type hash, services with honest *_UNKNOWN QoS (not observable externally), actions folded from their hidden _action/* constituents, and parameter descriptors/values via the target's parameter services with graceful degradation under a shared timeout ceiling. Tested in three layers: pure-builder unit tests (no executor), scenario graphs diffed against per-distro golden YAML files, and ROS-free rendering tests fed by those goldens. The goldens double as input fixtures for Describe (ros-tooling#53). Requires rosgraph_msgs >= 2.0.4 (Node.msg). Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Observes the target node via nodl_observe, prints the rosgraph_msgs/Node as YAML to stdout (or -o FILE with the format inferred from the .yaml/.yml/ .json extension), and publishes it latched (reliable, transient_local, keep_last(1)) on /nodl/observed_node. Publish-once semantics: delivery to currently-matched subscribers is confirmed via wait_for_all_acked, bounded by --timeout; the latched history dies with the publisher, so consumers must subscribe before the verb runs - locked in by the smoke test. --no-params skips the only part of observation that contacts the target. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
nodl_observe's tests are distro-aware: golden files live under test/expected/<ROS_DISTRO>/ (jazzy committed), and every test module skips cleanly on distros whose rosgraph_msgs predates 2.0.4 (no Node.msg), so the existing humble..rolling matrix stays green and flips tests on per distro as the rosgraph_msgs sync lands. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Signed-off-by: Luke Sy <sylukewicent@gmail.com>
__init__ now just re-exports the public surface (observe_node, NodeNotFoundError, latched_qos); the graph polling and endpoint collection move to _observe, matching the _-prefixed private-module pattern already used for _endpoints/_parameters/_qos. No behaviour change. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Adds rmw_cyclonedds_cpp alongside rmw_fastrtps_cpp in CI (distro x RMW matrix) and proves they observe QoS differently: cyclonedds propagates a remote endpoint's history policy and depth over discovery, fastrtps does not (history -> UNKNOWN, depth -> 0). Goldens are now resolved most-specific-first under expected/<distro>/<rmw>/, falling back to a shared expected/<distro>/ when RMWs agree, so identical sets are stored once. Commit a single YAML golden per case (the canonical, human-readable form); the JSON renderer is proven by an equivalence test instead of a duplicate JSON golden. The one RMW-divergent assertion is now keyed by RMW so each middleware's behaviour is locked in explicitly. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Node.msg ships on the jazzy line (>= 2.0.4) but is not yet in kilted (rosgraph_msgs 2.3.1 = Clock only) or current rolling, so version numbers are not comparable across distros. The package builds regardless; the import guard skips observation tests where Node.msg is absent. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
The graph messages landed in rolling first and were backported to jazzy (the only distro shipping them in a pullable image today). The earlier 'rolling lacks them' note was an artifact of probing the EOL Ubuntu 24.04 rolling image (frozen at 2.4.4); live rolling/lyrical run on Ubuntu 26.04 images that are not published yet. Kilted's backport release simply has not been cut (2.3.1 ships Clock only). Signed-off-by: Luke Sy <sylukewicent@gmail.com>
The RMW axis is now driven entirely by the CI matrix 'rmw:' list: the install step derives each apt package name (rmw_x_cpp -> ros-<distro>-rmw-x-cpp), the golden resolver is already (distro, RMW)-keyed, and the per-RMW history-over-discovery expectation is a documented _HISTORY_OVER_DISCOVERY map. Adding an RMW is 'drop in goldens' -- the harness needs no per-RMW setup (every scenario runs in one process / one session). Also closes the silent-skip trap: observation tests importorskip when rosgraph_msgs lacks Node.msg, which on a distro that *should* support observe reads as green-having-tested-nothing. A best-effort step pulls Node.msg from ros2-testing where it leads the main index (e.g. jazzy 2.0.4), and a matrix-gated assertion (requires_node_msg) fails the leg loudly if Node.msg is still missing where it is required. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Zenoh slots in with no harness change -- scenarios run in one process / one session, so it discovers without a router daemon. Behaviourally it matches cyclonedds (propagates history and depth over discovery; identical endpoint set and type hashes to the other RMWs). Adds its (jazzy) goldens and its _HISTORY_OVER_DISCOVERY entry. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Node.msg has now landed across jazzy (2.0.4), kilted (2.3.2), lyrical (2.4.5) and rolling (2.5.0) -- all via the best-effort ros2-testing install where it leads the main index -- so all four are flagged requires_node_msg and run the observation suite (humble still skips; no Node.msg yet). Empirically, every (distro, RMW) observes the same Node *except* two gaps: cyclonedds reports a KEEP_ALL queue's depth as 0 (every distro), and jazzy's older fastrtps drops history/depth entirely (kilted-onward fastrtps does propagate). Goldens are deduplicated to match: a single _base/ holds the full observation, with overrides only where a combination differs -- rmw_cyclonedds_cpp/s2 and jazzy/rmw_fastrtps_cpp/. The resolver searches <distro>/<rmw>/ -> <rmw>/ -> _base/. Nine files now cover 4 distros x 3 RMWs x 4 scenarios (verified green on noble and resolute). Drops the _HISTORY_OVER_DISCOVERY assertion map: history/depth is (distro, RMW)-specific (fastrtps differs by distro), and the per-combination golden already locks it exactly. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Node.msg has now reached every distro including humble (rosgraph_msgs 1.2.3 via ros2-testing), so the package gets exercised on pre-Iron rclpy for the first time -- and it did not import there. Two defensive fixes make it import-safe everywhere: map the BEST_AVAILABLE QoS enum only where rclpy defines it (added in Iron), and read TopicEndpointInfo.topic_type_hash via getattr (REP-2011, Iron+) so an absent hash is simply left unset like a service's. Full observation still needs Iron+ (type hashes, BEST_AVAILABLE, and an int32-safe infinite QoS deadline that humble's builtin_interfaces overflows on), so the observation/rendering tests and the describe smoke tests are capability-gated to Iron+ (BEST_AVAILABLE presence as the proxy). Humble's CI leg now builds, runs the pure-argument and serialization tests, and skips the rest -- green instead of crashing. Full pre-Iron support is tracked as a working-group follow-up. No change on Iron+ (jazzy 154/0). Signed-off-by: Luke Sy <sylukewicent@gmail.com>
Replace the ament_python nodl_observe with an ament_cmake C++ package: a reusable observe_node(...) library plus an `observe` executable that latch-publishes the observed rosgraph_msgs/Node on /nodl/observed_node. The Python implementation is kept locally as an untracked reference. Addresses the WG review on this PR (points ros-tooling#2-ros-tooling#5 of plan_observe_cpp.md). - Port the pure builders 1:1 (QoS enum mapping, topic/service endpoints with REP-2011 type hash, action folding, parameter pairing, FQN split), with gtest unit tests mirroring the Python tests. - Actions use the rcl_action C API directly (no rclcpp_action wrapper). - Parameters via AsyncParametersClient driven by a short-lived executor, with graceful degradation on an unresponsive target (covered by a test). - Canonicalise infinite/overflow QoS durations to {INT32_MAX, 0} uniformly on every distro -- CDR-valid for MCAP and fixes Humble's int32 overflow. - Humble (pre-Iron) is a supported, tested runtime target: the type hash and BEST_AVAILABLE QoS enum are compiled out via ROS2_<DISTRO>; the message stays structurally identical, differences live only in unfilled fields. - Replace the ~2k-line YAML goldens with MCAP fixtures (one per (distro, RMW), most-specific-first resolver: <distro>_<rmw> -> <rmw> -> base) plus a human-readable print/diff helper. Verified field-for-field parity with the rclpy output (only the duration sentinel differs). - Rewire `ros2 nodl describe` to shell out to the `observe` binary and render via rosidl_runtime_py. - CI: one job per distro, build once, re-run the RMW-sensitive integration test over fastrtps/cyclonedds/zenoh; drop requires_node_msg. Validated in Docker on humble, jazzy, kilted, and lyrical (build + gtest unit tests + integration across each distro's RMWs). Signed-off-by: Luke Sy <sylukewicent@gmail.com>
- ARCHITECTURE.md: layered data-flow diagram, module table, and the observe_node step-by-step for contributors. - mcap_fixtures.py: add node_to_json + 'print -f yaml|json' so the fixture viewer matches the verb's -o output (was YAML only). Signed-off-by: Luke Sy <sylukewicent@gmail.com>
…_msgs on humble - rosidl_runtime_py was transitive via the old Python nodl_observe; the C++ rewrite dropped it, but the integration test (mcap_fixtures.py) and the describe verb still use it -> declare it (+ rclpy, ament_index_python for the verb, which the C++ lib no longer provides transitively). - humble's rosgraph_msgs ships Node.msg only via ros2-testing; the bridge used --only-upgrade, a no-op when the package isn't pre-installed (the rostooling image), so rosdep then pulled the main version without Node.msg and the C++ build failed. Plain install + apt-mark hold instead. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
- Bridge silently no-op'd on humble: the keyring glob ros2*archive-keyring missed the actual ros-archive-keyring.gpg, so '[ -n key ] || exit 0' bailed and rosdep then installed main rosgraph_msgs (no Node.msg). Broaden the glob and fall back to [trusted=yes] instead of skipping. - Per-RMW integration steps passed the test but 'colcon test-result --all' defaulted --test-result-base to 'build'; the action-ros-ci workspace is 'ros_ws/build'. Point it there. Signed-off-by: Luke Sy <sylukewicent@gmail.com>
The CI image (rostooling/setup-ros-docker) installs ros2cli but not the ros2run package, so 'ros2 run rmw_zenoh_cpp rmw_zenohd' errors with "invalid choice: 'run'"; the zenoh router never starts, cross-process discovery fails, and every observation times out (this is why jazzy/kilted/ lyrical/rolling failed their zenoh leg while humble -- which has no zenoh -- passed). Locate and exec the rmw_zenohd binary directly. Validated end-to-end in the actual rostooling images: jazzy (fastrtps/cyclonedds/zenoh all 14 passed) and humble (fastrtps/cyclonedds). Signed-off-by: Luke Sy <sylukewicent@gmail.com>
…oling#53) Implements the interpret half of the Observe/Describe split: a pure, duck-typed transform from a captured rosgraph_msgs/Node into a validated NoDL document. observe records everything unfiltered; describe interprets. - describe/ package: _qos, _endpoints, _parameters, _filter, _convert (pure, no rclpy/rosgraph_msgs imports) + _source (live observe binary or --from yaml|mcap) + a thin verb. - Best-effort draft contract: reliably-recoverable fields are filled; genuinely-unknowable fields become Gaps. The verb warns the gap list and exits 0; --strict fails. Flags: --from/--no-params/--keep-hidden/--strict/ --raw/--topic/-o. - Mapping: QoS int->enum (UNKNOWN policy, INT32_MAX duration sentinel, conditional depth), action-type recovery (strip _SendGoal/_GetResult), the infrastructure denylist, and parameter type/value/range. - nodl_schema: export rostype_pattern() so the endpoint type regex has a single source of truth instead of a hardcoded copy in describe. - Tests: stub_msgs duck-typed builders (pure tests run with no ROS), real-message tests, a real-vs-stub fidelity guard, RMW/ParameterType constant-drift guards, and 20 golden fixtures across 5 distro/RMW pairs. - tools/run_describe_tests.sh: Docker harness that runs the real suite against the rosgraph_msgs fork (no ROS install needed on the host). - Docs: nodl/doc/describe.md page + the describe lane in ARCHITECTURE.md. Stacked on lsy3/observe (it drives the C++ observe binary); land after observe merges, then rebase onto main for a describe-only diff.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #53 — the interpret half of Observe/Describe: a pure transform from a captured
rosgraph_msgs/Nodeinto a validated NoDL document, behindros2 nodl describe(live, or--from yaml|mcap).What it does
describe/: pure, duck-typed_qos/_endpoints/_parameters/_filter/_convert+_source(live binary or from-file) + a thin verb._SendGoalstrip), filters framework infrastructure; output validates against the schema.--strictfails on any gap. Flags:--from/--no-params/--keep-hidden/--strict/--raw/-o.Testing
tools/run_describe_tests.shruns the real suite in the ROS image: 272 passed / 7 skipped (skips = live-observe smoke tests that need the builtobservebinary → CI).stub_msgs; 20 golden fixtures across 5 distro/RMW pairs; a real-vs-stub fidelity guard + RMW/ParameterType constant-drift guards.Mapping reference:
nodl/doc/describe.md.