Robonix — The Embodied AI Operating System

An EAIOS that turns the robot body into a uniform, capability-first runtime — so models, sensors, and actuators plug in once and reuse everywhere.

robonix-v0.1-demo-github.mp4

Status

Warning

Robonix is in early development. APIs, IDL layouts, and internal designs may change without notice. No API stability until a versioned release.

What it is

Robonix is the operating system between a robot's hardware and an embodied LLM/VLM/VLA/WAM brain. It standardises how device drivers, runtime services, user skills, and the planner discover and talk to each other; it owns identity, configuration, time, transport, logging, health, body model, scene model, execution, and safety as named, replaceable components.

Component	Responsibility
atlas	Capability registry and discovery: the catalog of every registered capability and its contract
chronos	Unified clock and cross-sensor timestamp alignment (PTP / IEEE-1588)
executor	RTDL plan execution and capability dispatch (sequence / parallel / do)
keystone	User identity, persistent configuration, and access policy
liaison	Human–machine interaction gateway: chat, voice, and TUI
nexus	Communication libraries for gRPC / MCP / ROS 2 (not a standalone process)
pilot	VLM-driven planning and decision loop; emits RTDL plans for the executor
scene	Live environment estimate: object registry, semantic relations, and occupancy grid
scribe	Structured, persistent, replayable system journal for audit
sentinel	Rule-based safety gate checked before each capability dispatch
soma	Robot self-description (body model): device topology and primitive abstraction
vitals	Robot power and component-health monitoring

On top of system, three open categories — provided as contracts (61 standard interfaces in capabilities/) and reference implementations alongside the system:

• primitive — one device per package (camera, lidar, chassis, arm). Lives in deployment repos and per-example folders (e.g. examples/webots/primitives/).
• service — runtime functionality (mapping, navigation, semantic map, memory, speech, voiceprint). Default reference implementations ship in services/; each can be swapped out by a deployment.
• skill — user-defined reusable execution flows (grasp, place, explore, fold-clothes …). Lives wherever the deploy/integrator wants.

Supported platforms

Arch	OS / Distribution	Status
x86_64	Ubuntu 22.04	✅ Tested
x86_64	Debian 13	✅ Tested
arm64	NVIDIA Jetson — JetPack 6.2 (L4T 36.4.3, Ubuntu 22.04)	✅ Tested
x86_64 / arm64	Ubuntu 24.04 and newer	🚧 Planned
x86_64 / arm64	Arch Linux	🚧 Planned
arm64	macOS	🚧 Planned

"Tested" means the full Robonix pipeline runs end-to-end on that platform — in simulation or on a real robot: voice & interaction, task execution, body movement, scene & mapping (semantic map + spatial map), navigation, and skill execution. Other Linux distributions will likely work but are not regularly verified.

Relationship with ROS 2. Robonix itself does not depend on ROS 2 — it is one of the transports nexus offers, not a requirement of the system. If a capability provider needs the ROS 2 communication libraries and the host OS has no ROS 2 support, run that provider in a Docker container. Within a single Robonix deployment, all ROS 2-based capability providers must use the same ROS 2 distribution (Foxy / Humble / Jazzy); Humble is recommended.

Quickstart

git clone --recursive https://github.com/syswonder/robonix
cd robonix
make install   # builds the Cargo workspace and installs
               # rbnx + robonix-{atlas,pilot,executor,liaison,codegen}
               # to ~/.cargo/bin, then registers this clone via `rbnx setup`

The Webots Tiago example (examples/webots/) is the standard end-to-end demo. Two terminals — the simulator and Robonix itself.

# (1) — simulation environment (Webots GUI; not a Robonix package — just docker compose)
bash examples/webots/sim/start.sh

# (2) — Robonix: system services + Tiago primitives + Nav2 + scene
export VLM_BASE_URL=https://api.openai.com/v1   # any OpenAI-compatible endpoint
export VLM_API_KEY=sk-...
export VLM_MODEL=gpt-5.5

# scene is the only component that consumes ROS topics, and its ROS distro
# is a build-time choice (default humble). To build it against another
# distro, set this BEFORE the first `rbnx build` — supported: humble, iron,
# jazzy, rolling. (Leave unset for the default.)
# export ROBONIX_SCENE_ROS_DISTRO=humble

cd examples/webots
rbnx build       # first run pulls model weights + docker images, may take a while
rbnx boot

Once rbnx boot reports the stack is up:

# (3)
rbnx caps          # list registered capabilities + interfaces
rbnx chat          # interactive TUI chat with the pilot

Keeping upstream packages fresh: some providers in the manifest are cloned from upstream git repos (e.g. mapping, nav2, explore declared with url:). They are cloned once and reused, so they don't advance on their own. rbnx boot and rbnx build print a notice when a local clone is behind its remote; sync to the latest upstream commit with rbnx update:

rbnx update                  # update every remote provider in this deploy (asks y/N)
rbnx update -p <package dir>  # or just one package

Tear-down:

cd examples/webots && rbnx shutdown    # reads rbnx-boot/state.json,
                                       # SIGTERMs each component's PGID
bash examples/webots/sim/stop.sh       # then stop the Webots container

Full first-run walkthrough: docs/src/getting-started/quickstart.md.

Repository layout

system/         system components, one directory each
services/       default reference service implementations (memsearch, voiceprint, speech)
pylib/          Python SDK (robonix-api on PyPI)
capabilities/   contract TOMLs + ROS-style IDL tree (capabilities/lib/)
tools/          dev tooling — rbnx CLI + codegen
examples/       end-to-end deployments (webots, test_ci)
docs/           mdBook developer guide (submodule)
Cargo.toml      workspace for the Rust components (4 in system/, 2 in tools/)
Makefile        top-level orchestrate (build / install / fmt / check)

system/<name>/ and services/<name>/ and tools/<name>/ are each self-contained packages — Rust ones carry their own Cargo.toml, Python ones their own pyproject.toml. There is no top-level rust/ / python/ split; the runtime role is what determines where a component lives, not the implementation language.

Architecture

Dive deeper:

• Overview — control plane, one full request end-to-end
• Namespaces & contracts — how robonix/primitive/* / robonix/service/* / robonix/skill/* / robonix/system/* work
• Interface catalog — every primitive + service contract

Ecosystem

Robonix is built from small, swappable packages, each implementing one or more capability contracts under a robonix/<kind>/<area>/* namespace (browse them in the interface catalog). Packages come in two flavours:

• Built-in reference packages ship in this repo under services/ and deploy as-is.
• Community packages live in their own repos and are pulled in at boot via the manifest's url: field — fork one as a template to add new hardware or behaviour.

Built-in services — services/

Package	Namespace	What it does
memsearch	robonix/service/memory/*	Long-term fact / preference memory; the planner queries it for relevant past context.
speech	robonix/service/speech/*	Voice I/O — ASR, TTS (incl. streaming), dialog, speaker listing.
voiceprint	robonix/service/voiceprint/*	Speaker identification (ECAPA-TDNN) — enroll / identify / list / delete.

scene (3D scene graph) and the core runtime (atlas, executor, pilot, liaison) are system components under system/, not services.

Community packages

Standalone repos, cloned at boot via url: in the deploy manifest.

Primitives — one hardware device per package:

Package	Hardware	Namespace
ranger_chassis_rbnx	AgileX Ranger Mini v3 chassis	robonix/primitive/chassis/*
mid360_lidar_rbnx	Livox MID-360 — point cloud	robonix/primitive/lidar/*
mid360_imu_rbnx	Livox MID-360 — IMU	robonix/primitive/imu/*
realsense_camera_rbnx	Intel RealSense camera	robonix/primitive/camera/*

Services — robot-level algorithms:

Package	What it does	Namespace
mapping_rbnx	SLAM mapping (RTAB-Map + FAST-LIO2)	robonix/service/map/*
nav2_wrapper_rbnx	Navigation (Nav2 wrapper)	robonix/service/navigation/*

Skills — LLM-triggered composite tasks:

Package	What it does	Namespace
explore_rbnx	Autonomous frontier room exploration	robonix/skill/explore/*
greet_rbnx	Greet passers-by — YOLO person detection → VLM line → speak	robonix/skill/greet/*

Tools & deployments:

Repo	What it is
Robonix Skill Toolkit	Train VLA-based skills: collect teleop data, fine-tune an OpenVLA-OFT policy, deploy on a real arm (AgileX Piper).
ranger_mini_deploy	Full deploy manifest for the AgileX Ranger Mini robot at Syswonder Lab.

Contributors

Thanks goes to these wonderful people:

wheatfox 💻 🚧	Zhaobo Zhang 💻	Guowei Li 💻	wuzheng 💻	Kaile Liu 💻
Feiyang Li 💻	MuJue 💻	Zhenyu Zhang 💻	lhw2002426 💻	Zihao Zheng 🔧 🔣
qingfeng123 🔧 🔣	longyunhou 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

License

Mulan Permissive Software License, Version 2 (MulanPSL-2.0). See LICENSE.