feat(runner): event-driven tool relay (fs.watch + Daytona long-poll watch, polling fallback)

[mmabrouk]

Context

Every tool call from a sandboxed agent pays a polling tax. The in-sandbox writer drops a request file and sleeps in 300ms steps until the response appears; the runner finds requests by listing the relay directory every 300ms (1.5s once idle). One tool call adds 0.3 to 1.8s of pure waiting, and a ten-tool turn wastes over ten seconds. On Daytona the poll is also traffic: about 200 remote ls calls per minute per active turn.

This PR makes the relay event-driven. A filesystem event wakes the waiter the moment a file lands, and polling stays underneath as the correctness fallback. If a watcher dies, the relay degrades to exactly today's polling behavior, never to a hang.

Design and open questions: docs/design/agent-workflows/projects/event-driven-tool-relay/ (reviewed on #5232).

Changes

Atomic publication. Both directions now write to a temp file and rename it into place, so a reader can never see half-written JSON. On Daytona the response rename is the daemon's moveFs, which is rename(2) (verified against the daemon source), atomic for a same-directory move.

Wake on events, poll as backup. The writer arms one filesystem watch on the relay directory before it first checks for the response, so the response wakes it in about a millisecond. The runner does the same for incoming requests. Every watch path has a poll racing underneath it as the safety net, and repeated watch failure demotes the turn back to plain polling with one log line.

Before: the response is noticed 0 to 300ms after it lands. After: one filesystem event, about 1ms, and the poll only runs if the watch fails.

At most once. The runner deletes each request file the instant it reads it, and its first list of a reused relay directory marks any leftover files as already seen instead of running them. Together these end crash-redelivery: a request runs at most once, and a request lost to a crash surfaces as a writer timeout and a tool error, never as a repeated side effect. (This closed a real storm: without the delete, an in-flight request re-triggered every watch window.)

Telemetry. One stage=relay_pickup id=... pickup_ms=... wake=... line per executed request.

Two knobs, both defaulting to safe: the hop-1 response watch is on (..._RESPONSE_WATCH_ENABLED, degrades to poll); the Daytona remote watch is off until its live pass (..._REMOTE_WATCH_ENABLED).

Request volume

Active Daytona polling costs about 200 requests/min today. A healthy watch costs about 4.4/min (window re-issues plus two safety polls), roughly 45x less. Each executed tool call adds two daemon calls: the delete on pickup and a stat for the timing log.

Tests

• 71 relay unit and integration tests across four new suites, plus the real watch script spawned as a process (pre-existing file, event, window bound, awkward directory names, missing directory), the window clamp and jitter bounds, the storm regression, the orphan-snapshot rule, and telemetry ordering.
• Full services/runner suite: 1011 passed. The 4 remaining failures are pre-existing on this stack, from other in-flight lanes, and byte-identical before and after.

Live QA

• Local, verified: one real Claude turn in the playground with a workflow-reference tool. The tool ran through the relay and the runner logged pickup_ms=1.28 wake=activity. Event-driven pickup at 1.3ms against the 0 to 300ms polling baseline.
• Daytona, pending and honest: the remote watch ships off. Its flag-on live pass, the capacity numbers at peak concurrency, and the pickup distributions are rollout gates before any default flip. Unit and fake-daemon coverage stand in until then.

What to QA

• Playground, local sandbox, any agent with a tool: send a message that triggers the tool. The result renders and the runner log shows one stage=relay_pickup line with wake=activity and a single-digit pickup_ms.
• Kill switch: set AGENTA_AGENT_TOOLS_RELAY_RESPONSE_WATCH_ENABLED=false and repeat. Behavior is identical, pickup_ms grows up to about 300, wake=poll.
• Regression to watch: approval-gated tools ride ACP, not the relay. Approve a gated call and confirm it runs exactly once.

Notes

• Follow-up review fixes live in fix(runner): safe hardening from the stack review pass #5248 (top of the stack): a timeout bound on the file deletes, and a demotion-counter fix. They apply on top of this lane.

https://claude.ai/code/session_01Fr4A5zjs5rsufRaWgWiUNC

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agenta-documentation	Ready	Preview, Comment	Jul 12, 2026 3:01am

[coderabbitai]

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 89.19% which is sufficient. The required threshold is 60.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: an event-driven tool relay with watch-based wakeups and polling fallback.
Description check	✅ Passed	The description is detailed and directly describes the relay, watcher, atomic publish, and fallback changes in the PR.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat-event-driven-tool-relay

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[mmabrouk]

Reviewed with Codex (high effort). The event-driven loop is sound: no missed wakeups, no partial reads, no double execution, and local pickup is verified at about 1.3ms.

The review found two failure-path bugs and one parse nit: a hung file delete could wedge a turn, the demotion counter could fail to accumulate a real miss, and the window fallback accepted partial garbage. All three are fixed in #5248 on top of the stack.

Nothing here needs a decision from you.

[coderabbitai]

Actionable comments posted: 3

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e5bf685f-cfd5-4bfa-b9ae-f00f277fac23

📥 Commits

Reviewing files that changed from the base of the PR and between a8257c9 and a1ec038.

📒 Files selected for processing (20)

• docs/design/agent-workflows/documentation/tools.md
• docs/design/agent-workflows/interfaces/README.md
• docs/design/agent-workflows/interfaces/cross-service/runner-to-mcp-server.md
• services/runner/scripts/build-extension.mjs
• services/runner/src/engines/sandbox_agent.ts
• services/runner/src/engines/sandbox_agent/pi-assets.ts
• services/runner/src/tools/dispatch.ts
• services/runner/src/tools/relay-client.ts
• services/runner/src/tools/relay-protocol.ts
• services/runner/src/tools/relay-watch.ts
• services/runner/src/tools/relay.ts
• services/runner/tests/unit/relay-atomic-publish.test.ts
• services/runner/tests/unit/relay-client.test.ts
• services/runner/tests/unit/relay-loop.test.ts
• services/runner/tests/unit/relay-watch.test.ts
• services/runner/tests/unit/sandbox-agent-orchestration.test.ts
• services/runner/tests/unit/sandbox-agent-pi-assets.test.ts
• services/runner/tests/unit/tool-callref-bindings.test.ts
• services/runner/tests/unit/tool-direct.test.ts
• services/runner/tests/unit/tool-relay-guard.test.ts

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n "AGENTA_AGENT_TOOLS_RELAY_REMOTE_WATCH_WINDOW_MS|REMOTE_WATCH_WINDOW_MS|watch window|parseInt" .

Repository: Agenta-AI/agenta

Length of output: 9202

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '60,100p' services/runner/src/tools/relay-watch.ts
printf '\n--- TESTS ---\n'
sed -n '522,620p' services/runner/tests/unit/relay-watch.test.ts
printf '\n--- DOCS ---\n'
sed -n '275,288p' docs/design/agent-workflows/documentation/tools.md

Repository: Agenta-AI/agenta

Length of output: 5923

---

Use full-string validation for the watch-window env var. resolveRemoteWatchWindowMs() uses Number.parseInt, so values like 30000junk are accepted as 30000. Either reject non-numeric suffixes or document that prefix parsing is allowed.

[coderabbitai]

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Align the inventory status with the contract page.

This row says “stdio wired”, but runner-to-mcp-server.md says user stdio servers are disabled and mcp-server.ts is a refusing stub. If “stdio wired” refers to another channel, name it; otherwise update the status.

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Validate RELAY_POLL_MS/RELAY_TIMEOUT_MS parsing — bare Number() silently yields NaN on malformed input.

Unlike relayEnvFlag below (explicit allow-list, safe fallback), these two constants have no validation. A malformed env value (e.g. "30000junk", empty string, or a negative number) makes Number(...) return NaN or a negative number. Since waitForRelayResponse's deadline = Date.now() + opts.timeoutMs would then be NaN/in the past, Date.now() < deadline is always false, so every relay call throws RelayTimeoutError immediately regardless of the actual response timing.

🛡️ Proposed fix

+function parsePositiveIntEnv(value: string | undefined, fallback: number): number {
+  const n = Number(value);
+  return Number.isFinite(n) && n > 0 ? n : fallback;
+}
+
-export const RELAY_POLL_MS = Number(
-  process.env.AGENTA_AGENT_TOOLS_RELAY_POLLING ?? 300,
-);
-export const RELAY_TIMEOUT_MS = Number(
-  process.env.AGENTA_AGENT_TOOLS_RELAY_TIMEOUT ?? 60000,
-);
+export const RELAY_POLL_MS = parsePositiveIntEnv(
+  process.env.AGENTA_AGENT_TOOLS_RELAY_POLLING,
+  300,
+);
+export const RELAY_TIMEOUT_MS = parsePositiveIntEnv(
+  process.env.AGENTA_AGENT_TOOLS_RELAY_TIMEOUT,
+  60000,
+);

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	export const RELAY_POLL_MS = Number(
	process.env.AGENTA_AGENT_TOOLS_RELAY_POLLING ?? 300,
	);
	export const RELAY_TIMEOUT_MS = Number(
	process.env.AGENTA_AGENT_TOOLS_RELAY_TIMEOUT ?? 60000,
	);
	function parsePositiveIntEnv(value: string | undefined, fallback: number): number {
	const n = Number(value);
	return Number.isFinite(n) && n > 0 ? n : fallback;
	}
	export const RELAY_POLL_MS = parsePositiveIntEnv(
	process.env.AGENTA_AGENT_TOOLS_RELAY_POLLING,
	300,
	);
	export const RELAY_TIMEOUT_MS = parsePositiveIntEnv(
	process.env.AGENTA_AGENT_TOOLS_RELAY_TIMEOUT,
	60000,
	);

[mmabrouk]

The at-most-once guarantee, worth reading closely. The runner deletes each request file the instant it reads it, and its first list of a reused relay directory marks any leftover files as seen (around line 692) instead of running them. Together these end crash-redelivery on both backends: a request runs at most once per publication, and a request lost to a crash surfaces as a writer timeout and a tool error, never as a replayed side effect. The follow-up fix in #5248 bounds these deletes with a timeout so a hung daemon call cannot wedge the loop here.

[mmabrouk]

where are these defined. are they on the compose level? we re starting to have to many env vars and ff, just want to make sure it is maintainable

[mmabrouk]

lgtm