ci: live MCP onboarding suite (opencode + Workers AI)#2496
ci: live MCP onboarding suite (opencode + Workers AI)#2496WcaleNieWolny wants to merge 4 commits into
Conversation
…d, contributor-only)
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Merging this PR will not alter performance
Comparing 
Footnotes
|
The TUI uploader hardcoded the builder-onboarding-tui/ prefix and a TUI-labeled step-summary heading, so a second workflow reusing it would collide with / overwrite the TUI report. Add backwards-compatible R2_PREFIX and REPORT_TITLE env overrides (defaults preserve the existing TUI behavior exactly) plus a trimSlashes helper that normalizes the prefix.
- Resolve PR context in a first step (mirrors the TUI workflow) so workflow_dispatch can publish: outputs number/sha from the pull_request event, the pr_number input, or gh pr list --head. When no PR resolves on dispatch, run on the branch ref and skip the R2/comment steps instead of hard-failing (PR_NUMBER was previously empty on dispatch, failing the R2 uploader). - Replace the unpinned 'curl | bash' opencode installer with a pinned registry install: bun install -g opencode-ai@1.17.4 (ships the opencode binary via platform optionalDependencies); $HOME/.bun/bin is added to $GITHUB_PATH. - SHA-pin actions/checkout@v6 and actions/setup-node@v6 (diverges from the repo's floating-tag convention, intentional per review). - Publish the MCP report under a distinct R2 prefix (R2_PREFIX=builder-onboarding-mcp) with its own REPORT_TITLE so it no longer collides with the TUI report. - Add a sticky PR comment with the report URL (justifies pull-requests: write), keyed by a unique marker, only when a PR is resolved. - Emit the cost line inside a fenced code block with backticks stripped so it can't inject markdown/HTML into the step summary. - Drop the dead cli/test/e2e-mcp/** path filter (that dir lives in the submodule, not capgo).
|
1 participant
What this is
Adds
.github/workflows/builder_onboarding_mcp_live.yml: a path-filtered, contributor-only GitHub Actions workflow that runs the live MCP onboarding suite against the capgo MCP server, driven by opencode as the agent engine and Cloudflare Workers AI as the model provider.⛔ Depends on (blockers) — do NOT merge yet
Cap-go/cli-mcp-testsmust merge first — it adds the opencode actor/judge, cost metering, token budget, per-path retry, and the tree report. Thetest:mcp:hermetic/test:mcp:livescripts this workflow runs live in that private submodule, not in capgo, so thebun runsteps do not exist until PR A is in.private/cli-mcp-testssubmodule must be bumped in this repo to the merged PR A commit on the submodule's default branch — not to a branch/PR-head commit. This PR intentionally does not bump the submodule, because bumping to an unmerged branch commit would pin capgo to a ref that can be rebased/force-pushed/deleted. The bump is a follow-up once PR A lands.This PR must not be merged until both land (PR A merged → submodule bumped to that merged SHA). Marked as DRAFT for that reason.
Required manual setup (one new secret)
CLOUDFLARE_API_TOKEN— a new repo secret scoped to Workers AI Read/Run, must be added manually (e.g.gh secret set CLOUDFLARE_API_TOKEN --repo Cap-go/capgo).CLOUDFLARE_ACCOUNT_ID, the R2 upload secrets (BUILDER_ONBOARDING_TUI_RESULTS_R2_UPLOAD_ACCESS_KEY_ID,TUI_RESULTS_R2_UPLOAD_SECRET_ACCESS_KEY), and the submodule token already exist.How it runs
ubuntu-latest.workflow_dispatch(with an optionalpr_numberinput) +pull_request(opened/synchronize/reopened/ready_for_review), path-filtered to the workflow,.gitmodules, theprivate/cli-mcp-testspointer,cli/src/build/onboarding/**,cli/src/mcp/**, and the onboarding test paths.prstep mirrors the TUI workflow — it outputsnumber/shafrom thepull_requestevent, thepr_numberdispatch input, orgh pr list --head. On a dispatch with no resolvable PR it runs on the branch ref and skips the R2 publish + PR comment instead of hard-failing.ifguards toworkflow_dispatchor PRs whose head repo is this repo (no fork secrets leak).bun install -g opencode-ai@1.17.4(theopencode-ainpm package ships theopencodebinary via platformoptionalDependencies);$HOME/.bun/binis added to$GITHUB_PATHandopencode --versionverifies it. Replaces the previous unversionedcurl … | bashinstaller.actions/checkout@df4cb1c…(v6.0.3) andactions/setup-node@48b55a0…(v6.4.0). This diverges from the repo's floating-tag convention — intentional supply-chain hardening for this workflow.continue-on-erroron those steps).CAPGO_E2E_MAX_ATTEMPTS=2,CAPGO_E2E_TOKEN_BUDGET=8000000.$GITHUB_STEP_SUMMARYfromresults/cost.jsoninside a fenced code block with backticks stripped, so the value can't inject markdown/HTML.upload-r2-results.mjsbut publishes under a distinct prefix (R2_PREFIX=builder-onboarding-mcp) with its ownREPORT_TITLE, so it no longer collides with / overwrites the TUI report in the shared bucket. The script gained backwards-compatibleR2_PREFIX/REPORT_TITLEenv overrides whose defaults preserve the TUI workflow's behavior exactly. Upload iscontinue-on-errorso a publish hiccup never masks a test failure.pull-requests: write), only when a PR number is resolved.Validation
python3 -c "import yaml; yaml.safe_load(...)"→ parses clean.actionlint(+shellcheck) run locally on both this workflow and the TUI workflow → 0 issues.node --checkonupload-r2-results.mjs→ clean; verified defaults reproduce the TUI prefix/title and the MCP overrides produce thebuilder-onboarding-mcpprefix.Test plan
Cap-go/cli-mcp-tests.private/cli-mcp-testssubmodule to the merged PR A commit.CLOUDFLARE_API_TOKENrepo secret (Workers AI Read/Run).gh workflow run "Builder onboarding MCP live (opencode + Workers AI)" --repo Cap-go/capgo→ expect green; job summary shows the fenced cost line; sticky PR comment + run link the R2tree-report.htmlunder thebuilder-onboarding-mcp/prefix.builder-onboarding-tui/(no collision).