Update serialisation to write RRIs#5148
Draft
backspace wants to merge 25 commits into
Draft
Conversation
Contributor
Preview deploymentsHost Test Results 1 files ±0 1 suites ±0 1h 55m 51s ⏱️ + 3m 13s Results for commit 122514f. ± Comparison against earlier commit da8336d. For more details on these errors, see this check. Realm Server Test Results 1 files ±0 1 suites ±0 13m 18s ⏱️ +21s Results for commit 122514f. ± Comparison against earlier commit da8336d. For more details on these errors, see this check. |
Core of the persistence-format migration to RealmResourceIdentifier (RRI) prefix form. The `@cardstack/base/` realm prefix is already registered in the VirtualNetwork; this makes the runtime canonicalize base-realm module identifiers to that prefix form everywhere they are resolved, compared, or persisted. - virtual-network.ts `unresolveURL`: chase a URL-shaped input through any registered virtual → real URL mapping and retry the realm-prefix match, so a virtual-alias URL canonicalizes to its RRI prefix. This is what flips `internalKeyFor` (and therefore index keys, `adoptsFrom`, `types`, and `deps`) to RRI form for base modules. - definition-lookup.ts / host realm.ts: normalize both sides of the local-realm membership check via `unresolveURL` so an RRI-form (or resolved-URL) input still matches a realm keyed by its virtual alias. - loader.ts: collapse dependency-tracker keys onto a single canonical URL form so a base module imported via the virtual alias and the same module imported via the RRI prefix don't fragment into two tracker entries. - constants.ts: base code refs (baseRef/baseCardRef/specRef/etc.) move to the `@cardstack/base/` prefix form via the new `baseRealmRRI`. - module-syntax.ts: emit RRI-form imports for new fields, reusing an existing equivalent import (URL or RRI) instead of duplicating. - realm.ts / render routes / index-query engines: canonicalize emitted deps, the module-not-found message, and the realm-config adoptsFrom check to RRI form. - realm-server main.ts/worker.ts/create-realm.ts: register the scoped `@cardstack/X/` prefix alongside the cardstack.com URL alias and emit RRI refs in bootstrapped realm config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Surfaces that generate source code, hold lint config, or are realm content now use the `@cardstack/base/` prefix form, matching what the runtime resolves and persists. - eslint-plugin-boxel import-utils + missing-card-api-import-config: the auto-import rule's configured target modules move to RRI form, and the rule treats URL-form and RRI-form base imports as equivalent so it merges into an existing import of either form rather than emitting a duplicate. - create-file-modal / edit-field-modal / item-button: generated import lines and default code refs emit RRI form. - base/cards-grid + cards-grid-layout, experiments-realm cards: realm content card refs move to RRI form. - boxel-cli parse, boxel-ui spec generator, software-factory smoke scripts, vscode-boxel-tools skills: tooling that emits or asserts base code refs moves to RRI form. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mechanical sweep across the test suites: fixtures and assertions that referenced base-realm modules in virtual-alias URL form (`https://cardstack.com/base/X`) now use the RRI prefix form (`@cardstack/base/X`), matching what the runtime resolves, serializes, and indexes after the preceding two commits. Covers card-source fixtures, `meta.adoptsFrom.module` assertions, `internalKeyFor`/types/deps expectations, deps-endpoint and search assertions, DOM `data-test-card` selectors, and prompt/error-message expectations across host, realm-server, ai-bot, matrix, boxel-cli, eslint-plugin-boxel, software-factory, and runtime-common tests. TypeScript module specifiers (`from '...'`, `typeof import('...')`) are intentionally left in URL form: they resolve through pnpm/node and flipping them breaks the dual-type-identity guarantee. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
backspace
force-pushed
the
serialise-rri-cs-10753
branch
from
576bea3 to
448bc0e
Compare
Temporary diagnostic to capture, for a missing contained-field definition lookup, the computed lookup key, the canonical module URL, and the definition keys actually present in the loaded module entry. Used to pin down where the RRI key form diverges between persistence and lookup. To be reverted once the root cause is fixed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The line-917 (missing-definition-key) diagnostic never fired, so the CardInfoField FilterRefers throw originates at an earlier branch. Add the same key/URL/realm logging to the no-context and no-module-entry / entry-error branches to pin down which one and with what canonical URL vs. registered realm URLs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts: # packages/eslint-plugin-boxel/lib/utils/import-utils.js
The definition lookup canonicalizes a code ref to a module URL that is then used both as the cache key and as the module-load / prerender target. For base-realm refs this resolved to the bare real URL (`http://localhost:4201/base/X`), but the base realm is reachable in-process only under its virtual-alias URL (`https://cardstack.com/base/X`) — the alias is what the loader's mounted handler / origin-matched auth interceptor recognizes. Fetching the bare real URL bypasses that and fails at the transport ("Failed to fetch"), which put the base card-api module entry into an error state and made every contained-field definition lookup during search throw FilterRefersToNonexistentType (HTTP 500 on _search). `canonicalURL` now maps the resolved real URL back to its virtual-alias form when a URL mapping is registered (only base today), so the module-load target is the fetchable form — matching what the pre-flip URL-form code path fetched. Realms with no alias map are served at their real URL and are returned unchanged. Handles both the RRI-prefix input and an already-resolved real-URL input. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The `missing-card-api-import` config emits the RRI prefix form, so when the `_lint` endpoint auto-adds a missing base import (e.g. StringField) it now writes `@cardstack/base/string`. Update the expected output for the cases that exercise an auto-added import. Imports the source already declares in URL form stay URL form — the rule merges the new named specifiers into the existing line via the URL↔RRI equivalence — so the `card-api` assertion that the prior sweep over-flipped to RRI is restored to the URL form the merged import actually keeps. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Drops the temporary console.error scaffolding from the FilterRefers throw sites now that the base module-load root cause is fixed (resolve to the fetchable virtual-alias form). The throw sites return to their original form. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Both exercise the virtual-network / header-sanitizer boundary, where
the value is and must remain a real URL — the prior fixture sweep
over-flipped them to the RRI prefix form, which is not a valid URL:
- virtual-network-test redirect: addURLMapping's virtual key and the
handler's Location header are virtual-alias URLs; `new URL('@cardstack/base/')`
is invalid. Restore `https://cardstack.com/base/`.
- consuming-realm-header sanitizer: `sanitizeConsumingRealmHeader`
accepts plain http(s) realm URLs and returns null for anything else,
so an `@cardstack/base/` input correctly returns null. Restore the
plain https URL the "accepts a plain https realm URL" case intends.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…entity)
A `typeof import('...')` annotation is a TypeScript module specifier,
which the migration deliberately keeps in virtual-alias URL form — the
prefix form resolves through the `@cardstack/base` node_modules symlink
and tsc then sees `card-api` as two distinct module identities
(symlink path vs the real `packages/base` path), yielding ~105
"BaseDef is not assignable to BaseDef" errors across card-api.gts.
The fixture sweep accidentally flipped this one specifier to the RRI
prefix form. Restore the URL form, consistent with every other TS
specifier in the codebase. This is the sole trigger of the host
`lint:types` dual-identity failure.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The generated-import canonicalization was rewriting every registered realm's module ref to its RRI prefix form, including cross-realm references (e.g. a card in workspace A extending one in workspace B became `@test-realm/test2/animal`). Generated user source should address the base realm by its stable alias (`@cardstack/base/X`, matching the emitted Component import) but keep cross-realm references as their absolute URL, which is how they resolve. Narrow the canonicalization to the base realm; other realms keep the absolute URL. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`Object.keys(embeddedHtml)` / `Object.keys(fittedHtml)` reflect the stored `embedded_html` / `fitted_html` key order, which is lexical after the storage round-trip, not semantically meaningful — consumers look these up by key. With the base CardDef key in RRI prefix form (`@cardstack/base/card-api/CardDef`) it sorts ahead of the test realm's `http://test-realm/...` keys, where the old `https://cardstack.com/...` key sorted after. Compare the key sets order-independently so the assertion tracks the contract (which types are present) rather than a storage-dependent ordering. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…sts) The two "indexing identifies an instance's ... references" tests assert the instance's `deps`. Two intended shifts now apply: - base modules serialize as RRI form (`@cardstack/base/X`), not the virtual-alias URL. - boxel-ui deps resolve to the bare prefix (`@cardstack/boxel-ui/X`) because the base-import root change (#5081) registers an `addRealmMapping('@cardstack/boxel-ui/', …)`; boxel-host has no such mapping, so its deps correctly stay `https://packages/@cardstack/boxel-host/…`. Both are intended runtime behavior (no code change); the expectations are regenerated from the runtime's own sorted output. The arrays are compared after `.sort()`, so the re-prefixed entries reorder into the `@`-prefixed group ahead of the `http(s)://` entries. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
backspace
force-pushed
the
serialise-rri-cs-10753
branch
2 times, most recently
from
198ba1f to
3a33465
Compare
# Conflicts: # packages/host/tests/integration/realm-indexing-test.gts
backspace
force-pushed
the
serialise-rri-cs-10753
branch
from
ff1a2e8 to
e4074e0
Compare
A reference in scoped RRI form (e.g. @cardstack/base/Theme/brand-guide) is an absolute cross-realm identifier and must be left verbatim during relativization. The guard only skipped references whose prefix was registered in the current VirtualNetwork, so a scoped reference to a realm this VN did not know fell through to relative URL resolution and came back with a spurious "./" prefix. Key on the leading "@" so any scoped reference is preserved whether or not its prefix is registered here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
These fixtures referenced base modules by the virtual-alias URL form (https://cardstack.com/base/X) for both the input source and the expected output. With field code refs carried in RRI prefix form, that left the input one form and the assertions another: extracted field code refs came back as RRI while the input imports were URL, and a generated import for a base module emitted RRI into an otherwise-URL file (and failed to merge with the equivalent URL import, duplicating it). Author the fixtures in RRI form throughout so input, generated imports, and extracted code refs all share the canonical identifier. No module-syntax behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
addField imported the field decorator and the field-type helper from the hardcoded virtual-alias URL form of base/card-api, while the field module itself was emitted in canonical RRI prefix form. In a module that imports base/card-api in RRI form, that mismatch produced a second, URL-form card-api import (with renamed `field0`/`contains0` bindings) instead of merging into the existing one, and the extracted field code refs then pointed at the duplicate's URL form. Resolve the decorator/field-type import to the canonical form and reuse the module's existing equivalent card-api import specifier, so the new identifiers merge into whatever form the file already uses. Populate the module-syntax test's VirtualNetwork with the base realm's URL and RRI mappings so this URL↔RRI equivalence is resolvable, as it is in the host and realm-server networks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # packages/host/tests/acceptance/code-submode/create-file-test.gts
The card serializer resolved a scoped RRI module reference (e.g. @cardstack/base/card-api) to the serializing realm's concrete real base URL before persisting it. Because a realm server's real base URL is environment-specific — and may differ in protocol/host from where a render host can reach base — that baked a non-portable, potentially cross-origin URL into the stored card. A render host configured for a different base origin (e.g. https base while the realm server resolves to http) then cannot fetch the baked URL and the card indexes as an error. A scoped RRI is already the canonical, deployment-independent form, so preserve it verbatim rather than resolving it. This mirrors how a URL-form alias round-trips (resolveURL leaves it unchanged); the prefix form must be left alone the same way. Truly bare loader specifiers still fall through to the importMap shim. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Different index paths recorded base dependencies in mixed forms: the instance render path already unresolves deps to the portable RRI prefix (@cardstack/base/...), but the file-extract path stored the virtual-alias URL form (https://cardstack.com/base/...). The same module therefore appeared under two identifiers across entries, and consumers expecting the canonical prefix form (e.g. fallback file-extractor deps, scoped-CSS deps) did not find it. Normalize every dependency URL through unresolveURL at the index-writer boundary so a single canonical prefix form is persisted. Dependency invalidation already searches both the real and prefix forms, so matching is unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The scoped-CSS dependency assertions matched the virtual-alias URL form (https://cardstack.com/base/...glimmer-scoped.css), but dependencies are persisted in the canonical RRI prefix form. Update the patterns to @cardstack/base/... so they match what the index stores — the assertion messages already name the prefix-form module. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # packages/host/tests/acceptance/code-submode/create-file-test.gts
Module dependencies are stored in canonical, deployment-independent form: base modules as the @cardstack/base/ RRI prefix, and the live test realm's modules at the standard localhost:4202 address even when served under the env-mode hostname. Update the realm-indexing expectations accordingly — the fallback file-extractor dep to @cardstack/base/file-api, and the test-realm module dep to its canonical localhost:4202 form. The cards' adoptsFrom still point at the running realm; only the indexed dep form is canonical. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RRI handling currently deals with an ambiguity where one could be the
https://cardstack.com/base/style we’re moving away from (or another URL that will remain valid) or@cardstack/whatever-style. The virtual network is used to normalise, and is therefore passed around.This moves
adoptsFrom.module, relationships, and other RRIs to all be in their correct forms, which opens the way to CS-11450 removing the virtual networks everywhere and delaying resolution of non-URL RRIs until the moment offetch.