feat: surface frontmatter parse errors via indexing diagnostics by lukemelia · Pull Request #5272 · cardstack/boxel

lukemelia · 2026-06-17T22:14:57Z

What & why

When a markdown file's YAML frontmatter fails to parse, MarkdownDef.extractAttributes indexes the body anyway and drops everything the frontmatter declared — a skill's commands, its boxel.kind, and so on. With only a console.warn, an author has no signal that those declarations were lost.

This routes frontmatter parse failures through the indexing-diagnostics surface the same way brokenLinks findings are: the row indexes cleanly (no has_error) but carries the failure so it's visible to the /_indexing-errors endpoint and the boxel realm indexing-errors CLI.

How

Capture (base/markdown-file-def.gts): on YAML parse failure, record { message, line?, column? } and route it out-of-band via Symbol.for('boxel:file-frontmatter-parse-error'), the same mechanism the polymorphic frontmatter field meta uses.
Lift (host/.../file-def-attributes-extractor.ts): pull the symbol off the search doc into FileDefExtractResult.frontmatterParseError.
Persist (runtime-common/index-runner/file-indexer.ts): merge it onto the row's diagnostics.frontmatterParseError on the success path.
Surface (runtime-common/realm.ts): the /_indexing-errors endpoint emits one resource per finding class. A healthy row (has_error = FALSE) carrying a diagnostics.frontmatterParseError yields a frontmatter-error finding; one carrying diagnostics.brokenLinks yields a broken-link finding; a row with both yields both, so neither signal masks the other. A row that yields more than one finding appends the finding class to its resource id (<type>::<url>::broken-link) to stay JSON:API-unique. The boxel realm indexing-errors CLI renders each (frontmatter parse error (line L:C): <message>). FrontmatterParseError type added to runtime-common.

Tests

Host integration: invalid YAML routes the parse-error marker and still indexes the body.
Realm-server endpoint: a frontmatter-error row surfaces with its payload, has_error = FALSE, no errorDoc; a healthy row with both broken links and a frontmatter error surfaces as two findings.
boxel-cli integration: endpoint → CLI round-trip, shortFrontmatterError formatting, and the combined-findings case.

🤖 Generated with Claude Code

Markdown frontmatter that failed to parse was swallowed with only a console.warn, silently dropping whatever the frontmatter declared (a skill's commands, its boxel.kind) while the file still indexed body-only. Capture the parse failure in MarkdownDef.extractAttributes and route it, mirroring the brokenLinks pattern, onto the index row's diagnostics.frontmatterParseError. The /_indexing-errors endpoint and the `boxel realm indexing-errors` CLI surface it as a new "frontmatter-error" finding class so authors can see and fix the YAML. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

github-actions · 2026-06-17T22:44:34Z

Preview deployments

Host staging preview

Host production preview

Host Test Results

1 files 1 suites 1h 54m 36s ⏱️
3 114 tests 3 099 ✅ 15 💤 0 ❌
3 133 runs 3 118 ✅ 15 💤 0 ❌

Results for commit cc8c56a.

Realm Server Test Results

1 files ±0 1 suites ±0 12m 24s ⏱️ -5s
1 737 tests +1 1 737 ✅ +1 0 💤 ±0 0 ❌ ±0
1 830 runs +1 1 830 ✅ +1 0 💤 ±0 0 ❌ ±0

Results for commit cc8c56a. ± Comparison against earlier commit 910ee27.

lukemelia · 2026-06-18T18:09:37Z

The Intern

The patch adds the intended frontmatter diagnostic path, but it can hide pre-existing broken-link findings on rows that also have frontmatter parse errors. That regression should be addressed before considering the change correct.

Review comment:

[P2] Preserve broken-link findings when frontmatter also fails — /Users/lmelia/p/cardstack/boxel/packages/runtime-common/realm.ts:6277-6280
When a healthy row has both diagnostics.brokenLinks and diagnostics.frontmatterParseError (for example, a markdown skill with invalid frontmatter and a broken card reference in its body), this classifies the row only as frontmatter-error. The non-JSON CLI then prints only the frontmatter summary, and JSON consumers filtering for type == "broken-link" no longer see an existing broken-link finding for that file. Consider emitting both findings or combining the summary so the broken-link signal is not hidden.

A healthy boxel_index row (has_error = FALSE) can carry both diagnostics.brokenLinks and diagnostics.frontmatterParseError — e.g. a markdown skill with invalid frontmatter and a dead card reference in its body. The _indexing-errors endpoint classified such a row only as frontmatter-error, so the CLI printed just the frontmatter summary and JSON consumers filtering for type == "broken-link" lost the broken-link signal entirely. Emit one resource per finding class instead of one per row. A row that yields more than one finding appends the finding class to its id so the two resources stay unique; single-finding rows keep the existing `<type>::<url>` id. Error rows are unchanged — brokenLinks still ride along as an attribute on the indexing-error resource. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

lukemelia · 2026-06-18T19:01:13Z

[Claude Code 🤖] Addressed in cc8c56a.

The _indexing-errors endpoint emits one resource per finding class. A healthy row (has_error = FALSE) carrying both diagnostics.brokenLinks and diagnostics.frontmatterParseError produces two resources — a broken-link finding and a frontmatter-error finding — so a consumer filtering by type sees both and the CLI prints a line per finding.

A row that yields more than one finding appends the finding class to its resource id (<type>::<url>::broken-link) to stay JSON:API-unique; a single-finding row keeps the <type>::<url> id. Error rows (has_error = TRUE) carry brokenLinks as an attribute on the indexing-error resource, since the render failure is the headline.

Endpoint and boxel-cli integration tests cover the combined case.

lukemelia · 2026-06-18T21:13:21Z

@habdelra I don't know too much about the indexing-errors endpoint, so give this PR a close look

habdelra

this is a perfect use the the diagnostics column and indexing errors endpoint. we handled broken links the very same way 👍

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Surfaces markdown YAML frontmatter parse failures as indexing diagnostics so authors can discover dropped frontmatter-declared behavior via /_indexing-errors and the boxel realm indexing-errors CLI.

Changes:

Capture frontmatter parse errors during markdown attribute extraction and route them out-of-band via a symbol.
Persist frontmatterParseError onto boxel_index.diagnostics and surface it as a new JSON:API finding type (frontmatter-error) alongside existing broken-link and indexing-error.
Extend CLI formatting/docs and add integration tests across host, realm-server, and CLI.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/base/markdown-file-def.gts	Captures YAML frontmatter parse failures and routes them via a symbol, while still indexing body-only.
packages/host/app/utils/file-def-attributes-extractor.ts	Lifts the parse-error symbol off the extracted attributes into the extract result for persistence.
packages/runtime-common/index-runner/file-indexer.ts	Merges lifted parse error into persisted `diagnostics.frontmatterParseError` on successful indexing.
packages/runtime-common/index.ts	Adds `FrontmatterParseError` type and wires it into extract response + diagnostics typing.
packages/runtime-common/realm.ts	Expands `/_indexing-errors` to also emit `frontmatter-error` findings and supports multiple findings per row.
packages/host/tests/integration/components/markdown-skill-frontmatter-test.gts	Verifies invalid YAML routes the marker and still indexes body content.
packages/realm-server/tests/realm-endpoints/indexing-errors-test.ts	Validates endpoint surfaces `frontmatter-error` and multi-finding rows.
packages/boxel-cli/src/commands/realm/indexing-errors.ts	Adds CLI support for `frontmatter-error` entries and short formatting helper.
packages/boxel-cli/tests/integration/realm-indexing-errors.test.ts	End-to-end coverage for CLI rendering of frontmatter errors and combined findings.
packages/boxel-cli/plugin/skills/indexing-errors/SKILL.md	Updates docs/examples to include the new finding type and usage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+          row.diagnostics.frontmatterParseError !== null
+            ? (row.diagnostics.frontmatterParseError as Record<string, unknown>)
+            : null;
        let hasError = row.error_doc != null;


+          type: 'indexing-error' | 'broken-link' | 'frontmatter-error';
+          attributes: Record<string, unknown>;
+        }[] = [];
        if (hasError) {


+        let frontmatterParseErrorSymbol = Symbol.for(
+          'boxel:file-frontmatter-parse-error',
+        );


+function toFrontmatterParseError(err: unknown): {
+  message: string;
+  line?: number;
+  column?: number;
+} {


… symbol drift Three small follow-ups to the frontmatter-error indexing surface (PR #5272): 1) `/_indexing-errors` now uses `has_error` from the SQL row as the discriminator instead of `error_doc != null`. The query filters on `has_error = TRUE`, so the previous branch silently dropped any row with `has_error = TRUE` but a NULL `error_doc` — it satisfied the filter but produced zero findings. Added a regression test. 2) Hoisted `'boxel:file-frontmatter-parse-error'` to a single exported constant `FRONTMATTER_PARSE_ERROR_SYMBOL` in runtime-common; the base markdown file-def, host extractor, and host test now all share it. 3) Replaced the inline `{ message, line?, column? }` shapes in `markdown-file-def.gts` with `FrontmatterParseError` (already exported from runtime-common) so the producer and consumer agree on one type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

lukemelia marked this pull request as ready for review June 18, 2026 21:12

lukemelia requested review from a team and habdelra June 18, 2026 21:12

habdelra requested a review from Copilot June 18, 2026 21:53

habdelra approved these changes Jun 18, 2026

View reviewed changes

Copilot started reviewing on behalf of habdelra June 18, 2026 22:00 View session

Copilot AI reviewed Jun 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: surface frontmatter parse errors via indexing diagnostics#5272

feat: surface frontmatter parse errors via indexing diagnostics#5272
lukemelia wants to merge 3 commits into
mainfrom
cs-11548-surface-frontmatter-parse-errors-via-indexing-diagnostics

lukemelia commented Jun 17, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 17, 2026 •

edited

Loading

Uh oh!

lukemelia commented Jun 18, 2026

Uh oh!

lukemelia commented Jun 18, 2026 •

edited

Loading

Uh oh!

lukemelia commented Jun 18, 2026

Uh oh!

habdelra left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

lukemelia commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What & why

How

Tests

Uh oh!

github-actions Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview deployments

Host Test Results

Realm Server Test Results

Uh oh!

lukemelia commented Jun 18, 2026

Uh oh!

lukemelia commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lukemelia commented Jun 18, 2026

Uh oh!

habdelra left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lukemelia commented Jun 17, 2026 •

edited

Loading

github-actions Bot commented Jun 17, 2026 •

edited

Loading

lukemelia commented Jun 18, 2026 •

edited

Loading