Skip to content

Fix pull attribute ordering#10

Merged
maxweber merged 2 commits into
mainfrom
pull-attr-ordering
Jul 16, 2026
Merged

Fix pull attribute ordering#10
maxweber merged 2 commits into
mainfrom
pull-attr-ordering

Conversation

@maxweber

Copy link
Copy Markdown
Owner

Summary

Two commits: the pull-attribute-ordering fix, plus the fixes for everything a high-effort multi-agent review of that commit confirmed.

cd8becb — Fix pull attribute ordering

dbval's indexes order attribute components by their serialized (pr-str) form, but pull's merge-join between the sorted pattern attributes and the :eavt datom stream compared attributes with keyword compare. The two orders disagree as soon as a pattern mixes plain and namespaced keywords (e.g. :type vs :instagram.story.image.preview/blob), desyncing the walk and silently dropping attributes from pull results. The fix centralizes the ordering in attr-sort-key/attr-compare and uses it in tuple-list, both pull comparison sites, and the parser's attribute sort (:db/id deliberately last). With regression test.

26dd323 — review findings, all confirmed by adversarial verification

  1. FilteredDB pulls leaked other entities' data: attrs-frame's non-DB branch had its take-while outside the ->> — dead code under util/cond+ — leaving the scan unbounded and entity-unguarded. Pulling an attribute an entity lacks assoc'd the next entity's value into the result (pre-existing bug; the ordering commit edited the dead form without noticing). The take-while now lives inside the pipeline.
  2. String-spelled keyword attributes silently dropped: \":name\" and \":db/id\" in a pattern — forms parse-attr-spec explicitly accepts — no longer matched keyword datoms. parse-attr-name now normalizes \":foo\" strings to keywords up front; bare strings still name genuine string attributes.
  3. UTF-16 vs UTF-8 ordering divergence: attr-compare used Java string compare (UTF-16 code units) while the store orders serialized attributes by UTF-8 bytes (code points) — the orders disagree for supplementary-plane characters (emoji). Comparison now runs in code-point order via compare-attr-keys. Fixing this surfaced one more spot the review hadn't isolated: the parser's sort-by also used default string compare, inverting first-attr/last-attr for such patterns and making the pull return nil outright.
  4. Hot-loop cleanup: the attribute sort key is computed once at parse time and cached on PullAttr instead of pr-str-ing the pattern attr's name on every frame-loop iteration.

Test plan

  • New regression tests: namespaced-attr-after-ref-attr ordering (the original bug), FilteredDB bounded scan / no cross-entity leak, string-spelled attrs (\":name\", \":db/id\"), supplementary-plane attribute pairs.
  • Full suite: 171 tests, 1075 assertions, 0 failures, 0 errors.

🤖 Generated with Claude Code

https://claude.ai/code/session_014dX8tTR4yFh5SyBGw3atpo

maxweber and others added 2 commits July 16, 2026 08:51
- attrs-frame's non-DB branch (FilteredDB pulls) had its take-while
  outside the ->> — dead code under util/cond+ — so the scan was
  unbounded and unguarded by entity: pulling an attribute an entity
  lacks leaked the next entity's value into the result. The take-while
  now lives inside the pipeline.
- The parser stopped normalizing string-spelled keyword attributes:
  ":name" (and ":db/id") in a pull pattern were silently dropped.
  parse-attr-name now normalizes ":foo" strings to keywords up front;
  bare strings still name genuine string attributes.
- attr-compare ordered attributes by Java's UTF-16 string compare, but
  the store orders their serialized form by UTF-8 bytes (code points);
  the orders disagree for supplementary-plane characters. Comparison now
  runs in code-point order (compare-attr-keys) — including the parser's
  sort-by, which also used default string compare and made pulls
  combining such attributes return nil.
- The attribute sort key is computed once at parse time and cached on
  PullAttr instead of pr-str-ing the pattern attr's name on every
  frame-loop iteration.

With regression tests for the FilteredDB bound, string-spelled attrs,
and supplementary-plane attribute pairs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@maxweber
maxweber merged commit bd3b257 into main Jul 16, 2026
1 check passed
@maxweber
maxweber deleted the pull-attr-ordering branch July 16, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant