Follow-up from PR #3819 (review by thepastaclaw).
Problem
sync_notes_across (packages/rs-platform-wallet/src/wallet/shielded/sync.rs) streams from aligned_start to the chain tip in a single call (the ShieldedSyncConfig.max_concurrent bound only caps in-flight fetches, not wallet-side buffers). Inside that one call it accumulates three in-memory buffers and only flushes them after the stream fully drains:
scanned_nullifiers: Vec<[u8; 32]> — pushed for every scanned action (sync.rs:429), consumed only at end-of-pass via apply_scanned_nullifier_spends (sync.rs:664). This is the dominant, dense growth: 32 B × every note.
decrypted_by_subwallet / recovered_outgoing_by_subwallet — sparse (only the wallet's own received / sent notes).
Resident memory therefore scales with the total scanned range, not the batch size:
| Pool scanned in one call |
scanned_nullifiers RAM |
| 1M notes |
~32 MB steady, ~48–64 MB peak (Vec capacity doubling) |
| 10M notes |
~320 MB |
The commitment tree itself is SQLite-backed, so it is not the RAM driver — this buffer is. On a full restore / large catch-up this is real heap pressure, and it is fatal in iOS app-extension contexts (notification-service / background-refresh extensions have ~24–30 MB jetsam caps — 32 MB kills them instantly).
Why the obvious quick fix does not work
thepastaclaw suggested applying apply_scanned_nullifier_spends per batch. That alone is incorrect: decrypted notes are only accumulated in decrypted_by_subwallet during the loop and saved via store.save_note once, end-of-pass (sync.rs:613). A per-batch spend replay would run against a store that has not saved this pass's receipts yet, missing every same-pass receive-then-spend.
Correct fix
Process each streamed batch fully instead of accumulate-then-flush — i.e. move the receipt-save + outgoing-record + spend-apply into the while let Some(item) = stream.next() loop:
- append commitments (already per-batch)
- trial-decrypt +
save_note this batch's receipts
- OVK-recover +
record_outgoing_note
apply_scanned_nullifier_spends for this batch's nullifiers (a per-batch Vec, or a reused/drained buffer)
This bounds all three buffers to one batch (~8192 notes ≈ 256 KB, flat regardless of pool size) and drops the accumulator maps entirely (cleaner, more streaming-friendly).
Correctness: the receipt-before-spend invariant still holds — batches stream in tree/position order, and each batch saves its receipts (step 2) before applying its spends (step 4); a note received in an earlier batch is already saved when a later batch's spend matches it.
One behavior change: incoming notes' block_height becomes the batch's height instead of the pass-wide max — which makes them consistent with outgoing notes, that already use batch.block_height.
Notes
- Pre-release v12 shielded pool; no migration concerns.
- The Part-A tests cover
apply_scanned_nullifier_spends in isolation but not the full sync_notes_across loop, so this restructure should add loop-level coverage (received-then-spent across batch boundaries; same-batch receive+spend).
Follow-up from PR #3819 (review by thepastaclaw).
Problem
sync_notes_across(packages/rs-platform-wallet/src/wallet/shielded/sync.rs) streams fromaligned_startto the chain tip in a single call (theShieldedSyncConfig.max_concurrentbound only caps in-flight fetches, not wallet-side buffers). Inside that one call it accumulates three in-memory buffers and only flushes them after the stream fully drains:scanned_nullifiers: Vec<[u8; 32]>— pushed for every scanned action (sync.rs:429), consumed only at end-of-pass viaapply_scanned_nullifier_spends(sync.rs:664). This is the dominant, dense growth: 32 B × every note.decrypted_by_subwallet/recovered_outgoing_by_subwallet— sparse (only the wallet's own received / sent notes).Resident memory therefore scales with the total scanned range, not the batch size:
scanned_nullifiersRAMThe commitment tree itself is SQLite-backed, so it is not the RAM driver — this buffer is. On a full restore / large catch-up this is real heap pressure, and it is fatal in iOS app-extension contexts (notification-service / background-refresh extensions have ~24–30 MB jetsam caps — 32 MB kills them instantly).
Why the obvious quick fix does not work
thepastaclaw suggested applying
apply_scanned_nullifier_spendsper batch. That alone is incorrect: decrypted notes are only accumulated indecrypted_by_subwalletduring the loop and saved viastore.save_noteonce, end-of-pass (sync.rs:613). A per-batch spend replay would run against a store that has not saved this pass's receipts yet, missing every same-pass receive-then-spend.Correct fix
Process each streamed batch fully instead of accumulate-then-flush — i.e. move the receipt-save + outgoing-record + spend-apply into the
while let Some(item) = stream.next()loop:save_notethis batch's receiptsrecord_outgoing_noteapply_scanned_nullifier_spendsfor this batch's nullifiers (a per-batchVec, or a reused/drained buffer)This bounds all three buffers to one batch (~8192 notes ≈ 256 KB, flat regardless of pool size) and drops the accumulator maps entirely (cleaner, more streaming-friendly).
Correctness: the receipt-before-spend invariant still holds — batches stream in tree/position order, and each batch saves its receipts (step 2) before applying its spends (step 4); a note received in an earlier batch is already saved when a later batch's spend matches it.
One behavior change: incoming notes'
block_heightbecomes the batch's height instead of the pass-wide max — which makes them consistent with outgoing notes, that already usebatch.block_height.Notes
apply_scanned_nullifier_spendsin isolation but not the fullsync_notes_acrossloop, so this restructure should add loop-level coverage (received-then-spent across batch boundaries; same-batch receive+spend).