Skip to content

perf(cpp): push down multi-value aligned offsets#863

Open
ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:perf/multi-aligned-offset-pushdown
Open

perf(cpp): push down multi-value aligned offsets#863
ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:perf/multi-aligned-offset-pushdown

Conversation

@ColinLeeo

Copy link
Copy Markdown
Contributor

Motivation

Multi-value aligned reads accepted row offsets at the upper query layer, but the offset was not pushed down into the chunk/page read path. As a result, rows before the requested offset could still force unnecessary page decompression and decoding.

Changes

  • Add chunk-level offset skipping for multi-value aligned series scans when the time chunk and all value chunks have matching row counts.
  • Allow multi-value aligned page reads to accept row_offset.
  • Add page-level offset skipping in the serial multi-value aligned page loop using the time page statistic count.
  • Preserve partial-page residual offsets so the row reader can consume the remaining in-page offset correctly.
  • Push dense aligned/no-filter table row offsets into the shared multi-value SSI.
  • Synchronize residual SSI offset back to the row reader to avoid double skipping.
  • Add targeted tests for partial-page residual offsets, whole-page offset skipping, and whole-chunk offset skipping.

Performance

Tested on a 6-file time-series dataset with 264 logical series and about 996 million points. The benchmark repeatedly reads fixed 2880-point windows with randomized offsets.

Before this change:

  • ~34 ms/read
  • ~29 reads/s

After this change:

  • ~0.65 ms/read
  • ~1538 reads/s

Overall speedup is about 52x for this offset-heavy random window read workload.

For high-offset reads on the longest series, the 90% offset probe improved from about 410 ms to about 0.99 ms.

Validation

cmake --build cpp/build/test-noantlr --target TsFile_Test -j 8

cpp/build/test-noantlr/test/lib/TsFile_Test \
  --gtest_filter='TsFileReaderTest.*'

cpp/build/test-noantlr/test/lib/TsFile_Test \
  --gtest_filter='TableQueryByRowTest.*Offset*:TableQueryByRowTest.DenseSingleDeviceSsiLevelPushdown:TableQueryByRowTest.DenseAlignedNullsMustUseTimeRowCount'

Results:

- TsFileReaderTest.*: 14 passed, 1 disabled
- table row offset related tests: 15 passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant