Skip to content

perf: skip lid blocks#407

Open
cheb0 wants to merge 1 commit into
mainfrom
0-lid-block-skip
Open

perf: skip lid blocks#407
cheb0 wants to merge 1 commit into
mainfrom
0-lid-block-skip

Conversation

@cheb0

@cheb0 cheb0 commented Apr 24, 2026

Copy link
Copy Markdown
Collaborator

Description

Summary

  • new frac version: v5
  • LID block ext1 now has firstLID and lastLID encoded
  • isContinued is only used for older frac versions
  • no legacy format TID adjusting for a new frac version
  • as previously, we always locate and load the first block eagerly but calling NextGeq allows to skip some blocks

Measurements

With 16k LID blocks, there are only 4 search requests (out of 50+ I tested) where this feature improves cold query perf. All four search requests fit into "intersect dense and sparse column" type.

Query env Total cold, ms   LID blocks read cold (branch), ms   LID blocks read (branch) cold diff LID blocks read, diff Trial_1 Trial_2 Trial_3
service:E and login:L and moduleName:S and (zone:A or zone:B or zone:C) and exists:message prod 0 22.29 ±1.03 164 20.6 ±1.10 62 -7.6% -62.2% -12.6% -8.9% -7.6%
request_host:huge AND remote_addr:X (found) lb 0 18.22 ±0.97 69 17.53 ±0.82 42 -3.8% -39.1% -3.8% -9.8% -3.8%
request_host:small AND response_status:200 lb 69 33.64 ±1.92 248 30.44 ±1.60 138 -9.5% -44.4% -2% -10.9% -9.5%
response_status:500 | by request_method lb 172 52.58 ±2.14 377 51.68 ±1.54 333 -1.7% -11.7% -7.9% -20.6% -1.7%

  • I have read and followed all requirements in CONTRIBUTING.md;
  • I used LLM/AI assistance to make this pull request;

@cheb0 cheb0 marked this pull request as ready for review April 24, 2026 11:33
Comment thread frac/sealed/lids/table.go
res := index
for i := index + 1; i < uint32(len(t.MinTIDs)); i++ {
if t.MinTIDs[i] == tid && nextLID >= t.FirstLIDs[i] {
res = i

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: here i is int, but in SeekBlockLeq() it is uint32. let's make it consistent, for example:

for i := int(index) + 1; i <len(t.MinTIDs); i++ {
	if t.MinTIDs[i] == tid && nextLID >= t.FirstLIDs[i] {
		res = uint32(i)
		continue
	}
	break
}

@cheb0 cheb0 added the performance Features or improvements that positively affect seq-db performance label May 12, 2026
@eguguchkin eguguchkin added this to the v0.72.0 milestone Jun 3, 2026
Base automatically changed from 330-lid-block-size-config to main June 5, 2026 12:12
@eguguchkin eguguchkin marked this pull request as draft June 8, 2026 11:49
@eguguchkin eguguchkin removed this from the v0.72.0 milestone Jun 8, 2026
@cheb0 cheb0 force-pushed the 0-lid-block-skip branch 3 times, most recently from 2ec59f6 to b7a0835 Compare July 15, 2026 13:37
@cheb0 cheb0 marked this pull request as ready for review July 15, 2026 13:37
@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Caution

Review failed

An error occurred during the review process. Please try again later.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cheb0 cheb0 force-pushed the 0-lid-block-skip branch from b7a0835 to b3f137c Compare July 15, 2026 13:42
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.33333% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.49%. Comparing base (42fb1c0) to head (b3f137c).

Files with missing lines Patch % Lines
frac/sealed_loader.go 42.85% 11 Missing and 1 partial ⚠️
frac/sealed/lids/table.go 74.07% 4 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #407      +/-   ##
==========================================
+ Coverage   70.47%   70.49%   +0.01%     
==========================================
  Files         232      232              
  Lines       18364    18406      +42     
==========================================
+ Hits        12942    12975      +33     
- Misses       4448     4455       +7     
- Partials      974      976       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
Indexer-4 42fb1c 0f79ba
572784732.00 B/op 782569576.00 B/op 1.37 🔴

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
Indexer-4 42fb1c 3b55ca
572784732.00 B/op 774026566.00 B/op 1.35 🔴

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table
Name Previous Current Ratio Verdict
Indexer-4 42fb1c 7aacd4
572784732.00 B/op 785417068.00 B/op 1.37 🔴

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Features or improvements that positively affect seq-db performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants