OAK-12089: add multi-target routing and per-target catch-up lifecycle#2818
Draft
bhabegger wants to merge 141 commits into
Draft
OAK-12089: add multi-target routing and per-target catch-up lifecycle#2818bhabegger wants to merge 141 commits into
bhabegger wants to merge 141 commits into
Conversation
* OAK-12010 Simplified index management * OAK-12010 Simplified index management * OAK-12010 Simplified index management - improve flaky tests
Updated documentation for Composite NodeStore, including changes to section headings and clarifications on seeding and mount behavior.
Corrected headings and improved clarity in the text regarding NodeStore initialization and design limitations.
…ated (apache#2795) * OAK-12133 With the segment store, binary properties can not be aggregated * OAK-12133 With the segment store, binary properties can not be aggregated
* OAK-12136 : add AGENTS.md for oak-store-document module * OAK-12136 : exclude oak-store-document/AGENTS.md from RAT license check * OAK-12136 : use wildcard to exclude all AGENTS.md files from RAT check * OAK-12136 : exclude all AGENTS.md files from RAT license check
…pache#2801) * OAK-12139 : add Claude Code skill for oak-store-document OSGi config * OAK-12139 : move skill to oak-store-document module level * OAK-12139 : fix RAT exclusion to cover nested .claude directories * OAK-12139 : split skill into supporting files for readability * OAK-12139 : fix RAT for submodule-level .claude skills and AGENTS.md * OAK-12139 : fix skill frontmatter and gitignore trailing newline * OAK-12139 : update AGENTS.md skill reference to include direct file path
…guava (apache#2803)" This reverts commit e94ee9f. Reverted due to downstream issues, to be investigated,
…re size (apache#2808) - fix test cleanup; data files should be deleted after FileStore is closed
* OAK-12147 : introduce Oak cache API interfaces (OakCache, OakLoadingCache, OakCacheStats etc.) * OAK-12147 : convert OakCacheStats to a record * OAK-12147 : add javadocs to OakCacheStatsTest
8 tasks
bhabegger
force-pushed
the
oak-12089-multitarget
branch
from
7096523 to
f891100
Compare
Contributor
Author
6 tasks
…pache#2811) * OAK-12145 : add compatibility tests for Caffeine migration (PR 2807) Add implementation-independent test coverage for all classes affected by the Guava-to-Caffeine cache migration in OAK-11946. Tests reference only Oak-level types (CacheLIRS, CacheStats, DiffCache, etc.) so the same suite can be cherry-picked to OAK-11946 and run unchanged; any failure there is a migration compatibility gap. Modules covered: - oak-core-spi: AbstractCacheStats, CacheLIRS, EmpiricalWeigher - oak-store-document: DocumentNodeStoreBuilder, NodeDocumentCache, MemoryDiffCache, LocalDiffCache, TieredDiffCache, CachingCommitValueResolver, DocumentNodeStore, PersistentCache - oak-run-commons: DocumentNodeStoreHelper - oak-search: ExtractedTextCache (stats tracking) - oak-search-elastic: ElasticIndexStatistics (cache, refresh, failure) - oak-segment-tar: SegmentCache (loader failure contract) - oak-blob: BlobIdSet (cache miss / persistence semantics) - oak-blob-cloud: S3Backend (expiry, cache enable/disable) - oak-blob-cloud-azure: AzureBlobStoreBackend, AzureBlobStoreBackendV8 (expiry, cache enable/disable) * OAK-12145: Clarify cache compatibility test intent * OAK-12145: Rename S3 compatibility test * OAK-12145: Make cache compatibility tests behavior-based * OAK-12145: Fix SegmentCache compatibility assertion
bhabegger
force-pushed
the
oak-12089-multitarget
branch
2 times, most recently
from
1633d58 to
d9c9208
Compare
…ediate node is missing (apache#2902) * OAK-12213 Tree store: child node list maybe be incorrect if an intermediate node is missing * OAK-12213 Tree store: child node list maybe be incorrect if an intermediate node is missing * OAK-12213 Tree store: child node list maybe be incorrect if an intermediate node is missing * OAK-12213 Tree store: child node list maybe be incorrect if an intermediate node is missing
…ath to UserManager API (apache#2910) (apache#2911) * Added UserManager API to create users and groups with oak path * Added UserManagerDelegator implementation for createUser/GroupWithAbsolutePath * OAK-12190: Use JCR path in createUser/GroupWithAbsolutePath API, provide default implementations, and convert to Oak path in UserManagerImpl * OAK-12190: Extract getOakPath helper and remove duplication in createUser/GroupWithAbsolutePath * Fix on node name * OAK-12190: Create authorizable at exact absolute path; extract buildUser/buildGroup helpers; add unit tests * Fixes from Angela * updated Exception message * Fixes on analyse-pr report * OAK-12190: Add remapping tests for createUser/GroupWithAbsolutePath Verify that absolute JCR paths supplied to the two new methods are properly converted from the session-local namespace mapping to the internal Oak path representation. Ai-Assisted-By: claude --------- Co-authored-by: Nicola Scendoni <nscendoni@adobe.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…xed (apache#2912) * OAK-12221 Cost estimation should be lower if more conditions are indexed * OAK-12221 Cost estimation should be lower if more conditions are indexed * OAK-12221 Cost estimation should be lower if more conditions are indexed * OAK-12221 Cost estimation should be lower if more conditions are indexed * OAK-12221 Cost estimation should be lower if more conditions are indexed
This reverts commit 0f7edd5.
This reverts commit f16bc60.
This reverts commit 2456475.
This reverts commit d984208.
…pache#2922) * OAK-12227 Exclude Tika from oak-run * OAK-12227 Exclude Tika from oak-run * Apply suggestions from code review Co-authored-by: Fabrizio Fortino <fabrizio.fortino@gmail.com> * OAK-12227 Exclude Tika from oak-run --------- Co-authored-by: Fabrizio Fortino <fabrizio.fortino@gmail.com>
…ead pool (apache#2925) * OAK-12234 Elastic async response processing should use a separate thread pool * OAK-12234 (minor) remove useless assertion in test * OAK-12234 move from a per-connection executor to a global executor * OAK-12234 upper bound (32) got high-core-count machines
Introduces oak-search-luceneNg, a new Oak module providing a Lucene 9 based index engine under type=lucene9, with full parity to the legacy lucene implementation for property queries, fulltext, sorting, excerpts, and facets (insecure, statistical, and secure ACL modes). Key changes: - New oak-search-luceneNg module: index editor, query index, tracker, index node, storage, and OSGi wiring - Facet parity: LuceneNgSecure/StatisticalSortedSetDocValuesFacetCounts ported to Lucene 9 APIs with null-safe MatchingDocs.bits handling - LuceneNgFacetCommonTest extends FacetCommonTest for JCR-level coverage - AbstractIndexComparisonTest inlined into oak-search test-jar; oak-search-test module removed - getRootBuilder removed from ContextAwareCallback and IndexUpdate - leaf OSGi property removed from LuceneIndexProviderService - README documents feature parity vs legacy Lucene and Elastic Made-with: Cursor
When indexNodeName=true, the index editor writes the namespace-stripped local name of each node into FieldNames.NODE_NAME. The query engine maps LOCALNAME() equality and LIKE restrictions to TermQuery/WildcardQuery on that field. Function restrictions prefixed with "function*@" (e.g. "function*@:localname") are generated alongside the dedicated ":localname" restriction by Oak's SQL2 parser; they are now silently dropped from plan evaluation, cost calculation, and the Lucene query to prevent false negatives. Adds NodeNameCommonTest (shared) and LuceneNgNodeNameCommonTest. Made-with: Cursor
…ADME Address PR review comments from thomasmueller: - Rename "Multi-index queries" to "Composite node store queries" and add a footnote explaining the composite node store scenario. - Add a footnote for "Index augmentors" describing the IndexFieldProvider / FulltextQueryTermsProvider extension points. Made-with: Cursor
Follows Maven/Oak convention of lowercase hyphenated artifact names. The Java package (org.apache.jackrabbit.oak.plugins.index.luceneNg) is unchanged as it is an internal implementation detail. Made-with: Cursor
…etCommonTest The three tests (basic faceting, multiple dimensions, facet with filter) are all already exercised by FacetCommonTest via the JCR API. LuceneNgFacetCommonTest runs that suite against Lucene 9 and is the canonical coverage. The ignored class added no value. Made-with: Cursor
- Add multi-target routing helpers and active-target resolution - Add per-target catch-up indexing lifecycle - Simplify catch-up execution and scope graduation to owning lane - Align with marker-based catch-up provider contract Made-with: Cursor
bhabegger
force-pushed
the
oak-12089-multitarget
branch
from
6eaf573 to
32a2018
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
storeTargets/activeTarget) and integrate write/query target selectionTest plan
mvn -pl oak-core -Dtest=\"IndexDefinitionHelperTest,AsyncIndexUpdateCatchUpTest,CatchUpRunnerTest\" testNotes
This branch is functionally split from lucene9 core work, but because GitHub cross-repo PRs require a base branch present in the target repository, this PR currently targets
OAK-12089directly.Depends on #2817 and should be reviewed/merged after it.
Made with Cursor