Add kamepoolalloc (kp) allocator#265
Open
northriv wants to merge 8 commits into
Open
Conversation
kamepoolalloc is a lock-free four-tier pool allocator (1 B .. multi-GiB: buckets / dedicated chunks / large mmap / huge) with a per-thread two-level recycle cache for the 32 KiB .. 32 MiB range. Dual-licensed Apache-2.0 OR GPL-2.0+. Production allocator of the KAME instrument-control framework since 2008; the chunk-claim / recycle / orphan-chain protocols are TLA+ and GenMC (RC11) model-checked. https://github.com/northriv/kamepoolalloc Pinned to v1.0.1 (banner gating + Linux malloc_usable_size co-interpose). Full local-suite soak passed on glibc/x86-64 (incl. redis, sh6/sh8bench).
So CI exercises kamepoolalloc on ubuntu/alpine/fedora (the bench.sh alloc_all registration alone is not enough — the GitHub Actions matrix is the hardcoded driver). glibc (ubuntu/fedora) is the soaked target; musl (alpine) is unverified — if it fails there, a follow-up alpine exclude with the actual error will mirror the existing per-allocator excludes (dh/fg/gd/...).
…lude fixes)
Temporary commit-SHA pin (reverts to a v1.0.2 tag once CI is green).
495c2485 on northriv/kamepoolalloc master carries two distro fixes
surfaced by this PR's CI:
- alpine/kp (musl) 6 h hang: strong-symbol free/malloc recursed into
themselves (no __libc_* on musl) -> dlsym(RTLD_NEXT) real-libc bypass.
- fedora/kp compile error 'atoi not declared': missing <cstdlib>
(libstdc++ 14+ dropped the transitive include) -> add the includes.
ubuntu/kp already passed. Verifies alpine/kp completes (no hang) and
fedora/kp compiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… <cstdlib>) No-op commit to fire all.yml on the fork now that Actions are enabled (all.yml is on: push; the previous version_kp pin push predated enablement). Squash before final PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
88a3007a on northriv/kamepoolalloc master adds Linux strong-symbol interception of memalign/posix_memalign/aligned_alloc, routing them through the pool's aligned path. Fixes the alpine/musl rptest+rocksdb SIGSEGV (musl implements those on top of the public malloc, so without interception they returned pool pointers wrapped by libc bookkeeping). Still a verification SHA pin; -> v1.0.2 tag once alpine/kp is green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
35177d74 adds the CrossDeallocBatch::flush at-teardown skip of the owner force-walk poke — fixes the alpine/musl rocksdb db_bench SIGSEGV at thread-pool join (musl __pthread_tsd_run_dtors frees the owner TLS before the force_walk pointer is nullified). Combined with the earlier aligned-family (rptest) and reallocarray/recursion fixes, this should take alpine/kp to signal-11=0. Still a verification SHA pin; -> v1.0.2 once alpine/kp is confirmed crash-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the verification SHA pin with the released tag v1.0.2 on northriv/kamepoolalloc (== 35177d74). v1.0.2 adds full musl/Alpine support + Fedora-GCC build over v1.0.1. mimalloc-bench fork CI: all three distros (ubuntu/fedora/alpine) kp jobs are signal-11 = 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add kamepoolalloc (kp) — a lock-free four-tier pool allocator (1 B .. multi-GiB: buckets / dedicated chunks / large mmap / huge) with a per-thread two-level recycle cache for the 32 KiB .. 32 MiB range. Dual-licensed Apache-2.0 OR GPL-2.0+. Production allocator of the KAME instrument-control framework since 2008; chunk-claim / recycle / orphan-chain protocols are TLA+ and GenMC (RC11) model-checked.
https://github.com/northriv/kamepoolalloc
Pinned to v1.0.2 (adds musl/Alpine support + Fedora-GCC build over v1.0.1; inherits the dylib banner gating + Linux
malloc_usable_sizeco-interpose + word-cache default-on). Three-hunk patch only (platform.yml matrix + bench.sh + build-bench-env.sh); no README/result edits.CI status —
kppasses on all three distros. The ubuntu / fedora / alpinekpjobs all run the full suite with zero crashes (grep -c 'signal 11'= 0 in each job log; note mimalloc-bench keeps a job green even if an individual bench crashes, so the log grep is the real check). v1.0.2 specifically fixed two musl-only issues that this CI surfaced — aligned-family interception (memalign/posix_memalign/aligned_alloc, forrptest) and a TSD-teardown dangling-pointer skip in the cross-dealloc batch (forrocksdb's thread-pool join under musl__pthread_tsd_run_dtors) — plus a Fedora-GCC<cstdlib>include.Note on the other red jobs: the failing jobs in the run (
je,sn,rp,tcg,mesh,nomesh,hmon fedora;tcgon ubuntu;rpon alpine) are pre-existing upstream failures — the same set fails onmaster(e.g. the weekly scheduled runs), unrelated to this PR. This change touches no allocator other than kp and no platform/Docker config, so it neither introduces nor fixes those.