Persistence of Memory

A pure C (though C++ compatible) allocator that works with mmap allowing you to retain data semi-transparently between process runs. This allows you to "run" within a persistent space without the typical overheads of file I/O, and for your memory structures to be recovered, either to continue running or to "archeologically" retrieve data from before a crash (depending on your needs and architecture).

Features

• thread safe && lock free - no blocking, no spin locks, just pure atomics
• malloc, free and realloc implementations
• archeology / forensic capabilities - rewrite pointers from previous memory spaces using persistent_ptr
• identifiable / tagged allocations - persistent_malloc_root allows "root" allocations to be tagged for later retrieval (between process runs)
• some C++ functionality
  • PersistentPtr header-only smart/fancy pointer which automatically keeps track of its allocator_space between address changes (restarts)
  • PersistentSpace header-only offset pointer & wrapper to an allocator_space, providing template alloc/free that works across restarts
  • RebasePtr header-only raw-pointer wrapper for the read-out-and-discard forensic path: a plain pointer during normal operation, but transparently read-through rebased while a RebaseScope is active on the thread

Limitations

• Fixed size - the allocator currently works strictly within a single fixed region and will not attempt to acquire more memory
• No calloc equivalent - if you want clean memory, use memset after persistent_malloc
• PersistentPtr cannot be made atomic
• No C++ STL Allocator

Realloc

The persistent_realloc function does not have same behaviour as realloc:

#define persistent_realloc(space, ptr, new_size) \
    persistent_realloc_impl((space), (void **)&(ptr), (new_size))
bool persistent_realloc_impl(allocator_space_t *space, void **ptr, size_t new_size);

persistent_realloc will update your pointer in-place if a relocation was required, and will return true or false to signal whether the reallocation was a success or failure. This behavior is significantly less bug-prone that the normal realloc footgun (returning NULL on failure, without releasing the memory).

The typical use pattern is simplified to:

char *big_string = persistent_malloc(space, expected_maximum_size);
ssize_t actual_size = strtcpy(big_string, unknown_original_size, expected_maximum_size);

if (actual_size == -1) actual_size = expected_maximum_size;
if (!persistent_realloc(space, big_string, actual_size)) {
  goto failure;
}

Pointer considerations

Care needs to be taken when using raw pointers within the space, persistent_ptr can be used to archeologically recover raw pointers from a previous run, but you should treat these spaces as read only. persistent_ptr works by adjusting the pointer based on the "old" and "new" memory address of the persistent space (new_raw_ptr = old_raw_ptr - &old_raw + &new_space) but since there is only a single slot for the origin to be stored we can only adjust from a single "old" space into our "new" space. For more flexibility C code can make use of the persistent_offset_t typealias along with the ptr_to_persistent_offset and persistent_persistent_offset_to_ptr macros to convert to and from raw pointers.

For C++ code there is a persistent_mem::PersistentPtr template class which can be treated like other fancy pointers, but remain internally consistent if they are properly initialised (they must be passed a allocator_space_t* at least once).

RebasePtr — read-through recovery

For the specific "resurrect a crashed space, read it out (to protobuf/JSON/…), unmap and delete" workflow, persistent_mem::RebasePtr<T> is a raw-pointer wrapper that stores an ordinary pointer (no offset, sizeof(void*)). During normal operation it is a raw pointer. When a RebaseScope (a thread-local RAII guard) is active, every dereference is instead routed through persistent_ptr — read-through, so the stored bytes are never modified and a torn pointer rebases to nullptr rather than crashing the reader. Nested RebasePtrs rebase structurally as you walk into them, so an entire object graph becomes traversable by opening a single scope, with no per-type restore() code:

allocator_space_t *space = load_private_persistent_allocator("crashed.bin");
Root *root = (Root *)persistent_find_root(space, ROOT_CLASS); // live re-entry handle
{
    persistent_mem::RebaseScope scope(space); // rebasing active on this thread
    for (persistent_mem::RebasePtr<Node> n = root->head; n; n = n->next) {
        serialise(n->value); // every deref rebased read-through
    }
} // scope closed; stored bytes untouched
destroy_persistent_allocator(space);

RebasePtr only ever wraps pointers stored within the space (always original-mapping-relative); the live handle returned by persistent_find_root is used directly and must not be wrapped. Contrast with PersistentPtr, which is always-valid but costs two offsets, and with the hand-written restore() / write-back approach, which is the right tool when you want to correct pointers in place and keep running.