fix(restorer): scope end-of-WAL markers by WAL timeline#277
Open
ardentperf wants to merge 2 commits into
Open
Conversation
Add timeline-aware end-of-WAL marker handling to `pkg/restorer.WALRestorer`. The previous global `end-of-wal-stream` marker could be set by a prefetch miss on one timeline and consumed by a later restore request on another timeline. In CNPG restore flows, that can make PostgreSQL skip a valid child-timeline WAL lookup and replay wrong parent-timeline bytes. Signed-off-by: Jeremy Schneider <schneider@ardentperf.com>
ardentperf
force-pushed
the
pr-fix-end-of-wal
branch
from
56471f8 to
1953fc7
Compare
ardentperf
added a commit
to ardentperf/cloudnative-pg
that referenced
this pull request
Jul 6, 2026
Update in-tree `wal-restore` to use the new `barman-cloud` restorer APIs, resolving a bug that can cause replicas to enter a broken state that CNPG cant recover from. Local marker naming, WAL timeline parsing, and restore-result filtering policy now lives in `github.com/cloudnative-pg/barman-cloud/pkg/restorer`. depends on cloudnative-pg/barman-cloud#277 Signed-off-by: Jeremy Schneider <schneider@ardentperf.com>
Route the legacy marker cleanup in ConsumeEndOfWALStreamForWAL through the private marker helpers instead of the deprecated exported methods, so the deprecated API can later be removed without touching this path. timelineFromWALName can only fail with ErrInvalidWALName, so collapse the callers' two-step error handling into a single check and document that contract on the function itself. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
armru
added a commit
to ardentperf/cloudnative-pg
that referenced
this pull request
Jul 8, 2026
…e-pg#277 Point barman-cloud at the fork branch behind cloudnative-pg/barman-cloud#277 so this PR builds and CI can run, and allow the replace directive in the linter configuration. Both changes must be reverted in favor of an upstream barman-cloud release when merging. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
armru
added a commit
to ardentperf/plugin-barman-cloud
that referenced
this pull request
Jul 8, 2026
…e-pg#277 Point barman-cloud at the fork branch behind cloudnative-pg/barman-cloud#277 so this PR builds and CI can run, and allow the replace directive in the linter configuration. Both changes must be reverted in favor of an upstream barman-cloud release when merging. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
armru
approved these changes
Jul 8, 2026
armru
pushed a commit
to ardentperf/cloudnative-pg
that referenced
this pull request
Jul 8, 2026
Update in-tree `wal-restore` to use the new `barman-cloud` restorer APIs, resolving a bug that can cause replicas to enter a broken state that CNPG cant recover from. Local marker naming, WAL timeline parsing, and restore-result filtering policy now lives in `github.com/cloudnative-pg/barman-cloud/pkg/restorer`. depends on cloudnative-pg/barman-cloud#277 Signed-off-by: Jeremy Schneider <schneider@ardentperf.com>
armru
added a commit
to ardentperf/cloudnative-pg
that referenced
this pull request
Jul 8, 2026
…e-pg#277 Point barman-cloud at the fork branch behind cloudnative-pg/barman-cloud#277 so this PR builds and CI can run, and allow the replace directive in the linter configuration. Both changes must be reverted in favor of an upstream barman-cloud release when merging. Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add timeline-aware end-of-WAL marker handling to
pkg/restorer.WALRestorer. The previous globalend-of-wal-streammarker could be set by a prefetch miss on one timeline and consumed by a later restore request on another timeline. In CNPG restore flows, that can make PostgreSQL skip a valid child-timeline WAL lookup and replay wrong parent-timeline bytes.The legacy global APIs remain as compatibility shims, but new consumers should use the WAL/result-oriented APIs.
WALRestorernow owns the full policy:Tests included. In addition to local tests, I also verified with a full E2E test https://github.com/ardentperf/bug-repro-wal-restore
Fixes #276