docs(topic): document topic-level offloadPolicies and unlimited backlog#409
Merged
Conversation
The PulsarTopic CRD has supported topic-level offloadPolicies and backlogQuotaLimitSize since the OffloadPolicies type was added, and the reconciler wires them through SetOffloadPolicies / SetBacklogQuota. The docs only listed offloadPolicies in the field table with no subfield documentation, so users had no way to discover that aggressive immediate offload (offloadThresholdBytes=0 / offloadDeletionLagSeconds=60) and unlimited backlog (backlogQuotaLimitBytes=-1) are already expressible declaratively per topic. This change documents the gap without touching code: - Add a dedicated offloadPolicies subsection covering all 14 fields, with an example matching the common "aggressive immediate offload" use case and a mapping note from pulsar-admin flag names to CR fields (including the millis-vs-seconds gotcha for the deletion lag). - Document that backlogQuotaLimitSize: "-1" allows an unlimited backlog without producer throttling. - Fix the persistent default in the field table (true, not false) and clarify that the topic-domain prefix in name takes precedence. - Refresh the modifiable-fields list in "Update A Pulsar Topic" to include offloadPolicies, the subscribe/replicator dispatch rates, the per-subscription/per-topic limits, deduplication settings, schema controls, and properties; note that clearing a field reverts to the namespace-level default. - Extend the comprehensive example with an offloadPolicies block. - Drop the duplicated/typo "## Sepcifications" heading. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This docs-only PR makes existing PulsarTopic support for topic-level offload policies and unlimited backlog quota discoverable, while correcting related field documentation.
Changes:
- Documents
offloadPoliciesfields, examples, andpulsar-adminfield mappings. - Clarifies
backlogQuotaLimitSize: "-1"and corrects thepersistentdefault totrue. - Refreshes the topic update guidance and comprehensive example.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
maxsxu
approved these changes
May 29, 2026
freeznet
added a commit
that referenced
this pull request
Jun 2, 2026
…og (#409) The PulsarTopic CRD has supported topic-level offloadPolicies and backlogQuotaLimitSize since the OffloadPolicies type was added, and the reconciler wires them through SetOffloadPolicies / SetBacklogQuota. The docs only listed offloadPolicies in the field table with no subfield documentation, so users had no way to discover that aggressive immediate offload (offloadThresholdBytes=0 / offloadDeletionLagSeconds=60) and unlimited backlog (backlogQuotaLimitBytes=-1) are already expressible declaratively per topic. This change documents the gap without touching code: - Add a dedicated offloadPolicies subsection covering all 14 fields, with an example matching the common "aggressive immediate offload" use case and a mapping note from pulsar-admin flag names to CR fields (including the millis-vs-seconds gotcha for the deletion lag). - Document that backlogQuotaLimitSize: "-1" allows an unlimited backlog without producer throttling. - Fix the persistent default in the field table (true, not false) and clarify that the topic-domain prefix in name takes precedence. - Refresh the modifiable-fields list in "Update A Pulsar Topic" to include offloadPolicies, the subscribe/replicator dispatch rates, the per-subscription/per-topic limits, deduplication settings, schema controls, and properties; note that clearing a field reverts to the namespace-level default. - Extend the comprehensive example with an offloadPolicies block. - Drop the duplicated/typo "## Sepcifications" heading. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
A user asked whether the Pulsar Resources Operator supports declarative topic-level offload (
offloadThresholdBytes,offloadDeletionLagSeconds) and an unlimited backlog quota (backlogQuotaLimitBytes: -1) via thePulsarTopicCRD. The code already supports all three (viaOffloadPoliciesandbacklogQuotaLimitSize, wired throughSetOffloadPolicies/SetBacklogQuotainpkg/admin/impl.goandpkg/connection/reconcile_topic.go), but the docs only listedoffloadPoliciesin the field table with no subfield reference or example — so the feature was effectively undiscoverable.While auditing the same doc against the API types, a few other small inconsistencies surfaced (wrong default for
persistent, missing-1semantics forbacklogQuotaLimitSize, stale modifiable-fields list, a duplicated/typo heading). Folded those in.Modifications
docs/pulsar_topic.mdonly — no code or API changes.offloadPoliciessubsection under "Topic-Level Policies": a 14-field table mirroring theOffloadPoliciesGo type, plus a worked example matching the common "aggressive immediate offload" use case (managedLedgerOffloadThresholdInBytes: 0+managedLedgerOffloadDeletionLagInMillis: 60000+backlogQuotaLimitSize: "-1"), plus a mapping from familiarpulsar-adminflag names to CR fields. Calls out the millis-vs-seconds gotcha for the deletion lag.backlogQuotaLimitSizefield row now documents"-1"as "unlimited backlog (no producer throttling)".persistentdefault corrected:false→true(matches+kubebuilder:default=trueonPulsarTopicSpec.Persistent). Also clarified that the topic-domain prefix innametakes precedence.offloadPolicies,backlogQuotaType,subscribeRate,subscriptionDispatchRate,replicatorDispatchRate,maxMessageSize,maxConsumersPerSubscription,maxSubscriptionsPerTopic,maxUnAckedMessagesPerConsumer/Subscription,deduplication,deduplicationSnapshotInterval,autoSubscriptionCreation,schemaValidationEnforced,schemaCompatibilityStrategy, andproperties. Added a note that clearing an optional field reverts to the namespace-level default.offloadPoliciesblock and a corresponding bullet in the feature list.## Sepcificationsheading.Verifying this change
This change is a trivial rework / code cleanup without any test coverage. Docs-only change — the operator behavior was already covered by existing reconciler tests; no code path is modified.
Documentation
doc🤖 Generated with Claude Code