doc: update generated developer documentation

doc/developer/generated/adapter-types/cluster_state.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter-types/src/cluster_state.rs
-revision: 74f18a3354
+revision: 6eeaca032b
 ---
 
 # adapter-types::cluster_state
@@ -11,15 +11,16 @@ These types carry the slices of a managed cluster's config that a caller reasons
 
 Key types:
 
-* `ExpectedClusterState` — compare-and-append witness over a managed cluster's durable config. A caller captures it from a config snapshot and pairs it with a conditional write. The applier applies the write only if the cluster's current config still projects to an equal witness. Fields: `size`, `replication_factor`, `availability_zones`, `logging`, `auto_scaling_policy`, `reconfiguration`, `burst`.
+* `ExpectedClusterState` — compare-and-append witness over a managed cluster's durable config. A caller captures it from a config snapshot and pairs it with a conditional write. The applier applies the write only if the cluster's current config still projects to an equal witness. Fields: `size`, `replication_factor`, `availability_zones`, `logging`, `arrangement_compression`, `schedule`, `auto_scaling_policy`, `reconfiguration`, `burst`.
 * `ReplicaShape` — the config dimensions that distinguish one replica from another. Two replicas with equal shape are interchangeable. `ReplicaShape::matches` compares availability zones as unordered pools so a reorder alone does not force a reprovision.
 * `AvailabilityZones` — the availability zones in configured provisioning order. Order is significant (the orchestrator round-robins placement across the list) and is part of the `ExpectedClusterState` witness. `AvailabilityZones::pool` converts to an unordered `AvailabilityZonePool` for interchangeability checks.
 * `AvailabilityZonePool` — unordered set of zones produced by `AvailabilityZones::pool`. Used for the one comparison that must ignore order: replica interchangeability.
 * `ReconfigurationRecord` — graceful reconfiguration record mirrored from durable state: `target`, `deadline`, `on_timeout`, `status`. `is_in_progress` returns true when `status` is `InProgress`.
 * `ReconfigurationStatus` — lifecycle status of a graceful reconfiguration: `InProgress`, `Finalized`, `TimedOut`, `Cancelled`, `ResourceExhausted`.
 * `ReconfigurationAudit` — the lifecycle transition a write to the `reconfiguration` record represents, declared by the writer at the decision point: `Started`, `Cancelled`, `Finalized { forced }`, `TimedOut`, `ResourceExhausted`.
 * `OnTimeout` — the action a graceful reconfiguration applies once its deadline passes with the target not yet hydrated: `Commit` (cut over anyway) or `Rollback` (revert to pre-reconfiguration shape).
-* `ReconfigurationTarget` — the full config shape a reconfiguration is moving to: `size`, `replication_factor`, `availability_zones`, `logging`. `shape()` returns the per-replica `ReplicaShape` (everything but `replication_factor`).
+* `ClusterSchedule` — the scheduling policy of a managed cluster: `Manual` (user-controlled replication factor) or `Refresh { hydration_time_estimate }` (cluster runs only around refresh times). A plain-data mirror of `mz_sql::plan::ClusterSchedule`.
+* `ReconfigurationTarget` — the full config shape a reconfiguration is moving to: `size`, `replication_factor`, `availability_zones`, `logging`, `arrangement_compression`. `shape()` returns the per-replica `ReplicaShape` (everything but `replication_factor`).
 * `BurstRecord` — active hydration-burst record: `burst_size`, `linger_duration`, `steady_hydrated_at`.
 * `BurstAudit` — lifecycle transition for a burst record: `Started` or `Finished { cause }`.
 * `BurstFinishCause` — why a hydration burst finished: `LingerElapsed` or `NoLongerWarranted`.

doc/developer/generated/adapter/catalog/_module.md

@@ -1,12 +1,13 @@
 ---
 source: src/adapter/src/catalog.rs
-revision: 605cc6fd1a
+revision: a60edac7f1
 ---
 
 # adapter::catalog
 
 The coordinator's catalog layer: wraps the `mz_catalog` crate's durable store and in-memory objects with the additional adapter-specific logic needed to drive DDL and serve catalog queries.
 `Catalog` is the top-level struct exposing the full catalog API: it holds a `CatalogState` (in-memory view), an `ExpressionCacheHandle`, a `transient_revision` counter, a `shared_transient_revision: Arc<AtomicU64>` shared across all clones, and delegates persistence to the durable store. `Catalog::transient_revision_is_current` compares the clone's frozen revision against the shared latest, letting a snapshot holder detect staleness from off-thread without a Coordinator round-trip.
+`CatalogUpperHandle` is a handle for advancing the durable catalog upper off the coordinator loop; its `advance_upper` method advances the upper to at least a given timestamp.
 Child modules divide responsibilities: `state` owns the in-memory data structures and `SessionCatalog` impl; `transact` executes atomic DDL operations; `apply` reconciles durable diffs into in-memory state; `open` bootstraps the catalog at startup; `migrate` rewrites stored SQL for syntax changes; `builtin_table_updates` maintains system-table row diffs; `consistency` provides invariant checking; `timeline` resolves timeline contexts for timestamp selection; and `cluster_state` projects live cluster config into plain-data `ExpectedClusterState` structs for conditional catalog writes.
 Re-exports include `InjectedAuditEvent` from `transact`. Builtin materialized views are handled alongside tables and views in descriptor tests.
 `is_reserved_role_name` returns true for names that match `is_reserved_name`, `is_public_role`, or the `RESERVED_ROLE_SPECIFICATION_NAMES` list. `RESERVED_ROLE_SPECIFICATION_NAMES` contains the five lowercase PostgreSQL role-specification keywords (`current_user`, `current_role`, `session_user`, `user`, `none`) that would be ambiguous in statements like `GRANT ... TO CURRENT_USER`; only the lowercase spellings are reserved, matching what unquoted identifiers normalize to.

doc/developer/generated/adapter/catalog/apply.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter/src/catalog/apply.rs
-revision: 584bb9030c
+revision: fca741734d
 ---
 
 # adapter::catalog::apply

doc/developer/generated/adapter/catalog/cluster_state.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter/src/catalog/cluster_state.rs
-revision: 74f18a3354
+revision: 6eeaca032b
 ---
 
 # adapter::catalog::cluster_state

doc/developer/generated/adapter/catalog/open.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter/src/catalog/open.rs
-revision: dbd2c3fc06
+revision: fca741734d
 ---
 
 # adapter::catalog::open

doc/developer/generated/adapter/catalog/open/_module.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter/src/catalog/open.rs
-revision: dbd2c3fc06
+revision: fca741734d
 ---
 
 # adapter::catalog::open

doc/developer/generated/adapter/catalog/transact.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter/src/catalog/transact.rs
-revision: 74f18a3354
+revision: 6eeaca032b
 ---
 
 # adapter::catalog::transact
@@ -10,13 +10,12 @@ The module enforces referential integrity, privilege checks, and naming constrai
 Zero-downtime deployment logic (0dt) is also managed here; DDL operations can be gated behind deployment-readiness checks during the cutover window.
 `InjectedAuditEvent` struct and `Op::InjectAuditEvents` variant allow manually appending audit log entries at the current oracle write timestamp. `CREATE ROLE` audit events use `EventDetails::CreateRoleV1` which includes the `auto_provision_source` field.
 `Op::AlterAddColumn` emits an `AlterAddColumnV1` audit log event recording the table ID, column name, column type, and nullability. `Op::AlterTimestampInterval` emits an `AlterSourceTimestampIntervalV1` audit log event recording the item ID, old interval, and new interval.
-`Op::WeirdStorageUsageUpdates` has been removed; storage usage id allocation is handled outside the catalog transaction via `Catalog::allocate_storage_usage_id`.
 `Op::UpdateScopedSystemParameters { scoped, prune_scope }` persists the durable cache of scoped (per-cluster and per-replica) system parameters. `scoped` is the desired sparse state; `prune_scope` is an optional `ScopedParametersScope` bounding which objects' rows may be removed. The handler diffs against the current durable contents: it upserts changed or added entries, removes entries for objects within the prune scope that are no longer present in the desired state (a `None` prune_scope means all objects are in scope), and lazily prunes entries whose owning object id is absent from the catalog (object ids are never reused, so such entries are inert).
 `Op::CreateClusterReplica` carries a required `replica_id: ReplicaId` field that must be pre-allocated out-of-band by the caller via the durable allocator (mirroring how cluster and item IDs are allocated). The apply path always uses `insert_cluster_replica_with_id` and never allocates a replica ID in-apply.
 `Op::CheckClusterState { cluster_id, expected }` is a precondition op, not a mutation: it aborts the whole transaction with `AdapterError::ClusterStateChanged` unless the cluster's current managed config equals `expected`. This gives a compare-and-append semantic over the cluster's config, inseparable from the commit it guards. It is a purely internal op and is never emitted by SQL DDL.
 `ReplicaCreateDropReason::Retired` is the audit-log reason emitted by the cluster controller when it drops a replica because the cluster's configuration no longer calls for it (e.g. a replication-factor decrease). It maps to `CreateOrDropClusterReplicaReasonV1::Retired` with no scheduling-decisions blob.
 `ReplicaCreateDropReason::HydrationBurst` is the audit-log reason emitted by the cluster controller's hydration-burst strategy when it creates the transient burst replica it runs while a cluster's objects are not yet hydrated. It maps to `CreateOrDropClusterReplicaReasonV1::HydrationBurst` with no scheduling-decisions blob.
-`Op::UpdatePrivilege` carries `privileges: Vec<MzAclItem>` (previously a single `privilege: MzAclItem`), allowing a bulk `GRANT`/`REVOKE` touching one object for many grantees to land as a single durable write while still emitting one audit event per grantee.
+`Op::UpdatePrivilege` carries `privileges: Vec<MzAclItem>`, allowing a bulk `GRANT`/`REVOKE` touching one object for many grantees to land as a single durable write while still emitting one audit event per grantee.
 When dropping a cluster replica, the code expands dependent objects (including materialized views that target the replica) using `state.cluster_replica_dependents(cluster_id, replica_id, &mut seen)`, records their comments for cleanup, and adds them to the drop list; `seen` is seeded from already-collected items so the plan-driven path (which processes dependents before the replica in reverse-dependency order) does not re-add them.
 When dropping items, all storage collections associated with a dropped entry are scheduled for removal unconditionally; whether the item was a replacement target is not considered at this stage.
 `Op::UpdateItem` enforces that a non-temporary item must not depend on a temporary one, mirroring the same invariant already enforced by `Op::CreateItem`. Temporary objects are session-scoped and never persisted; a durable item referencing one is a dangling reference that would panic the coordinator when its `create_sql` is re-planned on catalog apply (apply emits durable items before temporary ones). This prevents ALTER paths such as `ALTER SINK ... SET FROM` from repointing a persistent item at a temporary object.

doc/developer/generated/adapter/coord/_module.md

@@ -1,19 +1,21 @@
 ---
 source: src/adapter/src/coord.rs
-revision: 55e361b0a6
+revision: a60edac7f1
 ---
 
 # adapter::coord
 
 The core coordinator: `coord.rs` defines the `Coordinator` struct (the central state machine), `ExecuteContext` (the per-statement execution handle), the internal `Message` enum, and the `serve` / `Config` entry points.
 The coordinator owns the `Catalog`, active compute sinks, pending peeks, read-policy manager, timeline oracles, and all inter-subsystem handles (controller, storage collections, secrets, orchestrator). The `Coordinator` struct holds a `reconcile_now: Arc<Notify>` field that wakes the cluster controller task to reconcile immediately after catalog transactions that change durable cluster state, rather than waiting out the tick interval.
 The file also defines `IdPool`, a pre-allocated pool of user `GlobalId` integers that amortizes per-DDL persist writes by reserving batches of IDs at once; the pool is owned by the coordinator and access is serialized through its single-threaded event loop.
+`ArrangementSizeRecord` is a struct for rows destined for `mz_object_arrangement_size_history`, carrying `replica_id`, `object_id`, `size`, and `hydration_complete`; records are prepared off-thread by the arrangement sizes snapshot task and stamped with a collection timestamp on the coordinator loop at write time.
 Child modules partition the coordinator's responsibilities: `command_handler` handles external `Command` messages; `message_handler` handles internal async `Message` responses; `sequencer` executes SQL plans; `appends` manages table and builtin-table writes; `catalog_implications` derives and applies downstream effects from catalog state changes, including pushing replica-scoped dyncfg override layers to the compute controller via `push_replica_dyncfg_overrides`; `ddl` wraps catalog transactions; `peek` and `read_policy` manage query execution and compaction; `read_then_write` implements the read-then-write protocol for DML (INSERT, UPDATE, DELETE) including dependency validation; `group_sync` computes the diff between sync-managed and manually-granted role memberships for SSO group synchronization; `timestamp_selection` and `timeline` handle temporal reasoning; `catalog_serving` serves catalog snapshots; `info_metrics` owns the catalog `*_info` Prometheus series background task; `cluster_controller` scaffolds the cluster controller task that reconciles durable cluster state with the live controller; and supporting modules cover cluster scheduling, unified compute introspection subscribes, consistency checking, index management, and statement logging.
 The `Coordinator` struct holds a `scoped_frontend: Option<Arc<SystemParameterFrontend>>` field populated by `Command::InstallScopedSystemParameterFrontend`. `reconcile_scoped_system_parameters` persists the diff between the current durable working copy and the desired `ScopedParameters` via `Op::UpdateScopedSystemParameters`, which also updates the in-memory working copy and introspection relations. `scoped_overrides_create_op` evaluates scoped overrides for freshly-created clusters and replicas from explicit `ClusterEvalContext` / `ReplicaEvalContext` values and returns an `Op::UpdateScopedSystemParameters` to fold into the same transaction that creates them, so the committed diff drives the replica-scoped controller push before `create_replica`. `push_replica_dyncfg_overrides` propagates the replica-scoped overrides from the catalog working copy to the compute controller's per-replica dyncfg layer. `cluster_scoped_optimizer_overrides` retrieves the cluster-coherent `OptimizerFeatureOverrides` from the catalog working copy for use at plan time.
 The `Coordinator` struct holds a `catalog_info_metrics_registry: MetricsRegistry` field used to hand the metrics registry to the catalog info-metrics background task spawned at bootstrap.
 Bootstrap handles derived builtin storage collections (builtin MVs) separately: after registering input-less collections in dependency order, it bumps their sinces based on transitive dependency frontiers to satisfy as-of selection invariants. Bootstrap restores compaction policies for all MV version IDs (via `global_ids()`, not just the write ID) so that no version blocks capability propagation through the `primary` ownership chain and pins compaction. When applying replacement MVs, each `CollectionDescription` receives a `primary` field pointing to its predecessor's latest collection ID, chaining shard ownership from the oldest version through each replacement in order. Bootstrap also calls `push_replica_dyncfg_overrides` once before dataflows are rendered so that replica-scoped dyncfg flags take effect on the first configuration of each replica, and calls `spawn_cluster_controller_task` to start the cluster controller background task.
 The `clusters_caught_up_check` built during `serve` excludes both migrated MVs and new builtin MVs (and their transitive dependents) from the caught-up frontier check: migrated MV dataflows do not advance their write frontier in read-only mode, and new builtin MVs have no writer until the deployment promotes, so both sets would otherwise stall the check.
 The `Message` enum includes a `ClusterControllerRequest(cluster_controller::ClusterControllerRequest)` variant, carrying one pull/apply call from the cluster controller task to be answered on the main coordinator message loop.
+`Message::ArrangementSizesWrite(Vec<ArrangementSizeRecord>)` carries the records prepared by the off-thread arrangement sizes snapshot task back to the coordinator loop for stamping and appending to `mz_object_arrangement_size_history`.
 `ship_dataflow`, `try_ship_dataflow`, and `ship_dataflow_and_notice_builtin_table_updates` accept `DataflowDescription<LirRelationExpr>` (previously `DataflowDescription<Plan>`).
 `ExecuteContextInner` carries a `response_barriers: Vec<BuiltinTableAppendNotify>` field (Debug-ignored). `ExecuteContext::from_parts` is a convenience wrapper around `from_parts_with_response_barriers`, which accepts an explicit barrier list. `into_parts` returns the barrier list as a fifth element alongside `tx`, `internal_cmd_tx`, `session`, and `extra`; callers that repack an `ExecuteContext` must preserve and forward these barriers. `delay_response_until` appends a `BuiltinTableAppendCompletion` barrier; on retirement, the response is held until all barriers resolve.
 `ExplainTimestampStage` has four variants in order: `Optimize`, `RealTimeRecency`, `LinearizeTimestamp` (carries `ExplainTimestampLinearizeTimestamp`: validity, format, optimized_plan, cluster_id, source_ids, when, real_time_recency_ts), and `Finish` (carries `ExplainTimestampFinish`: validity, format, cluster_id, source_ids, when, real_time_recency_ts, timeline_context, oracle_read_ts). The `LinearizeTimestamp` stage reads the oracle timestamp off the coordinator loop and passes it forward as `oracle_read_ts` in `ExplainTimestampFinish`.

doc/developer/generated/adapter/coord/appends.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter/src/coord/appends.rs
-revision: 1c17d34993
+revision: a60edac7f1
 ---
 
 # adapter::coord::appends
@@ -10,3 +10,4 @@ Implements all append operations executed by the coordinator: group-commit of us
 `BuiltinTableAppendNotify` is a pinned future that resolves when a builtin-table write completes. `BuiltinTableAppendCompletion` is a completion handle wrapping a `BuiltinTableAppendNotify`; callers obtain one to use as a response barrier and convert it to a notify via `into_notify`. `GroupCommitWriteLocks` manages the locking primitives used to serialise group commits.
 `group_commit` always performs the append call even when there are no user writes, relying on the storage layer to periodically advance the upper of all tables; it does not iterate catalog entries to inject empty advancement entries per table.
 `UserWriteResponder` is an enum with a single `Session(PendingTxn)` variant that wraps the per-session `PendingTxn` for `PendingWriteTxn::User`; the `User` variant's `pending_txn` field is replaced by `responder: UserWriteResponder`. The write timestamp for `User` writes is picked by the oracle during group commit; the write lock is either handed off from the submitting session (`write_locks: Some(..)`) or acquired during group commit (`write_locks: None`).
+`GroupCommitter` serializes runtime txns-shard writes off the coordinator loop via `TableWriteCmd` variants: `GroupCommit`, `Register`, and `Forget`. `DeferredOp` represents an operation awaiting a resource; `DeferredOp::Plan` wraps a `DeferredPlan` that must uniquely hold write locks (e.g. UPDATE), while `DeferredOp::Write` wraps a `DeferredWrite` for blind inserts that can be optimistically retried. `GroupCommitRequest` accumulates appends, responses, statement-logging IDs, notifies, write locks, and permits for a single commit batch.

doc/developer/generated/adapter/coord/catalog_implications.md

@@ -1,6 +1,6 @@
 ---
 source: src/adapter/src/coord/catalog_implications.rs
-revision: 1c17d34993
+revision: a60edac7f1
 ---
 
 # adapter::coord::catalog_implications