design: durable temporary objects (SQL-150)#37878
Draft
SangJunBak wants to merge 7 commits into
Draft
Conversation
Design doc for making temporary tables and views durable in the catalog shard, so that mz_tables and mz_views can become materialized views over mz_internal.mz_catalog_raw and temporary state survives in a multi-envd world. Records why sessions stay in the mz_sessions builtin table instead of becoming durable catalog records (connect latency coupling to the catalog compare-and-append, churn contention with DDL on the catalog single writer), and distills that lesson into the adapter guide.
Mechanical version bump: objects_v91.rs is a byte-for-byte copy of objects.rs, the v90 to v91 migration is a no-op, and the v91 encoding snapshot is generated. No type or data changes. The next change adds the ephemeral_owner_session field on top, so its review diff shows the field edits instead of a whole-file snapshot add.
…L-150) ItemValue and SchemaValue gain ephemeral_owner_session: Option<Uuid>, on top of the catalog version bump in the previous change. None means a normal durable object. Some(uuid) marks a temporary object owned by the session with that UUID, visible only to it. Existing records are backfilled as None by the v90 to v91 migration, since temporary objects were never written to the catalog before this field existed. Durable item name uniqueness is scoped by the owning session, since every session may reuse the same temporary item names. Opening the catalog with write intent fences out every previous owner, so it reclaims all ephemeral items via the new remove_ephemeral_items, which covers crashes and same-generation restarts. No behavior change: nothing writes Some(..) yet, callers pass None.
mz_sessions gains a deploy_generation column recording the envd incarnation serving the session, from the durable catalog fence token. Only one envd serves an environment at a time, fenced by the deploy generation, so rows whose generation is not the current one belong to a dead envd. Today stale rows are removed by the boot-time blanket reset of system tables. With multiple envds per environment (SQL-118) that blanket reset must become generation-scoped, and this column is the first piece of that. The column addition changes the catalog fingerprint, so the table gets a builtin schema migration replacement step.
Temporary items now write real durable Item rows, marked with ephemeral_owner_session = the creating session's UUID and parented to the temporary schema sentinel id (u0). The in-memory TemporaryItem side-channel is deleted: temp items flow through the normal durable update pipeline. On apply, an ephemeral item is routed into the owning connection's in-memory temporary schema, resolved through a session-UUID-to- connection mapping that the coordinator registers at connect and removes at terminate, strictly after the transaction dropping the session's temporary items has been applied. Applying an ephemeral item whose owner is not a session served by this process is a no-op, which covers following the catalog read-only during zero-downtime deployments, where the leader's temporary items arrive but their sessions live elsewhere. Session close drops the session's temporary items, dependents included, in one catalog transaction, before the mz_sessions retraction is queued, so a crash in between leaves a session row without items rather than orphaned items. Opening the catalog with write intent reclaims all ephemeral items (added previously), covering crashes and kill -9. Orphaned temp-table shards are finalized by the existing storage metadata reconciliation. BootstrapStateUpdateKind is deleted: it existed only to exclude the TemporaryItem variant, and memory StateUpdateKind now serves both uses. Verified manually: temp create/insert/query/cascade-drop, same temp table name in two concurrent sessions, graceful-close cleanup, kill -9 + restart reclamation, DISCARD ALL, temporary_objects.slt.
With temporary items durable in the catalog shard, mz_tables and mz_views can be derived from mz_internal.mz_catalog_raw instead of being written by the coordinator. Both show every item including temporary ones, matching the previous builtin tables. Temporary rows keep the temporary schema sentinel "0" in schema_id. Builtin tables and views are reported through two new generated constant views, mz_internal.mz_builtin_tables and mz_internal.mz_builtin_views, following the mz_builtin_materialized_views pattern. mz_builtin_views lists every builtin view except itself, since its definition cannot contain itself, so that one (new) view is absent from mz_views and relations derived from it. The declared RelationDesc keys of the generated views must exactly match the keys the optimizer derives from their VALUES lists (verify_builtin_descs enforces this), so both declare the keys that hold for the current builtin sets, with a NOTE on what to do when a future builtin breaks one. parse_catalog_create_sql now exposes 'definition' for views and 'source_id' for tables created from sources. The old mz_tables and mz_views shards are released through builtin schema migration replacement steps and finalized by the storage layer, like the other builtin table conversions. Verified manually: boot, user/temp tables and views appear with correct schema ids and definitions, pg_tables/pg_views joins, builtin arms populated. SLT: information_schema_tables, oid, temporary_objects, autogenerated/mz_internal.
Implementation log for the SQL-150 work: exploration facts, decisions, stage progress, and the pivot plan that moved sessions back out of the durable catalog. Not needed to understand the code changes. Drop this change before merging if the notes should not land in main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design doc for making temporary tables and views durable in the catalog
shard, so that mz_tables and mz_views can become materialized views over
mz_internal.mz_catalog_raw and temporary state survives in a multi-envd
world. Records why sessions stay in the mz_sessions builtin table
instead of becoming durable catalog records (connect latency coupling to
the catalog compare-and-append, churn contention with DDL on the
catalog single writer), and distills that lesson into the adapter guide.Remove these sections if your commit already has a good description!
Motivation
Why does this change exist? Link to a GitHub issue, design doc, Slack
thread, or explain the problem in a sentence or two. A reviewer who has
no context should understand why after reading this section.
If this implements or addresses an existing issue, it's enough to link to that:
Closes
Fixes
etc.
Description
What does this PR actually do? Focus on the approach and any non-obvious
decisions. The diff shows the code --- use this space to explain what the
diff can't tell a reviewer.
Verification
How do you know this change is correct? Describe new or existing automated
tests, or manual steps you took.