diff --git a/docs/content/architecture/_meta.yml b/docs/content/architecture/_meta.yml new file mode 100644 index 00000000..35dee8bd --- /dev/null +++ b/docs/content/architecture/_meta.yml @@ -0,0 +1,3 @@ +label: Architecture +order: 8 +collapsed: true diff --git a/docs/content/architecture/crates.mdx b/docs/content/architecture/crates.mdx new file mode 100644 index 00000000..bbd78396 --- /dev/null +++ b/docs/content/architecture/crates.mdx @@ -0,0 +1,73 @@ +--- +title: Crates & Modules +sidebar: + order: 1 +--- + +Dolos is a Cargo workspace. The code is split so that generic data-node abstractions, Cardano-specific ledger logic, storage engines, and API servers each live in their own crate and depend on one another in a single direction. This page maps the crates and shows how the main `dolos` binary is organized. + +## Workspace crates + +| Crate | Path | Purpose | +| ----- | ---- | ------- | +| `dolos-core` | `crates/core/` | Chain-agnostic traits and types: `Domain`, `ChainLogic`, `WorkUnit`, and the `StateStore` / `ArchiveStore` / `WalStore` / `IndexStore` / `MempoolStore` storage traits. Knows nothing about Cardano. | +| `dolos-cardano` | `crates/cardano/` | The Cardano implementation of the core traits: block processing, ledger state, era rules, epoch transitions, and reward calculations. | +| `dolos-redb3` | `crates/redb3/` | Storage backend built on [redb](https://github.com/cberner/redb) (embedded, ACID, B+tree). Implements WAL, state, archive, indexes, and mempool. | +| `dolos-fjall` | `crates/fjall/` | Storage backend built on [fjall](https://github.com/fjall-rs/fjall) (LSM-tree), an alternative for write-heavy state and index stores. | +| `dolos-minibf` | `crates/minibf/` | Blockfrost-compatible HTTP API server. | +| `dolos-minikupo` | `crates/minikupo/` | Kupo-compatible HTTP API for datum/script resolution and pattern matching. | +| `dolos-trp` | `crates/trp/` | Transaction Resolver Protocol (Tx3) JSON-RPC server. | +| `dolos-testing` | `crates/testing/` | Shared test utilities and fixtures. | +| `xtask` | `xtask/` | Developer automation (`cargo xtask …`): test instances, ground-truth generation, DBSync queries. | +| `dolos` (root) | `src/` | The binary crate: CLI, sync pipeline, relay, serve orchestration, and the runtime adapters that wire everything together. | + +The API-server crates and several subsystems are gated behind Cargo features. The default build enables `mithril`, `utils`, `grpc`, `minibf`, `minikupo`, and `trp`, so a standard `dolos` binary ships with every API surface. `dolos-trp` is an optional feature-gated dependency rather than a workspace member. + +## Dependency layering + +Dependencies point in one direction — from concrete implementations toward the abstract core. Nothing in `dolos-core` depends on a storage engine or an API server. + +```mermaid +flowchart BT + core["dolos-core
traits: Domain · ChainLogic · WorkUnit · stores"] + cardano["dolos-cardano
ChainLogic impl"] + backends["dolos-redb3 / dolos-fjall
StateStore/… impls"] + services["dolos-minibf / minikupo / trp
serving adapters over Domain"] + bin["dolos
root binary: CLI · sync · relay · serve · adapters"] + + cardano --> core + backends --> core + services --> core + bin --> cardano + bin --> backends + bin --> services +``` + +The root binary is the only place that picks concrete backends and API servers and assembles them into a running node. + +## The `dolos` binary + +The binary lives in `src/bin/dolos/` and dispatches a set of top-level subcommands: + +| Command | What it does | +| ------- | ------------ | +| `init` | Interactive wizard to generate configuration. | +| `daemon` | Full node — runs the sync pipeline and all enabled API servers together. | +| `sync` | Sync only — pull blocks from the upstream peer and apply them to the ledger. | +| `serve` | Serve only — expose the APIs over already-synced data. | +| `data` | Inspect and export stored data (dump state, logs, blocks, WAL; prune; import archive). | +| `eval` | Evaluate a transaction against current ledger state (phase-1 and phase-2). | +| `doctor` | Diagnose and repair (WAL integrity, rollback, reset genesis/WAL, catch-up stores). | +| `bootstrap` | Initialize a node from a Mithril snapshot, a Dolos snapshot, or a relay. | + +The `daemon`, `sync`, and `serve` commands map directly onto the sync pipeline and serving layer described in the following pages. For how to choose between them operationally, see [Operations → Modes](../operations/modes); for `bootstrap`, see the [Bootstrap section](../bootstrap). + +## Key dependencies + +Dolos leans on a small number of foundational crates: + +- **[Pallas](https://github.com/txpipe/pallas)** — Cardano building blocks: CBOR codecs, ledger primitives, the Ouroboros mini-protocols, and phase-2 (Plutus) evaluation. +- **[gasket](https://github.com/construkts/gasket-rs)** — the staged, concurrent pipeline framework used by the sync loop. +- **redb** and **fjall** — the embedded storage engines behind the two storage backends. +- **tokio** — the async runtime shared by the pipeline and every API server. +- **tonic** (gRPC), **axum** (Mini-Blockfrost, Mini-Kupo), and **jsonrpsee** (TRP) — the serving frameworks. diff --git a/docs/content/architecture/data-layer.mdx b/docs/content/architecture/data-layer.mdx new file mode 100644 index 00000000..f1e0c299 --- /dev/null +++ b/docs/content/architecture/data-layer.mdx @@ -0,0 +1,62 @@ +--- +title: Data Layer +sidebar: + order: 2 +--- + +Dolos keeps everything it needs on local disk in a handful of embedded stores — key-value engines for live data, plus append-only flatfiles for archived block bodies. Rather than one monolithic database, it uses several purpose-built stores — each optimized for a different access pattern — behind a common set of traits, so the underlying engine can be swapped without touching the rest of the system. + +## The stores + +Under the configured `storage.path`, a running node maintains four on-disk stores plus an in-memory (or persistent) mempool: + +``` +/ +├── wal/ Write-Ahead Log — recent blocks as reversible deltas (rollback buffer) +├── state/ Ledger state — current UTxO set, pools, accounts, epoch state, protocol params +├── archive/ Historical blocks — full block bodies + temporal entity logs +└── index/ Secondary indexes — reverse lookups (address → UTxO, asset → UTxO, …) +``` + +| Store | Holds | Why it's separate | +| ----- | ----- | ----------------- | +| **WAL** | The most recent blocks, stored as `EntityDelta`s that can be applied or undone. | Rollbacks (chain reorganizations) are resolved by replaying these deltas in reverse. It is a bounded buffer, not long-term history. | +| **State** | The *current* ledger: UTxO set, stake pool registry, account/stake state, DReps, protocol parameters, and per-epoch ledger state. | This is the hot, always-current view the ledger rules read and write on every block. | +| **Archive** | Immutable history: full block bodies plus time-indexed logs of entity changes. | History is append-only and read differently from current state, so it uses its own layout (see below). | +| **Index** | Reverse lookups over UTxOs by address, payment credential, stake credential, policy, and asset. | The APIs need "find all UTxOs for X" queries that the primary state store isn't keyed for. | +| **Mempool** | Submitted-but-unconfirmed transactions and their lifecycle state. | Pending transactions are transient and overlaid on top of committed state during validation. | + +## Storage traits and pluggable backends + +Each store is defined as a trait in `dolos-core` — `WalStore`, `StateStore`, `ArchiveStore`, `IndexStore`, and `MempoolStore` — and the rest of Dolos only ever talks to those traits. Concrete implementations are selected at runtime in `src/adapters/storage.rs`, which wraps each backend in an enum so a node can mix engines per store. + +Available backends: + +- **redb** (`dolos-redb3`) — an embedded ACID B+tree store. It backs the WAL (the only WAL implementation), and can back state, archive, and indexes. +- **fjall** (`dolos-fjall`) — an LSM-tree engine tuned for write-heavy workloads with many hot keys, available for the state and index stores. +- **no-op** — a store that silently discards writes, used to *disable* the archive and/or index stores. +- **in-memory** — non-persistent variants used for testing and ephemeral nodes. + +## Storage modes + +The three storage modes exposed in configuration are really combinations of backend selection and history-pruning limits: + +| Mode | Archive / Index | History | Use case | +| ---- | --------------- | ------- | -------- | +| **Ledger-only** | no-op | none | Only the tip of the chain — smallest footprint, query-light deployments. | +| **Sliding history** | enabled, pruned | a rolling window (`sync.max_history`) | Recent history for most dApp queries without a full archive. | +| **Full archive** | enabled, unpruned | complete | The entire chain history — research, validation, explorers. | + +Pruning is driven by `prune_history()` on the WAL and archive stores. See the [configuration schema](../configuration/schema#storage-section) for the exact keys. + +## Data model primitives + +A few concepts recur across the stores: + +- **Namespaced entities.** State and archive data are organized into namespaces (UTxOs, pools, accounts, rewards, …), each a logical table keyed by an entity key. A store may hold a single value per key or multiple values per key. +- **`EntityDelta` (apply / undo).** Every state change is expressed as a delta that carries enough of the previous value to reverse itself. Applying deltas rolls the ledger forward; undoing them (from the WAL) rolls it back. This is the mechanism that makes rollbacks exact — see the [Sync Pipeline](./sync-pipeline). +- **`EpochValue` snapshot window.** Cardano staking reads state as it was several epochs ago. State entities that participate in staking are stored as a rotating window of snapshots (live / mark / set / go / next). The details are covered in the [Ledger Model](./ledger-model). +- **Archive dual storage.** Block bodies are written to append-only flatfile segments (one segment per Cardano epoch), while a compact index maps each slot to its `(segment, offset, length)` location. Historical entity changes are stored as `(slot, entity-key) → value` logs, enabling range queries over time. +- **Index dimensions.** The index store maintains multimaps keyed by address, payment credential, stake credential, policy id, and asset id, each pointing at the matching UTxOs (for current state) or slots (for historical lookups). + +Source: `crates/core/src/{state,archive,wal,indexes,mempool}.rs`, `crates/redb3/src/*`, `crates/fjall/src/*`, and storage configuration in `crates/core/src/config.rs`. diff --git a/docs/content/architecture/index.mdx b/docs/content/architecture/index.mdx new file mode 100644 index 00000000..8a67a7cd --- /dev/null +++ b/docs/content/architecture/index.mdx @@ -0,0 +1,53 @@ +--- +title: Overview +sidebar: + order: 0 +--- + +This section describes how Dolos is built. It is a macro view of the system — the layers, the crates, the storage engine, the sync pipeline, and the serving layer — meant for contributors, integrators, and operators who want to understand the internals without reading the source. It is not an API reference; for endpoint-level detail see the [APIs section](../apis/grpc). + +Dolos is a data node, not a consensus node. It connects to the Cardano network as an Ouroboros peer, keeps an up-to-date copy of the ledger, and serves that data through several query interfaces. Everything in the architecture follows from that goal: read the chain efficiently, keep resource usage low, and expose the result through the tools developers already use. + +## A layered system + +At the highest level, data flows in one direction — from an upstream peer, through the sync pipeline, into local storage — and is then read back out by the serving layer. + +```mermaid +flowchart TD + peer["Upstream peer
Ouroboros node-to-node (chainsync + blockfetch)"] + sync["Sync pipeline
pull → apply (→ submit) — gasket stages"] + storage["Storage
WAL · State · Archive · Index · Mempool (redb / fjall)"] + serve["Serving layer
gRPC · Mini-Blockfrost · Mini-Kupo · Ouroboros n2c socket · TRP"] + clients["Clients
dApps · wallets · explorers · cardano-cli"] + + peer -- "blocks / rollbacks" --> sync + sync -- "work units (apply / undo deltas)" --> storage + storage -- "read-only queries" --> serve + serve --> clients +``` + +Each layer is covered in its own page: + +- [Crates & Modules](./crates) — how the codebase is split into a Cargo workspace. +- [Data Layer](./data-layer) — the stores, their traits, and the pluggable storage backends. +- [Sync Pipeline](./sync-pipeline) — how blocks get from the network into storage. +- [Ledger Model & Epoch Transitions](./ledger-model) — how Cardano ledger state evolves. +- [Serving Layer & APIs](./serving-layer) — how the query interfaces read that state. + +## Chain-agnostic core, Cardano implementation + +The most important structural idea in Dolos is the split between a **generic core** and a **chain-specific implementation**. + +The `dolos-core` crate defines the abstractions that any UTxO blockchain data node needs, as a set of traits: `Domain` (the umbrella that bundles all the stores together), `ChainLogic` (how blocks turn into state changes), `WorkUnit` (a unit of pipelined ledger work), and the storage traits `StateStore`, `ArchiveStore`, `WalStore`, `IndexStore`, and `MempoolStore`. None of this code knows anything about Cardano specifically. + +The `dolos-cardano` crate implements those traits for Cardano — block parsing, ledger rules, epoch transitions, and reward calculations. Everything else in the system is either a **storage backend** (an implementation of the store traits, e.g. `dolos-redb3` or `dolos-fjall`) or a **serving adapter** (an implementation of an API surface that reads from a `Domain`). This is what keeps the sync pipeline, the storage engine, and the API servers decoupled from each other and from the ledger rules. + +## One domain, shared by sync and serve + +At runtime the whole node is wired together by a single `DomainAdapter` (in `src/adapters/`), which holds handles to every store plus the live Cardano ledger state (`Arc>`). The sync pipeline takes the write side of that lock to apply blocks; the serving layer takes read handles to answer queries concurrently. A tip broadcast channel lets streaming endpoints (chain follow, watch) react to new blocks as they are applied. + +Because state, archive, and index writes are committed as the pipeline advances, the serving layer reads an up-to-date view without the two sides blocking each other. Commits to the individual stores are sequential rather than atomic across stores, so a reader can briefly observe one store slightly ahead of another while a block is being applied. + +## Trust model + +Dolos relies on an honest upstream peer for block data — it validates transactions (phase-1 and phase-2) but does not run consensus or independently verify block production. This is the deliberate tradeoff that lets it stay lightweight. For the full rationale, see [What is Dolos](../what). diff --git a/docs/content/architecture/ledger-model.mdx b/docs/content/architecture/ledger-model.mdx new file mode 100644 index 00000000..3808e189 --- /dev/null +++ b/docs/content/architecture/ledger-model.mdx @@ -0,0 +1,68 @@ +--- +title: Ledger Model & Epoch Transitions +sidebar: + order: 4 +--- + +This page is the deepest part of the architecture overview: how Cardano ledger state actually evolves inside Dolos. It stays at a conceptual level — enough to understand the moving parts and where they live in the code. Contributors working on ledger correctness should also read the "Fix Journal" and ledger notes in the repository's `CLAUDE.md`, which document the rules and edge cases in detail. + +The Cardano ledger is implemented in the `dolos-cardano` crate as the `CardanoLogic` type, which fulfills the generic `ChainLogic` trait from `dolos-core`. + +## Work units + +When the sync pipeline hands a block to `CardanoLogic`, it does not mutate state directly — it produces **work units**, the pipelined units of ledger work introduced in the [Sync Pipeline](./sync-pipeline). Cardano defines these variants (`CardanoWorkUnit`): + +| Work unit | When | What it does | +| --------- | ---- | ------------ | +| **Genesis** | node bootstrap | Builds the initial ledger from the genesis configuration. | +| **Roll** | every block | Applies a block: inputs, outputs, mints/burns, certificates, withdrawals, and parameter updates. | +| **Rupd** | mid-epoch (stability window) | Computes the epoch's reward distribution from a stake snapshot. | +| **Ewrap** | epoch boundary (close) | Closes the epoch: applies rewards, refunds deposits, enacts governance, and finalizes the pots. | +| **Estart** | epoch boundary (open) | Opens the next epoch: rotates snapshots, transitions pools/DReps/proposals, updates nonces, and advances the era. | +| **ForcedStop** | testing | Halts at a target epoch (used by the integration test harness). | + +## The snapshot window + +Cardano does not reward stake based on its *current* delegation — it uses delegation as it stood a few epochs earlier, so that the stake distribution used for leader election and rewards is stable and known in advance. Dolos models this with a rotating window of snapshots, `EpochValue`, attached to every entity that participates in staking (accounts, pools, DReps). + +Each `EpochValue` holds several named snapshots — the arrows show where each value moves on an epoch transition: + +```mermaid +flowchart LR + next["next
upcoming epoch (E+1)"] + live["live
current epoch (E)"] + mark["mark
previous epoch (E-1)"] + set["set
two epochs ago (E-2)
stake used for this epoch's rewards"] + go["go
three epochs ago (E-3)"] + + next --> live --> mark --> set --> go +``` + +Reward calculations read from the appropriate historical slot (e.g. the `mark`/`set` snapshots), which is why a correct implementation must write each change into the right slot at the right time. Getting a snapshot's timing wrong is a common class of ledger bug. + +## Epoch boundary phases + +The transition between epochs happens in a defined order — this ordering matters, because rewards, snapshots, and deposit refunds interact. + +```mermaid +flowchart LR + rupd["RUPD
snapshot rewards at stability window (mid-epoch)"] + ewrap["EWRAP
close epoch (at boundary)"] + estart["ESTART
open next epoch (at boundary)"] + + rupd --> ewrap --> estart +``` + +1. **RUPD** — partway through the epoch, at the randomness-stability window, Dolos snapshots the stake distribution and computes the rewards that will be paid out. Registration status at this moment determines who is eligible. +2. **EWRAP** — at the boundary, the epoch is *closed*: pending rewards are applied to accounts (filtered for anyone who deregistered since RUPD), deposits are refunded, governance actions are enacted, and the treasury/reserves accounting is settled. +3. **ESTART** — the next epoch is *opened*: the snapshot window rotates, pool/DRep/proposal state transitions forward, epoch nonces update, era changes take effect, and the chain cursor advances. + +## The pots + +Dolos tracks the Cardano money supply as a set of **pots** — reserves, treasury, the rewards pot, the UTxO total, and collected fees — plus outstanding deposits. These satisfy the ledger invariant that they always sum to the fixed maximum supply. During an epoch, changes accumulate as a delta; at the boundary the delta is applied to produce the new pots: monetary expansion is drawn from reserves, the treasury takes its cut, rewards are distributed, and fees roll over. The formulas (monetary expansion η, epoch incentives, and how unspendable rewards are routed) are documented alongside the implementation in `crates/cardano/src/pots.rs` and in `CLAUDE.md`. + +## Sharding + +The epoch boundary touches every stake account, which on mainnet is a large set. To keep transitions fast, EWRAP and ESTART split the per-account work across a fixed number of shards (`ACCOUNT_SHARDS`), each covering a prefix range of the account key space, and run them in parallel through the work-unit `load` step before a final global pass. Progress is tracked per shard so an interrupted boundary can resume rather than restart. + +Source: `crates/cardano/src/{work,model/*,rupd,ewrap,estart,pots,rewards,shard}.rs`. diff --git a/docs/content/architecture/serving-layer.mdx b/docs/content/architecture/serving-layer.mdx new file mode 100644 index 00000000..66c019c0 --- /dev/null +++ b/docs/content/architecture/serving-layer.mdx @@ -0,0 +1,52 @@ +--- +title: Serving Layer & APIs +sidebar: + order: 5 +--- + +The serving layer is everything that reads ledger data back out and exposes it to clients. Dolos offers several API surfaces so developers can use the tools they already know, but they all share the same foundation: each is a thin adapter that queries the shared domain read-only. This page explains that shared foundation, the surfaces built on top of it, and the transaction-submission path. + +## One domain, many adapters + +Every API server receives a handle to the same `DomainAdapter` (see the [Overview](../architecture)) and reads through the storage traits — it never owns its own copy of the data. Read access is provided through query facades (`Facade` / `AsyncQueryFacade`) that translate a request into lookups against the state, archive, and index stores. Because the sync pipeline commits as it advances, these reads always see an up-to-date, consistent view. + +Streaming endpoints (chain-follow, tip-watch) subscribe to the tip broadcast channel that the sync pipeline publishes to, so they push new blocks to clients as they are applied. + +This design means the serving layer adds no new storage and imposes no coupling on the ledger rules — adding an API surface is a matter of writing another adapter over `Domain`. + +## API surfaces + +| Surface | Protocol / framework | Crate / module | Purpose | +| ------- | -------------------- | -------------- | ------- | +| **UTxO RPC** | gRPC (tonic) + gRPC-Web | `src/serve/grpc` | The primary programmatic interface — query, submit, and follow the chain over the [UTxO RPC](https://utxorpc.org) spec. | +| **Mini-Blockfrost** | HTTP REST (axum) | `dolos-minibf` | A Blockfrost-compatible API so existing Blockfrost integrations work unchanged. | +| **Mini-Kupo** | HTTP REST (axum) | `dolos-minikupo` | Kupo-compatible pattern-matching over UTxOs, plus datum and script resolution. | +| **Ouroboros node-to-client** | Unix socket / named pipe | `src/serve/o7s_unix`, `o7s_win` | The local node protocol, for compatibility with `cardano-cli`, Ogmios, and similar tooling — no full node required. | +| **TRP** | JSON-RPC (jsonrpsee) | `dolos-trp` | The Transaction Resolver Protocol — resolve [Tx3](../integrations/tx3) templates into concrete transactions and submit them. | + +Each surface is enabled by a Cargo feature and configured independently. For endpoint-level details and configuration, see the [APIs section](../apis/grpc) and the per-API pages ([UTxO RPC](../apis/grpc), [Mini-Blockfrost](../apis/minibf), [Mini-Kupo](../apis/minikupo)). + +## Mempool and transaction submission + +Submitting a transaction — whether over gRPC, TRP, or the node-to-client socket — flows through the mempool, which performs validation locally before the transaction ever reaches the network. + +```mermaid +flowchart TD + client["client"] + validation["validation
phase-1 (ledger rules) · phase-2 (Plutus evaluation)"] + mempool["mempool
overlays in-flight UTxOs → enables tx-chaining"] + submitstage["submit stage
txsubmission → upstream peer"] + confirmed["confirmed on chain → finalized"] + + client -- "submit tx" --> validation + validation --> mempool + mempool --> submitstage + submitstage --> confirmed +``` + +- **Local validation.** Before accepting a transaction, Dolos runs phase-1 checks (structure and ledger rules) and phase-2 checks (Plutus script evaluation via Pallas), so invalid transactions are rejected immediately with a report. +- **In-flight overlay and tx-chaining.** The mempool overlays the UTxOs produced by pending transactions on top of committed state during validation, so a client can submit a transaction that spends the output of another not-yet-confirmed transaction. +- **Lifecycle.** Each transaction moves through a set of stages — pending, propagated/in-flight, confirmed, and finalized (with dropped/rolled-back as failure paths) — tracked in the mempool store and queryable by clients. +- **Network path.** The sync pipeline's [submit stage](./sync-pipeline) takes pending transactions from the mempool and offers them to the upstream peer over the Ouroboros txsubmission protocol. + +Source: `src/serve/*`, `src/adapters/mod.rs`, the `dolos-minibf` / `dolos-minikupo` / `dolos-trp` crates, and the mempool/submit traits in `crates/core/src/{mempool,submit}.rs`. diff --git a/docs/content/architecture/sync-pipeline.mdx b/docs/content/architecture/sync-pipeline.mdx new file mode 100644 index 00000000..42e67e4c --- /dev/null +++ b/docs/content/architecture/sync-pipeline.mdx @@ -0,0 +1,70 @@ +--- +title: Sync Pipeline +sidebar: + order: 3 +--- + +The sync pipeline is how Dolos gets blocks from the Cardano network into local storage and keeps the ledger in step with the chain tip. It connects to an upstream peer over Ouroboros, streams blocks through a small staged pipeline, and applies each one to the stores as reversible work. + +## Connecting to the network + +Dolos speaks the Ouroboros **node-to-node** mini-protocols via Pallas: + +- **chainsync** — follows the upstream peer's chain, receiving block headers and rollback notifications as the tip advances. +- **blockfetch** — fetches the full block bodies for the headers chainsync announced. + +The same protocols run in the other direction too: the `relay/` module (`src/relay/`) lets Dolos *serve* chainsync and blockfetch to downstream peers, so a node can act as a relay for others. + +## Staged pipeline + +Synchronization runs as a [gasket](https://github.com/construkts/gasket-rs) pipeline — independent stages connected by bounded channels, which gives natural backpressure when one stage is slower than another. The stages live in `src/sync/`: + +```mermaid +flowchart LR + peer["upstream peer"] + pull["pull"] + apply["apply"] + stores["stores
state · archive · index · WAL"] + mempool["mempool"] + submit["submit"] + + peer --> pull + pull -- "PullEvent
RollForward / Rollback" --> apply + apply --> stores + mempool --> submit + submit -- "txsubmission" --> peer +``` + +- **pull** (`pull.rs`) — drives chainsync and blockfetch. It validates header continuity (parent hash, slot order) against a chain fragment, fetches bodies, and emits `PullEvent::RollForward(block)` or `PullEvent::Rollback(point)`. +- **apply** (`apply.rs`) — consumes those events and applies them to the ledger via the domain's roll-forward / rollback entry points. It also triggers periodic housekeeping. +- **submit** (`submit.rs`) — the reverse path: it takes transactions from the mempool and offers them to the upstream peer over the txsubmission protocol. + +On a local devnet, pull + apply are replaced by an `emulator` stage that produces blocks on an interval instead of fetching them from a peer. + +## Applying a block (roll forward) + +When the apply stage receives a block, the Cardano chain logic turns it into one or more **work units** — the pipelined unit of ledger work defined by `dolos-core`'s `WorkUnit` trait. Each work unit runs through a fixed lifecycle: + +``` +initialize → load (per shard) → commit_state → commit_archive → commit_indexes → finalize → tip_events +``` + +- `initialize` loads the context the unit needs. +- `load` does the actual computation, optionally split across shards for parallelism. +- the `commit_*` steps write the resulting deltas into the state, archive, and index stores. +- `finalize` runs any global post-processing; the chain cursor is advanced here (only at an epoch start). +- `tip_events` emits notifications so streaming API clients see the new tip. + +The kinds of work a Cardano block produces — ordinary block application, reward updates, and the two halves of an epoch boundary — are covered in the [Ledger Model](./ledger-model). + +## Rolling back + +Cardano chains can reorganize, so any applied block may later be undone. Dolos handles this without recomputation: the WAL stores each recent block as a set of `EntityDelta`s that know how to reverse themselves. To roll back to a point, the apply stage reads WAL entries from the current tip backward and applies each delta's inverse, restoring the exact prior state, then resets the cursor to the target point. + +This is why the WAL exists as a separate store — it is the bounded buffer of reversible history that makes rollbacks precise. See the [Data Layer](./data-layer) for how it is stored. + +## Housekeeping + +As the tip advances, the pipeline periodically prunes history it no longer needs — trimming the WAL and, in sliding-history mode, the archive — according to the configured retention window. The `dolos doctor` and `dolos data` commands expose the same maintenance operations for manual use. + +Source: `src/sync/{mod,pull,apply,submit,emulator}.rs`, `src/relay/*`, and the `WorkUnit` / roll traits in `crates/core/src/{work_unit,sync}.rs`.