[3.x] Support pages with more output file extensions (Refactors)

EPIC_NON_HTML_PAGES.md

@@ -50,9 +50,10 @@ require every output format to have a matching filesystem-discovered page class.
 - `BuildService::getPageTypes()` derives page classes from the live page collection,
   and `StaticPageBuilder` writes whatever `getOutputPath()` returns — dynamic pages
   need zero build-service changes.
-- `HydeCoreExtension::discoverPages()` is the proven registration point for
-  feature-gated virtual pages (search index, redirects), and users/packages can do
-  the same via `Hyde::kernel()->booting()` callbacks or a `HydeExtension`.
+- Extension page discovery is the proven registration mechanism for feature-gated
+  virtual pages. Normal pages use `discoverPages()`; framework fallbacks use the
+  later `discoverDefaultPages()` phase so user routes are present before defaults
+  fill any remaining gaps.
 - The route-key-with-extension convention is already de facto established:
   `DocumentationSearchIndex` uses route key `docs/search.json` equal to its output path.
 
@@ -176,13 +177,11 @@ a standalone feature in its own right.
 
 ### D4: Generators become container-resolved pages; generator actions stay
 
-Each generated file is registered as an `InMemoryPage` whose compiled contents
-resolve its generator **from the container at build time**, e.g. a `sitemap.xml`
-page whose compile step returns `app(SitemapGenerator::class)->generate()`. The XML
-generator actions (`SitemapGenerator`, `RssFeedGenerator`) are untouched and remain
-the default implementations — this is still the `DocumentationSearchIndex` →
-`GeneratesDocumentationSearchIndex` split, but the generator is *resolved*, not
-`new`'d.
+Each generated file is registered as an internal `GeneratedFilePage`, built on
+`InMemoryPage` exact-path semantics. Its compiled contents resolve a generator
+**from the container at build time**. All generated-file generators expose a small
+internal final-string adapter while retaining their existing public `generate()`
+behavior; the XML generators also retain `getXml()`.
 
 Resolving through the container is the point: a user can rebind `SitemapGenerator`
 (or the page's content closure) to swap the output without replacing the page — a
@@ -191,9 +190,9 @@ produced **lazily** (a `compile` macro / closure or a thin subclass `compile()`)
 never eager string content, since generation must run at build time against the
 final route set.
 
-Registration happens in `HydeCoreExtension::discoverPages()` behind the existing
-`Features::hasSitemap()` / `Features::hasRss()` conditions, replacing the
-registrations in `BuildTaskService::registerFrameworkTasks()`.
+Definitions for sitemap, RSS, robots, and llms output live in one internal registry.
+`HydeCoreExtension::discoverDefaultPages()` loops over enabled definitions after all
+normal page discovery has completed and registers only routes that are still absent.
 
 **Plain page vs. thin subclass is deferred to PR 5.** D3 already defaults non-HTML
 pages out of the sitemap, so a subclass is *not* needed merely to stop a generated
@@ -215,40 +214,29 @@ instantiate. Lean toward a plain `InMemoryPage` registered with a container-boun
 > unaffected: `compile()` resolves `SitemapGenerator` from the container, verified
 > by a rebind test. Part B should mirror this with `RssFeedPage`.
 
+> **Superseded by the post-implementation simplification:** command construction no
+> longer requires page class identity. Commands build the registered route, and the
+> shared registry supplies the fallback route information. One internal
+> `GeneratedFilePage` therefore replaces `SitemapPage`, `RssFeedPage`,
+> `RobotsTxtPage`, and `LlmsTxtPage` without changing generator rebinding or route
+> override behavior.
+
 ### D5: User-defined pages beat generators
 
 If the page collection already contains a user-defined page with a route key such
-as `robots.txt`, the framework does not register its generated `RobotsTxtPage`.
-This follows the pattern of `discoverDocumentationRootRedirect()`, which skips when
-a user-defined route exists.
+as `robots.txt`, the framework does not register the matching generated default.
 Users can register an `InMemoryPage` from a service provider or provide a custom page
 class through an extension. Combined with D4's container-resolved generators, this
 gives a smooth escalation path:
 feature default → config tweaks → rebind the generator (or content closure) in the
 container → fully custom page in code.
 
-> **Timing caveat — the skip check is ordering-sensitive.** "Is a `robots.txt` route
-> already registered" is evaluated at `discoverPages()` time, so whether a user's page
-> wins depends on it being visible at that moment. A page registered via a late
-> `booting()` callback may or may not be present depending on boot ordering. The
-> `discoverDocumentationRootRedirect()` precedent suggests this is fine, but "fine and
-> ordering-dependent" is exactly what passes in our tests and breaks for the one user
-> who registers late. PR 5/6 MUST include an end-to-end test asserting that a
-> user-registered `robots.txt` (via both a `HydeExtension` page class and a `booting()`
-> `addPage()` callback) suppresses the generated one — this is the D5 contract and the
-> most likely silent failure for the power-user audience.
-
-> **Verified for the sitemap (PR 5 part A):** both user paths win, through two
-> different mechanisms that the tests pin down end-to-end. `booting()` callbacks run
-> before the page collection boots (`BootsHydeKernel::boot()`), so a callback-registered
-> `sitemap.xml` page is visible to the core extension's `hasPageWithRouteKey()` skip
-> check. A user `HydeExtension` runs *after* the core extension (registration order),
-> so the skip check cannot see its pages; instead the user page replaces the generated
-> one under the same collection key (`addPage()` keys by source path). Both are
-> asserted through the real `build` command output. The robots.txt equivalent remains
-> mandatory for PR 6. *(Part B: both paths verified the same way for the feed page.)*
-> *(PR 6: both paths verified the same way for the robots.txt page.)*
-> *(PR 7: both paths verified the same way for the llms.txt page.)*
+> **Superseded ordering design:** page discovery now has two explicit phases. Every
+> extension completes `discoverPages()` before any extension runs
+> `discoverDefaultPages()`. Framework-generated files are registered in the default
+> phase only when their route is absent. User precedence is therefore independent of
+> page-collection source keys and normal extension registration order. Default
+> handlers still run in extension registration order relative to other defaults.
 
 ### D6: No built-in `TextPage` or `.txt` autodiscovery
 
@@ -270,6 +258,11 @@ an extension point.
 
 ## Work breakdown (planned PR sequence, in dependency order)
 
+> **Historical note:** the PR 5–7 implementation notes below record the intermediate
+> thin-page implementation that originally landed. The post-implementation
+> simplification recorded in D4 and D5 supersedes references to the four specialized
+> page subclasses and registration through `discoverPages()`.
+
 ### PR 1 — Foundation: explicit output paths and page-class extensions ✅ Implemented
 
 Goal: any page class can emit a non-`.html` file without overriding `getOutputPath()`.

HYDEPHP_V3_PLANNING.md

@@ -38,6 +38,11 @@ Having this document in code lets us know the devlopment state at any given poin
 
 ### Minor Changes and Cleanup
 
+- Added `HydeExtension::discoverDefaultPages()` for fallback pages. All extensions
+  complete normal `discoverPages()` handling before any default-page handler runs,
+  allowing framework-generated routes to fill gaps without depending on normal
+  extension registration order. Default handlers retain extension registration order
+  relative to each other.
 - Fixed documentation search index files leaking into the generated sitemap: `search.json` (and any other page compiled to a non-HTML output file) no longer appears in `sitemap.xml`. The sitemap generator now asks each page through `HydePage::showInSitemap()` instead of only filtering out redirect pages.
 - The `Redirect` page class constructor now accepts an optional `$matter` parameter, used by the framework to hide the generated documentation root redirect from navigation menus. Existing usages are unaffected.
 - The realtime compiler now resolves registered page routes before proxying static assets, replacing the hardcoded `search.json` exemption, so `hyde serve` serves any registered route regardless of its output extension. Registered pages now always win over a static file at the same path; the previous behavior of serving such a shadowing file only affected the dev server and no real setups are expected to be affected.

TASK_NON_HTML_PAGE_SIMPLIFICATION.md

@@ -0,0 +1,119 @@
+# Non-HTML Page Simplification Task Specification
+
+## Objective
+
+Reduce the accidental complexity left by the first-class non-HTML page work while preserving its user-facing behavior:
+
+- generated files remain registered pages and routes;
+- `InMemoryPage::file()` remains the explicit arbitrary-file API;
+- registered routes continue to win over static assets in the realtime compiler;
+- sitemap inclusion remains a page policy through `showInSitemap()`;
+- generated-file generators remain replaceable through the service container;
+- user-defined pages continue to replace framework-generated files;
+- the existing `build:sitemap` and `build:rss` commands remain available.
+
+The implementation is split into isolated issues so each change can be reviewed, tested, and committed independently.
+
+## Issue 1: Derive route keys from resolved output paths
+
+### Problem in the current code
+
+`HydePage` constructs its stored route key with `RouteKey::fromPage()` before the instance output path is considered. `RouteKey::fromPage()` independently combines the page class output directory, identifier, and static `$outputExtension`. `InMemoryPage::file()` then overrides `getOutputPath()` at the instance level.
+
+This creates two representations of the same output semantics:
+
+- `packages/framework/src/Support/Models/RouteKey.php` derives a route from class-level declarations;
+- `packages/framework/src/Pages/InMemoryPage.php` can resolve a different exact instance output path.
+
+`DocumentationPage` also overrides both `getRouteKey()` and `getOutputPath()` to keep flattened documentation routes aligned, illustrating the same duplication.
+
+### Implementation
+
+1. Add `RouteKey::fromOutputPath(string $outputPath)`, with the single route rule: remove a trailing `.html`; retain every other path and extension verbatim.
+2. Change `HydePage` construction so the stored route key is derived from the instance's resolved `getOutputPath()`.
+3. Change `HydePage::outputPath()` to construct the output path directly from the normalized page identifier, output directory, and output extension instead of round-tripping through a route key.
+4. Keep `RouteKey::fromPage()` as a compatibility/helper API, but implement it by passing the page class's resolved static output path to `fromOutputPath()`.
+5. Remove `DocumentationPage::getRouteKey()` once its resolved `getOutputPath()` automatically determines the stored route key.
+
+### Acceptance criteria
+
+- HTML output `foo/bar.html` has route key `foo/bar`.
+- Non-HTML output `foo/bar.json` has route key `foo/bar.json`.
+- Extensionless exact output `feed` has route key `feed`.
+- An exact-path `InMemoryPage::file('robots.txt')` has output path and route key `robots.txt`.
+- A page subclass overriding only `getOutputPath()` receives a matching route key without also overriding `getRouteKey()`.
+- Documentation numerical-prefix stripping, post date-prefix stripping, output directories, non-HTML extension de-duplication, and flattened documentation output keep their existing behavior.
+- Existing page, route, build, manifest, and realtime compiler tests remain green.
+
+## Issue 2: Consolidate generated-file page wiring
+
+### Problem in the current code
+
+The following internal classes repeat the same page shell:
+
+- `Framework/Features/XmlGenerators/SitemapPage.php`
+- `Framework/Features/XmlGenerators/RssFeedPage.php`
+- `Framework/Features/TextGenerators/RobotsTxtPage.php`
+- `Framework/Features/TextGenerators/LlmsTxtPage.php`
+
+Each class supplies an exact path, hidden navigation, a container-resolved generator, and a small `compile()` adapter. `HydeCoreExtension` repeats a feature check and discovery method for each shell. XML generators return themselves from `generate()` and require `getXml()`, while text generators return their final strings directly.
+
+### Implementation
+
+1. Add an internal `GeneratedFileGenerator` contract with one final rendering method returning a string.
+2. Implement the contract in the text generators and `BaseXmlGenerator` through compatibility adapters, retaining the existing public `generate()` behavior for generator subclasses and callers.
+3. Add one internal `GeneratedFilePage` that stores an exact output path and generator class, hides itself from navigation, resolves the generator from the container at compile time, and returns the contract's final string.
+4. Add a generated-file definition/registry containing the four feature checks, output paths, and generator bindings. The configured RSS filename is resolved when definitions are created.
+5. Make `HydeCoreExtension` loop over the registry and add only missing routes.
+6. Update `build:sitemap`, `build:rss`, and the robots sitemap link to use registry paths instead of specialized page classes.
+7. Remove the four specialized generated page classes and consolidate their tests around `GeneratedFilePage` plus route behavior.
+
+### Acceptance criteria
+
+- The same four generated routes appear under the same feature conditions and configured RSS filename.
+- Generated pages are exact-path pages, hidden from navigation, and excluded from the sitemap by default.
+- All four files compile through the standard build and remain present in the build manifest and route list.
+- Rebinding any concrete generator in the service container changes the compiled route output.
+- Existing `generate()` and XML `getXml()` generator APIs continue to work.
+- `build:sitemap` and `build:rss` retain their current success, failure, configured-filename, and user-page override behavior.
+
+## Issue 3: Register framework defaults after user pages
+
+### Problem in the current code
+
+`PageCollection::runExtensionHandlers()` invokes extensions in registration order. `HydeCoreExtension` is registered first, so generated pages are added before user extensions. A user extension currently wins because adding an exact-path page happens to replace the generated page under the same page-collection source key, or later wins in the route collection.
+
+That makes “user pages beat framework defaults” dependent on collection keys and timing rather than a lifecycle guarantee.
+
+### Implementation
+
+1. Add a documented `discoverDefaultPages(PageCollection $collection)` extension hook for fallback pages.
+2. Run all normal `discoverPages()` hooks first, then all `discoverDefaultPages()` hooks.
+3. Move the generated-file registry loop from `HydeCoreExtension::discoverPages()` into `discoverDefaultPages()`.
+4. Keep each generated definition's route-existence check so defaults only fill gaps.
+5. Add a regression test where a user extension registers the same route under a different page-collection source key; assert the default is not added at all.
+
+### Acceptance criteria
+
+- Source-discovered pages, booting-callback pages, and normal extension pages exist before framework defaults are evaluated.
+- A user page with a generated route key suppresses that default regardless of its source path/collection key.
+- Only one page and one route exist for an overridden generated output.
+- Existing extension discovery order remains unchanged within each phase.
+- Other core-discovered pages (redirects and documentation search pages) retain their existing discovery behavior.
+
+## Verification
+
+For each issue:
+
+1. Run its focused unit and feature tests before committing.
+2. Run the complete framework test suite after the final issue.
+3. Run realtime compiler tests because route/output semantics cross package boundaries.
+4. Run the repository formatting/static checks available for changed PHP files.
+5. Confirm the worktree is clean after the final commit.
+
+## Deliberate non-goals
+
+- Do not introduce an `OutputPath` value object in this refactor. Existing framework APIs expose paths as strings; changing all of them would add migration surface without being necessary to establish one source of truth.
+- Do not add a public general-purpose `hyde build:route` command. Removing command-driven page subclasses solves the immediate coupling; a new CLI API should be evaluated separately.
+- Do not revisit the `$fileExtension` to `$sourceExtension` rename here. It is an independent v3 API cleanup already implemented on this branch, not a dependency of these simplifications.
+- Do not change route-first realtime compiler behavior, sitemap policy, exact-path validation, generated content formats, or feature configuration.

UPGRADE.md

@@ -17,6 +17,21 @@ InMemoryPage::file('robots.txt', contents: $text);
 // _site/robots.txt
 ```
 
+Route keys are always derived from the resolved output path by removing only its final `.html` extension. This also
+applies to exact-path files, so the page and file constructors intentionally differ for an identifier ending in HTML:
+
+```php
+InMemoryPage::file('download.html');
+// output: download.html, route key: download
+
+InMemoryPage::make('download.html');
+// output: download.html.html, route key: download.html
+```
+
+Custom pages that override `getOutputPath()` no longer need to override `getRouteKey()`. The route key is resolved
+during parent construction and remains the page's stable identity. If the output path depends on subclass instance
+state, initialize that state before calling `parent::__construct()`.
+
 ## Before You Begin
 
 ### Prerequisites
@@ -212,6 +227,10 @@ Hyde::kernel()->booting(function ($kernel): void {
 });
 ```
 
+Framework fallback pages are now registered only after every extension has completed `discoverPages()`. If an extension
+previously inspected or modified a framework-generated page from that handler, move that work to
+`discoverDefaultPages()` or a later lifecycle hook. Default handlers still run in extension registration order.
+
 The `build:sitemap` and `build:rss` commands still work and now compile the registered pages. When the output
 cannot be generated (no base URL, disabled in the configuration, or — for the feed — no Markdown posts), they
 fail with an error instead of generating an empty or unwanted file. `build:sitemap` reports this

packages/framework/src/Console/Commands/BuildRssFeedCommand.php

@@ -8,7 +8,8 @@
 use Hyde\Console\Concerns\Command;
 use Hyde\Foundation\Facades\Routes;
 use Hyde\Framework\Actions\StaticPageBuilder;
-use Hyde\Framework\Features\XmlGenerators\RssFeedPage;
+use Hyde\Framework\Features\GeneratedFiles\GeneratedFileRegistry;
+use Hyde\Support\Models\RouteKey;
 
 use function sprintf;
 
@@ -25,7 +26,8 @@ class BuildRssFeedCommand extends Command
 
     public function handle(): int
     {
-        $page = Routes::find(RssFeedPage::routeKey())?->getPage();
+        $routeKey = RouteKey::fromOutputPath(GeneratedFileRegistry::rssOutputPath())->get();
+        $page = Routes::find($routeKey)?->getPage();
 
         if ($page === null) {
             $this->error('Cannot generate the RSS feed as the feature is not enabled');

packages/framework/src/Console/Commands/BuildSitemapCommand.php

@@ -8,7 +8,7 @@
 use Hyde\Console\Concerns\Command;
 use Hyde\Foundation\Facades\Routes;
 use Hyde\Framework\Actions\StaticPageBuilder;
-use Hyde\Framework\Features\XmlGenerators\SitemapPage;
+use Hyde\Framework\Features\GeneratedFiles\GeneratedFilePaths;
 
 use function sprintf;
 
@@ -25,7 +25,7 @@ class BuildSitemapCommand extends Command
 
     public function handle(): int
     {
-        $page = Routes::find(SitemapPage::routeKey())?->getPage();
+        $page = Routes::find(GeneratedFilePaths::SITEMAP)?->getPage();
 
         if ($page === null) {
             $this->error('Cannot generate the sitemap as the feature is not enabled');