ci: build and validate the PyPI sdist/wheel#16
Merged
Conversation
Publishing scolta to PyPI is manual and nothing in CI built the distribution artifacts, so packaging breakage or cruft only surfaced at `twine upload` time. Add a `dist` job that builds both artifacts (`uv build`), runs `twine check dist/*`, and runs a new `scripts/validate-dist.py` content gate that also runs locally. The gate asserts (fail-closed on the file sets that are ours to enumerate): - wheel carries every vendored browser runtime asset (css/js/pagefind/ wasm) — the failure mode that matters most: a wheel that imports but ships no search UI because vendor_assets.py was never run; - wheel has nothing outside the scolta package + dist-info (no tests, __pycache__, *.pyc, .sha256/.d.ts/.map sidecars); - sdist is a buildable source set with no local build-dir junk; - both artifacts under size caps (~2x the measured good build). Also fix a real sdist bloat: hatchling's source dist defaulted to "everything on disk minus VCS-ignored", which pulled in tests/js/node_modules (61 MB, incl. .idea files) and tools/stemmer-golden/target (Rust artifacts) — a 7.9 MB sdist. An enumerated [tool.hatch.build.targets.sdist] exclude list prunes those and caches/IDE/pyc junk, dropping the sdist to ~2.35 MB. The wheel was already clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
distjob toci.ymlthat builds and validates the PyPI artifacts on every push/PR, plus a fix for a real sdist bloat found while building it.Publishing scolta is manual, and nothing in CI built the sdist/wheel — so packaging breakage or cruft only surfaced at
twine uploadtime. The new job:uv build— builds both sdist and wheel exactly as a release would.twine check dist/*— metadata / long-description (PyPI render) validity.scripts/validate-dist.py— content + size asserts (runs locally too, afteruv build).What the gate checks
Fail-closed on the file sets that are ours to enumerate; every failure message says what leaked/is missing and where the controlling filter lives (
pyproject.tomlhatch targets, orscripts/vendor_assets.py).Wheel:
vendor_assets.pywas never run or hatch dropped the data files:scolta/assets/css/scolta.cssscolta/assets/js/scolta.jsscolta/assets/pagefind/pagefind-worker.js,pagefind.js,wasm.en.pagefind,wasm.unknown.pagefindscolta/assets/wasm/scolta_core.js,scolta_core_bg.wasmscoltapackage + dist-info (notests/,__pycache__,*.pyc, or.sha256/.d.ts/.mapsidecars).Sdist:
pyproject.toml,src/scolta/__init__.py,PKG-INFO,src/scolta/assets/).node_modules, cargotarget/, caches,.idea/.vscode,.pyc,.DS_Store.Size caps (shared pattern — ~2x the measured good artifact, value documented in the script):
Real find fixed in the same PR
The sdist was 7.9 MB. Hatchling's source dist defaults to "everything on disk minus VCS-ignored", which does not prune nested untracked trees — it pulled in
tests/js/node_modules(61 MB, including.ideaIDE files from a vendored npm dep) andtools/stemmer-golden/target(Rust build artifacts). An enumerated[tool.hatch.build.targets.sdist].excludelist prunes those plus cache/IDE/pyc junk, dropping the sdist to ~2.35 MB while keeping the full ported test corpus and stemmer fixtures. The wheel was already clean (packages = ["src/scolta"]).Why
Same dist-cruft class as the scolta-wp 13 MB plugin-zip incident and the WP.org dist-cruft compliance flags — caught now in PR instead of at publish.
Tested
uv build+twine check dist/*— both PASSED.uv run python scripts/validate-dist.pyagainst the real build — passed (wheel 729,525 B, sdist ~2.35 MB).tests/) and bad sdist (node_modules, cargotarget/) — all 14 expected asserts fired with actionable messages.uv run ruff check .anduv run ruff format --check .— green.