Attach provenance and SBOM attestations to published images#1350
Merged
Conversation
Docker Scout reports "Required supply chain attestations missing" on every skiplabs/* image, and without provenance it has to guess where an image came from, printing "Base image was auto-detected" and attributing findings to a base it inferred rather than the one actually used. Emit provenance (mode=max) and SBOM attestations via bake's --provenance and --sbom shorthands. These are deliberately restricted to push flows. Attestations require the docker-container driver, which is what the named builder used for --push/--push-only provides. Local builds export through --load on the default docker driver, which rejects them outright: ERROR: Attestation is not supported for the docker driver. so enabling them unconditionally -- e.g. by putting attest in docker-bake.hcl -- would break every local build. Verified both directions with `bake --print`: attest is present on the push path and absent on the local path. Other drawbacks considered: - mode=max records build args, so a secret passed as a build arg would be embedded. The only ones passed here are STAGE and SKIP_CAPACITY, neither sensitive. Noted in a comment so it stays that way. - Attestations add unknown/unknown entries to the image index. Normal pulls ignore them, and nothing in this repo parses manifest lists, so no tooling is affected. - The skipruntime target exports with --output type=local and is never pushed, so it is unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mbouaziz
added a commit
that referenced
this pull request
Jul 16, 2026
Closes #1348. ## Problem The `skiplabs/*` images CI runs on were published by a maintainer running `bin/release_docker_ci_images.sh` locally, with their own Docker Hub credentials. Nothing else ever rebuilt them, and it shows: - `skiplang-bin-builder` sat ~a year stale carrying **3 critical / 68 high** CVEs (#1338), because the only thing that ever published it was a human remembering to. - `skdb-base` silently stopped building after the Ubuntu 24.04 bump and no one noticed until a manual rebuild happened to surface it (#1290) — CI runs *against* the published images, so it structurally cannot catch this. - `skiplang:latest` and `skip:latest` on Docker Hub are currently a week apart despite being stages of the *same* Dockerfile, so CI jobs run against inconsistent compiler builds. ## What this does Builds and pushes the images from GitHub Actions instead, on three triggers: - **`workflow_dispatch`** — on demand - **`push` to `main`**, path-filtered to the files the images are built from — this is what fixes the #1338 class of rot (pin bumped, republish forgotten) - **weekly `schedule`** — picks up base-image security updates, since with no layer cache every run re-resolves `FROM` and re-runs `apt` ### Scope: four toolchain images Publishes the new **`ci` group** in `docker-bake.hcl`: `skiplang`, `skiplang-bin-builder`, `skip`, `skdb-base`. These are toolchain images — their final stages hold compiled binaries and apt packages but no repo source, so any green `main` commit can rebuild them and republishing is not a release. `skdb` and `skdb-dev-server` are deliberately excluded: they bake in source and their tags carry release semantics. (`skiplabs/skdb` has in fact never existed on Docker Hub, despite being in bake's `default` group.) ### Source of truth The `ci` group is authoritative. The workflow derives its per-image tag overrides from `bake --print ci` at runtime rather than listing them, `release_docker_ci_images.sh` now takes its list from the same group (so the manual path and the automated one can't drift, and a manual publish finally includes `skiplang-bin-builder`), and a new `bin/check_ci_images.sh` asserts every image CircleCI pulls is in the group — replacing the old `grep .circleci/base.yml` derivation, which structurally could not express `skiplang-bin-builder`. ### Runner strategy Native `ubuntu-24.04` + `ubuntu-24.04-arm` in parallel (arm64 hosted runners are free on public repos), each pushing by digest, then a merge job creates the multi-arch tags. QEMU was rejected: ~4× slower and it would emulate a self-hosting compiler bootstrap. **Nothing rewrites a `:latest` tag until every platform of every image has pushed successfully**, and the merge validates all images before publishing any. Provenance (`mode=max`) + SBOM attestations are emitted via bake-action's `provenance`/`sbom` inputs, matching what `bin/docker_build.sh` already does for the manual path (#1350), so images published either way carry the same attestations. ## Verification The parts that can't be exercised without pushing were verified locally against a registry on buildx v0.35.0: - a target that is both published **and** consumed as a named context (`skdb-base` builds `FROM skip`) keeps its own push-by-digest output — settled at solve time, not just `bake --print`; - each per-platform push is an OCI index carrying its attestation manifest inline; - `imagetools create` preserves **both** platforms' attestations into the merged tag (verified: merged index = 2 platform manifests + 2 attestation manifests). `actionlint` (1.7.7) and `shellcheck` are clean on everything here. ## Notes for the reviewer - **Requires org secrets `DOCKERHUB_USERNAME` / `DOCKERHUB_TOKEN`** (scoped push token) before the first run — the repo has none today. The first run should be a watched `workflow_dispatch`, not the weekly schedule. - **Stacked on the actionlint fix** (first commit): CI currently installs actionlint **1.7.6** — the `v1.7.7` in the URL only selects the downloader script, which defaults to 1.7.6 — and 1.7.6 rejects `runs-on: ubuntu-24.04-arm`. That commit stands on its own but this workflow needs it to lint. - **Conscious tradeoff:** the workflow does not run CI against the freshly built images before overwriting `:latest`. That would roughly double the pipeline; the digest/attestation guards catch push-level failures but not "builds fine, breaks CI". `cancel-in-progress: false` and a watched first run are the mitigations. - Actions are pinned to major tags (`@v4`/`@v7`), matching repo convention (no dependabot). SHA-pinning would harden this first credential-holding workflow but the actions would then never update — happy to switch if preferred. - Expect `unknown/unknown` rows on Docker Hub for these images — that's how BuildKit stores attestation manifests, not a bug. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Docker Scout reports "Required supply chain attestations missing" on every
skiplabs/*image. Worse, without provenance Scout has to guess an image's origin — it printsBase image was auto-detectedand attributes findings to an inferred base rather than the real one. (That guessing is not theoretical: it produces confidently wrong base-image attributions.)This emits provenance (
mode=max) and SBOM attestations, using bake's native--provenance/--sbomshorthands.Scoped to push flows on purpose
Attestations require the docker-container driver — which is exactly what the named builder used for
--push/--push-onlyprovides. Local builds export via--loadon the default docker driver, which rejects them outright:So enabling them unconditionally — e.g. by putting
attestindocker-bake.hcl— would break every local build. Gating on push avoids that while still attesting everything we publish.Verified both directions with
bake --print:attest--provenance/--sbom)[{type: sbom}, {mode: max, type: provenance}]✅--load)Other drawbacks considered
mode=maxrecords build args, so a secret passed as a build arg would be embedded. The only ones passed here areSTAGEandSKIP_CAPACITY, neither sensitive. A comment records this so it stays true.unknown/unknownentries to the image index. Normaldocker pullignores them, and nothing in this repo parses manifest lists or iterates platforms, so no tooling is affected.skipruntimetarget exports with--output type=local(to extractlibskipruntime.so) and is never pushed, so it's unaffected.Test plan
bake --printconfirmsattestpresent on the push path, absent on the local pathdockerdriver rejects attestations (reproduced the error) — the reason for the gatingdocker-containerdriver produces them: exported an OCI index containingplatform=unknown/unknown type=attestation-manifestalongside the image manifestshellcheck --exclude SC2181 --exclude SC2002clean (matchesMakefile:104)Base image was auto-detectedno longer appearsRelates to #1348 (automating publishing — a GitHub Actions workflow would emit these natively).