Add GENERATED package lock mode (pnpm/npm style, commit-pinned)#707
Add GENERATED package lock mode (pnpm/npm style, commit-pinned)#707jamierpond wants to merge 2 commits into
Conversation
`CPMUsePackageLock(<file> GENERATED)` (or the `CPM_GENERATE_PACKAGE_LOCK` option) keeps the lock in sync automatically, the way `pnpm-lock.yaml` / `package-lock.json` do: - The lock is (re)written into the source tree on every configure, so there is no separate `cpm-update-package-lock` step. Delete it and reconfigure to refresh, just like deleting a pnpm lock. - Git packages are pinned to the exact commit that was checked out rather than the (possibly moving) GIT_TAG, making the lock reproducible even when deps are declared against a branch like `main`. To resolve the commit, the lock entry is now written after the fetch (so the source dir exists) and `cpm_get_git_commit_hash` reads the checked-out HEAD. Non-generated behaviour is unchanged. Adds a hermetic unit test (local throwaway git repo, no network) covering auto-generation, commit pinning, and idempotency, plus README docs.
There was a problem hiding this comment.
Thanks for this PR, automatically locking the current commit hash is a big step for CPM's supply chain security and reproducibility - especially when moving targets are specified! I have some thoughts regarding using a auto-updating package lock in nested CPM dependencies and documentation, but looks great otherwise!
| find_package(Git QUIET) | ||
| if(NOT GIT_EXECUTABLE OR NOT EXISTS "${repoPath}/.git") | ||
| return() | ||
| endif() |
There was a problem hiding this comment.
In case of nested CPM dependencies, the value of ${result} may still be set in the parent scope leading to an incorrect resolution, so we should unset ${result} in the parent scope here.
| # Includes the package lock file if it exists and creates a target `cpm-update-package-lock` to | ||
| # update it. |
There was a problem hiding this comment.
The docstring is no longer up-to-date: the cpm-update-package-lock target is no longer created when GENERATED or CPM_GENERATE_PACKAGE_LOCK is set.
| # Author the lock directly in the source tree and keep it current on every configure. | ||
| set(CPM_PACKAGE_LOCK_GENERATED | ||
| TRUE | ||
| CACHE INTERNAL "" | ||
| ) | ||
| set(CPM_PACKAGE_LOCK_FILE | ||
| "${CPM_ABSOLUTE_PACKAGE_LOCK_PATH}" | ||
| CACHE INTERNAL "" | ||
| ) | ||
| file( | ||
| WRITE ${CPM_PACKAGE_LOCK_FILE} | ||
| "# CPM Package Lock\n# This file is generated by CPM.cmake and should be committed to version control\n\n" | ||
| ) |
There was a problem hiding this comment.
It's a bit unclear to me what will happen if a dependency also uses CPM with a package lock file, I think this might overwrite the existing package lock again. I think calling CPMUsePackageLock should be a no-op if it has already been called at any point before.
| ### Generated package lock (pnpm/npm style) | ||
|
|
||
| Pass `GENERATED` to keep the lock in sync automatically, the way `pnpm-lock.yaml` or | ||
| `package-lock.json` work: | ||
|
|
||
| ```cmake | ||
| CPMUsePackageLock(package-lock.cmake GENERATED) | ||
| ``` | ||
|
|
||
| In this mode the lock at the given path is rewritten in your source tree on **every configure**, | ||
| and git packages are pinned to the **exact commit that was checked out** rather than the (possibly | ||
| moving) `GIT_TAG`. This makes the lock reproducible even when dependencies are declared against a | ||
| branch like `main`. No separate `cpm-update-package-lock` step is needed. | ||
|
|
||
| To refresh the pins (e.g. to pull in newer upstream commits), delete the lock file and reconfigure, | ||
| just like deleting a `pnpm-lock.yaml`: | ||
|
|
||
| ```bash | ||
| rm package-lock.cmake | ||
| cmake -Bbuild | ||
| ``` | ||
|
|
||
| The same behaviour can be enabled globally without editing `CMakeLists.txt` by setting the | ||
| `CPM_GENERATE_PACKAGE_LOCK` option (or environment variable). |
There was a problem hiding this comment.
Thanks for adding documentation! We should add that also changing any additional arguments like the OPTIONS will not work without removing the lockfile or the entry from the lockfile first.
Maybe it would also make sense to mark this as experimental for now, as for a proper npm-style lock one might expect.
- Changing the version or explicitly setting a commit hash in the source would also update the locked hash accordingly
- Any other flags from
CPMAddPackagewould still be respected, the lock should only impact theGIT_TAGandGIT_REPOSITORYentries.
Implementing this in the future could be a breaking change or lead to unexpected updates, so it makes sense to advertise caution for now.
|
@TheLartians thanks so much for looking at this! Meant for it to be marked as draft initially, am very grateful for your feedback. Thanks again for giving this serious consideration, will revise with your feedback in mind! |
Motivation
The current package lock records each dependency's declaration verbatim and only updates when you build the
cpm-update-package-locktarget. For projects that declare deps against a moving ref (e.g.GIT_TAG main), this has two rough edges compared topnpm-lock.yaml/package-lock.json:GIT_TAG main, not the commit that was actually used.What this adds
A
GENERATEDkeyword onCPMUsePackageLock(and an equivalentCPM_GENERATE_PACKAGE_LOCKoption / env var):In this mode:
cpm-update-package-lockstep. Delete it and reconfigure to refresh the pins, just like deleting a pnpm lock.GIT_TAG. The pin is then consumed on the next configure (declarations override call-site args), so it's stable/idempotent:main→ resolved commit on the first run, identical lock thereafter.Existing (non-
GENERATED) behaviour is unchanged.Implementation notes
cpm_get_git_commit_hash()reads the checked-outHEAD(falls back gracefully — no git / not a repo / URL packages → unchanged args).CPMUsePackageLocknowcmake_parse_argumentsfor theGENERATEDflag, includes any existing lock first (to apply current pins), then redirectsCPM_PACKAGE_LOCK_FILEat the source path and rewrites it.Tests
Adds
test/unit/package-lock-generated.cmake— hermetic (spins up a throwaway local git repo, no network) — asserting that after a plain configure (no target build) the lock is written into the source tree, pins the dependency to the resolved commit (and no longer references the branch), and is byte-identical across a reconfigure.Formatted with the repo's
cmake-format0.6.11 config; existing package-lock unit tests still pass.