Skip to content

ci(podvm): build tdx-snp-nvidia image variant#53

Open
yousef-cohere wants to merge 11 commits into
coherefrom
yousef/tdx-snp-nvidia-podvm
Open

ci(podvm): build tdx-snp-nvidia image variant#53
yousef-cohere wants to merge 11 commits into
coherefrom
yousef/tdx-snp-nvidia-podvm

Conversation

@yousef-cohere

Copy link
Copy Markdown

Summary

  • Switch the Cohere PodVM build to the tdx-snp-nvidia guest-components AA variant.
  • Pin guest-components to the PR commit that publishes TDX + SNP + Azure SNP-vTPM + NVIDIA attesters.
  • Derive GCP guest OS features from measurements.json and mark tdx-snp-nvidia images as both TDX and SEV-SNP capable.
  • Add an opt-in Azure Compute Gallery deploy workflow for confidential VM images.

Test plan

  • YAML parse passed for edited workflow/version files.
  • actionlint passed for edited GitHub Actions workflows.

Made with Cursor

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread .github/workflows/deploy-azure-cohere.yaml Fixed
Comment thread .github/workflows/deploy-azure-cohere.yaml Fixed
Comment thread .github/workflows/deploy-azure-cohere.yaml Fixed
Comment thread .github/workflows/deploy-azure-cohere.yaml Fixed
Comment thread .github/workflows/deploy-azure-cohere.yaml Fixed
Comment thread .github/workflows/deploy-azure-cohere.yaml
*)
echo "ERROR: unsupported tee_platform '$TEE_PLATFORM'" >&2
exit 1
;;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing legacy tee_platform mapping

Low Severity

GCP guest OS feature selection rejects any tee_platform not listed in the new case statement. Artifacts built before this change recorded tdx-nvidia in measurements.json, so manual redeploy of those GHCR tags via deploy-gcp-cohere.yaml now fails even though the prior workflow always published them as TDX-capable.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit edc4a45. Configure here.

Co-authored-by: Cursor <cursoragent@cursor.com>
if [ "$workflow_ref" != "refs/heads/main" ] && [ "$workflow_ref" != "refs/heads/cohere" ]; then
echo "Workflow ref mismatch: expected refs/heads/main or refs/heads/cohere, got $workflow_ref"
if [ "$workflow_ref" != "refs/heads/main" ] && [ "$workflow_ref" != "refs/heads/cohere" ] && [ "$workflow_ref" != "refs/heads/yousef/tdx-snp-nvidia-aa-v2" ]; then
echo "Workflow ref mismatch: expected refs/heads/main, refs/heads/cohere, or refs/heads/yousef/tdx-snp-nvidia-aa-v2, got $workflow_ref"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personal branch in provenance allowlist

High Severity

verify-provenance.sh now treats attestations from refs/heads/yousef/tdx-snp-nvidia-aa-v2 as trusted alongside main and cohere, widening supply-chain verification beyond intended release branches.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 84e6078. Configure here.

yousef-cohere and others added 2 commits June 30, 2026 17:45
Resolve broken-package build failure: the CUDA apt repo now serves
580.173.02-1ubuntu1, which conflicted with the pinned
580.159.04-1ubuntu1 transitive deps. Bump all four nvidia-* pins to
the currently available version.

Co-authored-by: Cursor <cursoragent@cursor.com>

@alhassankhedr-cohere alhassankhedr-cohere left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address github comments, but otherwise looks good.

Azure gallery image-version create rejects VHDs whose virtual size is
not a whole number of MiB ("unsupported virtual size"). mkosi's raw
image isn't necessarily 1 MiB-aligned, so round it up with qemu-img
resize before the fixed-VHD conversion; force_size preserves the
aligned size in the footer.

Signed-off-by: Alhassan Khedr <alhassan.khedr@cohere.com>
The initrd only bundled virtio/nvme/SATA storage drivers, so on Azure
(Hyper-V Gen2, all devices on VMBus) the guest never enumerates its OS
disk: it hangs in the initrd waiting for the dm-verity root partitions,
times out after 90s, and drops to an emergency shell. Confirmed via the
peer pod VM serial console (no hv_vmbus/hv_storvsc, "PCI: System does
not support PCI", "Timed out waiting for dev-mapper-root.device").

Add hv_vmbus (VMBus core), hv_storvsc (OS disk), hv_netvsc (networking),
and pci_hyperv (VMBus PCI for GPU passthrough) so the image boots on
Azure while retaining virtio for GCP/KVM.

Signed-off-by: Alhassan Khedr <alhassan.khedr@cohere.com>
…letes

The Ubuntu podvm image enables afterburn-checkin.service (via the shared
preset + drop-in) but never shipped afterburn: it is not packaged for
Ubuntu/Debian (only Fedora) and upstream publishes no prebuilt binary. As a
result the peer-pod VM boots fine but never runs afterburn --check-in, so it
never reports provisioning completion to the Azure fabric. Azure keeps the VM
at provisioningState=Creating and the CAA CreateInstance LRO times out,
leaving pods stuck in ContainerCreating.

Build afterburn from source in the Ubuntu binaries image (builder and target
are both ubuntu:24.04, so the dynamic glibc/OpenSSL link is ABI-compatible)
and ship both the binary and the upstream base afterburn-checkin.service unit
(on Fedora that unit comes from the RPM). Add openssl to the package set for
the libssl3 runtime dependency.
…no inline expansion)

Clears the zizmor findings on the Azure publish workflow so PR #53 can merge
cleanly into cohere:

- move the workflow-level permissions (incl. id-token: write) to job level and
  set top-level `permissions: {}` (fixes the excessive-permissions error)
- pass caa_commit through an env var instead of inlining ${{ }} in run:
  (fixes the template-injection finding)
- correct the SHA-pin version comments to the exact tags (checkout v6.0.2,
  docker/login-action v4.1.0)
- add a per-image-tag job concurrency guard so overlapping publishes can't race
  on the same immutable gallery version
…ket is up

api-server-rest.path only gates on the CDH socket, but api-server-rest
--features all also dials the attestation-agent socket at startup. On the
cc-nvidia image the attestation-agent creates its socket a few seconds late
(large binary + NVIDIA attester init), so api-server-rest exhausts systemd's
default 5-starts/10s limit and fails permanently ("Start request repeated too
quickly"), leaving the attestation endpoint on 127.0.0.1:8006 down for the
VM's whole lifetime.

Add a drop-in that disables the start-rate give-up (StartLimitIntervalSec=0)
so Restart=always keeps retrying, plus an ExecStartPre that blocks until both
the attestation-agent and CDH sockets exist to avoid the noisy failed-start
loop.
Resolves the three Go stdlib advisories flagging PR #53's govulncheck job:
GO-2026-5037 (crypto/x509), GO-2026-5038 (mime), GO-2026-5039 (net/textproto),
all "Fixed in go1.25.11". Bumps versions.yaml (CI setup-go), the go directive in
all five modules, and the pinned golang builder base image digest so shipped
binaries are also built against the patched stdlib.

Validated locally with GOTOOLCHAIN=go1.25.11: govulncheck reports no
vulnerabilities for peerpod-ctrl, csi-wrapper, and webhook.
Clears the three fixable containerd advisories flagged by govulncheck:
GO-2026-5378 (fixed 1.7.32), GO-2026-5475 and GO-2026-5758 (fixed 1.7.33).
Also pulls opencontainers/selinux 1.13.0 -> 1.13.1 transitively.

The remaining govulncheck findings (docker/docker + 3 containerd advisories)
have no upstream fix (Fixed in: N/A), are unrelated to this PR, and exist on
the base branch's identical dependency versions.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6f4974a. Configure here.

# patch is time-based, so overlapping runs could collide on the same version.
concurrency:
group: deploy-azure-cohere-${{ inputs.image_tag }}
cancel-in-progress: false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gallery version collision across tags

Medium Severity

Azure gallery versions use minute-granularity Major.Minor.Patch, but concurrency is keyed only on inputs.image_tag, so two deploy jobs for different tags that share the same image_family can run in parallel and fight over the same immutable version string.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6f4974a. Configure here.

# glibc/OpenSSL as the target image, so the dynamic link is ABI-compatible.
FROM builder AS afterburn_builder
ARG AFTERBURN_VERSION=5.10.0
RUN curl -fsSL https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Agentic Security Review
Severity: MEDIUM
afterburn_builder bootstraps Rust via curl -fsSL https://sh.rustup.rs | sh and then builds from a mutable git tag (v${AFTERBURN_VERSION}) without an integrity check or immutable commit pin.

This weakens build-time supply-chain integrity: if the installer channel or tag resolution is compromised, attacker-controlled build inputs can be incorporated into the generated PodVM image artifacts.

Fix in Cursor Fix in Web

Reviewed by Cursor Security Reviewer for commit 6f4974a. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants