Skip to content

GCP - Add exception for onHostMaintenance Migrate on confidential compute n2d instances #1406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
WTFKr0 wants to merge 3 commits into
base: main
Choose a base branch
from
+25 −4
Open

GCP - Add exception for onHostMaintenance Migrate on confidential compute n2d instances #1406

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4b68515
Add exception for onHostMaintenance Migrate on confidential compute n…
WTFKr0 Aug 22, 2025
32563f3
GCP: adapt tests for onHostMaintenance Migrate on confidential comput…
WTFKr0 Aug 22, 2025
a824403
Add failure for ConfidentialComputePolicySEVSNP and OnHostMaintenance…
WTFKr0 Sep 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions pkg/webhooks/machine_webhook.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,10 @@ var gcpConfidentialTypeMachineSeriesSupportingSEV = []string{"n2d", "c2d", "c3d"
var gcpConfidentialTypeMachineSeriesSupportingSEVSNP = []string{"n2d"}
var gcpConfidentialTypeMachineSeriesSupportingTDX = []string{"c3"}

// GCP onHostMaintenance Migrate with Confidential Compute is supported only on certain series:
// reference: https://cloud.google.com/confidential-computing/confidential-vm/docs/troubleshoot-live-migration
var gcpConfidentialTypeMachineSeriesSupportingOnHostMaintenanceMigrate = []string{"n2d"}

@bgartzi bgartzi Jul 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c3d is also supported now.


// defaultInstanceTypeForCloudProvider returns the default instance type for the given cloud provider and architecture.
// If the cloud provider is not supported, an empty string is returned.
// If the architecture is not supported, the default instance type for AMD64 is returned as a fallback.
Expand Down Expand Up @@ -1325,14 +1329,15 @@ func validateShieldedInstanceConfig(providerSpec *machinev1beta1.GCPMachineProvi
func validateGCPConfidentialComputing(providerSpec *machinev1beta1.GCPMachineProviderSpec) field.ErrorList {
var errs field.ErrorList
if providerSpec.ConfidentialCompute != "" && providerSpec.ConfidentialCompute != machinev1beta1.ConfidentialComputePolicyDisabled {
// Get machine series
machineSeries := strings.Split(providerSpec.MachineType, "-")[0]
// Check on host maintenance
if providerSpec.OnHostMaintenance != machinev1beta1.TerminateHostMaintenanceType {
if providerSpec.OnHostMaintenance != machinev1beta1.TerminateHostMaintenanceType && !slices.Contains(gcpConfidentialTypeMachineSeriesSupportingOnHostMaintenanceMigrate, machineSeries) {

@bgartzi bgartzi Sep 11, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition would let SEV-SNP n2d machines to be configured with OnHostMaintenance: Migrate. However, that is not supported by GCP.

Could you rewrite the condition so this configuration is only accepted for AMD-SEV? (i.e. providerSpec.ConfidentialCompute == Enabled OR AMDEncryptedVirtualization)

@WTFKr0 WTFKr0 Sep 21, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK i see
Need this test to fail

		{
			testCase: "with ConfidentialCompute AMDEncryptedVirtualizationNestedPaging and onHostMaintenance set to Migrate on n2d instances",
			modifySpec: func(p *machinev1beta1.GCPMachineProviderSpec) {
				p.ConfidentialCompute = machinev1beta1.ConfidentialComputePolicySEVSNP
				p.OnHostMaintenance = machinev1beta1.MigrateHostMaintenanceType
				p.MachineType = "n2d-standard-4"
				p.GPUs = []machinev1beta1.GCPGPUConfig{}
			},
			expectedOk: true,
		},

@WTFKr0 WTFKr0 Sep 21, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK with last commit i think

@bgartzi bgartzi Jul 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Machine types should be only checked for SEV:

Suggested change
if providerSpec.OnHostMaintenance != machinev1beta1.TerminateHostMaintenanceType && !slices.Contains(gcpConfidentialTypeMachineSeriesSupportingOnHostMaintenanceMigrate, machineSeries) {
isSev := providerSpec.ConfidentialCompute == machinev1beta1.ConfidentialComputePolicyEnabled || providerSpec.ConfidentialCompute == machinev1beta1.ConfidentialComputePolicySEV
if providerSpec.OnHostMaintenance != machinev1beta1.TerminateHostMaintenanceType && !(isSev && slices.Contains(gcpConfidentialTypeMachineSeriesSupportingOnHostMaintenanceMigrate, machineSeries)) {

Or something like that.

This way we can also get rid of the SEV-SNP specific check for maintenance type below.

errs = append(errs, field.Invalid(field.NewPath("providerSpec", "onHostMaintenance"),
providerSpec.OnHostMaintenance,
fmt.Sprintf("ConfidentialCompute %s requires OnHostMaintenance to be set to %s, the current value is: %s", providerSpec.ConfidentialCompute, machinev1beta1.TerminateHostMaintenanceType, providerSpec.OnHostMaintenance)))
}
// Check machine series supports confidential computing
machineSeries := strings.Split(providerSpec.MachineType, "-")[0]
switch providerSpec.ConfidentialCompute {
case machinev1beta1.ConfidentialComputePolicyEnabled, machinev1beta1.ConfidentialComputePolicySEV:
if !slices.Contains(gcpConfidentialTypeMachineSeriesSupportingSEV, machineSeries) {
Expand All @@ -1348,6 +1353,12 @@ func validateGCPConfidentialComputing(providerSpec *machinev1beta1.GCPMachinePro
fmt.Sprintf("ConfidentialCompute %s requires a machine type in the following series: %s", providerSpec.ConfidentialCompute, strings.Join(gcpConfidentialTypeMachineSeriesSupportingSEVSNP, `,`))),
)
}
// Check on host maintenance for ConfidentialComputePolicySEVSNP
if providerSpec.OnHostMaintenance != machinev1beta1.TerminateHostMaintenanceType {
errs = append(errs, field.Invalid(field.NewPath("providerSpec", "onHostMaintenance"),
providerSpec.OnHostMaintenance,
fmt.Sprintf("ConfidentialCompute %s requires OnHostMaintenance to be set to %s, the current value is: %s", providerSpec.ConfidentialCompute, machinev1beta1.TerminateHostMaintenanceType, providerSpec.OnHostMaintenance)))
}
case machinev1beta1.ConfidentialComputePolicyTDX:
if !slices.Contains(gcpConfidentialTypeMachineSeriesSupportingTDX, machineSeries) {
errs = append(errs, field.Invalid(field.NewPath("providerSpec", "machineType"),
Expand Down
14 changes: 12 additions & 2 deletions pkg/webhooks/machine_webhook_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3934,13 +3934,23 @@ func TestValidateGCPProviderSpec(t *testing.T) {
expectedError: "providerSpec.confidentialCompute: Invalid value: \"invalid-value\": ConfidentialCompute must be Enabled, Disabled, AMDEncryptedVirtualization, AMDEncryptedVirtualizationNestedPaging, or IntelTrustedDomainExtensions",
},
{
testCase: "with ConfidentialCompute enabled while onHostMaintenance is set to Migrate",
testCase: "with ConfidentialCompute enabled while onHostMaintenance is set to Migrate on n2d instances",
modifySpec: func(p *machinev1beta1.GCPMachineProviderSpec) {
p.ConfidentialCompute = machinev1beta1.ConfidentialComputePolicyEnabled
p.OnHostMaintenance = machinev1beta1.MigrateHostMaintenanceType
p.MachineType = "n2d-standard-4"
p.GPUs = []machinev1beta1.GCPGPUConfig{}
},
expectedOk: true,
},
{
testCase: "with ConfidentialCompute enabled while onHostMaintenance is set to Migrate on non n2d instances (c2d)",
modifySpec: func(p *machinev1beta1.GCPMachineProviderSpec) {
p.ConfidentialCompute = machinev1beta1.ConfidentialComputePolicyEnabled
p.OnHostMaintenance = machinev1beta1.MigrateHostMaintenanceType
p.MachineType = "c2d-standard-4"
p.GPUs = []machinev1beta1.GCPGPUConfig{}
},
expectedOk: false,
expectedError: "providerSpec.onHostMaintenance: Invalid value: \"Migrate\": ConfidentialCompute Enabled requires OnHostMaintenance to be set to Terminate, the current value is: Migrate",
},
Expand Down Expand Up @@ -4006,7 +4016,7 @@ func TestValidateGCPProviderSpec(t *testing.T) {
expectedError: "",
},
{
testCase: "with ConfidentialCompute AMDEncryptedVirtualizationNestedPaging and onHostMaintenance set to Migrate",
testCase: "with ConfidentialCompute AMDEncryptedVirtualizationNestedPaging and onHostMaintenance set to Migrate on n2d instances",
modifySpec: func(p *machinev1beta1.GCPMachineProviderSpec) {
p.ConfidentialCompute = machinev1beta1.ConfidentialComputePolicySEVSNP
p.OnHostMaintenance = machinev1beta1.MigrateHostMaintenanceType
Expand Down