Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/en/learn/supported-configurations-for-2.x.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This page lists the currently maintained Alauda AI versions in the component mat
| Alauda Build of DCGM-Exporter | Cluster Plugin | v4.2.3-413-1 | v4.2.3-413-1 |
| Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.8.3 |
| Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.0 |
| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.17.4 | v0.17.4 |
| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.18.3 | v0.17.4 |
| Alauda Build of Kueue | Cluster Plugin | v0.17.0 | v0.17.0 |
| Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | v0.8.0-1 |
| Alauda Build of JobSet (1) | Operator | - | v0.12.0 |
Expand Down Expand Up @@ -63,7 +63,7 @@ This page lists the currently maintained Alauda AI versions in the component mat
| Alauda Build of NPU Operator (3) | Cluster Plugin | v1.1.3 | v1.2.4 |
| Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.8.3 |
| Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.0 |
| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.17.4 | v0.17.4 |
| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.18.3 | v0.17.4 |
| Alauda Build of Kueue | Cluster Plugin | v0.17.0 | v0.17.0 |
| Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | v0.8.0-1 |
| Alauda Build of JobSet (1) | Operator | - | v0.12.0 |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,57 +18,74 @@ In a Kubernetes cluster, inconsistencies in GPU models and CUDA driver versions

3. High maintenance overhead: Manually managing the CUDA version dependencies between nodes and applications increases operational complexity.

This document provides a step-by-step guide for scheduling inference services based on the CUDA runtime version and Nvidia Driver version.
With these settings, you can resolve the CUDA Runtime and CUDA Driver version mismatch at the Kubernetes scheduling level to ensure that applications are scheduled to compatible GPU nodes.

This document provides a step-by-step guide for scheduling inference services
based on the CUDA compatibility version reported by the NVIDIA driver. With
these settings, you can reduce CUDA application and driver incompatibility by
scheduling workloads to compatible GPU nodes.

## Steps

<Steps>
### Adding CUDA version in node labels

1. On each GPU node, run the following command to retrieve the supported CUDA runtime version:
1. On each GPU node, run the following command to retrieve the CUDA
compatibility version reported by the NVIDIA driver:

```bash
nvidia-smi | sed -n 's/.*CUDA Version: \([0-9.]\+\).*/\1/p'
```
For example, the output might be 12.4.

For example, the output might be 12.4.

:::note
The **CUDA Version** displayed by `nvidia-smi` is the latest CUDA version
supported by the installed NVIDIA driver. It does not confirm that a CUDA
toolkit or runtime of that version is installed on the node.
:::

2. On the control node, label the GPU node with the corresponding major and minor version:
```bash
kubectl label node <node-name> \
nvidia.com/cuda.runtime.major=12 \
nvidia.com/cuda.runtime.minor=4
```
```bash
kubectl label node <node-name> \
nvidia.com/cuda.runtime.major=12 \
nvidia.com/cuda.runtime.minor=4
```

:::tip
If your cluster has many GPU nodes, it is difficult to label them manually. You can install the `Node Feature Discovery` cluster plugin.
By deploying the Node Feature Discovery(NFD) cluster plugin and turning on the GFD extension, GPU nodes will automatically be labeled with the CUDA version.
`Node Feature Discovery` cluster plugin can be retrieved from Customer Portal. Please contact Consumer Support for more information.
For clusters with many GPU nodes, install
[Alauda Build of Node Feature Discovery](../../../nfd/install) and enable its
**gfd Extension**. The GFD extension generates the
`nvidia.com/cuda.runtime.major` and `nvidia.com/cuda.runtime.minor` labels from
the CUDA compatibility version reported by the NVIDIA driver.
:::

### Schedule inference services based on the CUDA version

#### Scheduling Inference Services based on the CUDA version
Starting from Alauda AI 1.5, the product will automatically schedule pod of inference services by the CUDA version. For earlier versions, you can follow the following steps:
1. Determine which ClusterServingRuntime you need to select when creating an inference service.
2. Parse the ClusterServingRuntime label:
If `cpaas.io/accelerator-type` is nvidia, further parse `cpaas.io/cuda-version` (example 11.8).
3. Add nodeAffinity field in the inference service, example:
```yaml
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
spec:
predictor:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: nvidia.com/cuda.runtime.major
operator: In
values: ["11"]
- key: nvidia.com/cuda.runtime.minor
operator: Gt
values: ["7"] # Since the k8s operator only supports Gt, which means greater than but not equal to, we use the rt version minus one to meet the requirements.
```
Alauda AI 1.5 and later automatically schedule inference service pods based on
the CUDA version. For earlier versions, complete the following steps:

1. Determine which ClusterServingRuntime you need to select when creating an inference service.
2. Parse the ClusterServingRuntime label:
If `cpaas.io/accelerator-type` is `nvidia`, also read the
`cpaas.io/cuda-version` label, for example `11.8`.
3. Add a `nodeAffinity` field to the inference service. The following example
selects nodes reporting CUDA major version 11 and minor version 8 or later:
```yaml
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
spec:
predictor:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: nvidia.com/cuda.runtime.major
operator: In
values: ['11']
- key: nvidia.com/cuda.runtime.minor
operator: Gt
values: ['7'] # Gt is strict, so 7 expresses a minimum minor version of 8.
```

</Steps>
7 changes: 7 additions & 0 deletions docs/en/nfd/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
weight: 91
---

# Alauda Build of Node Feature Discovery

<Overview />
79 changes: 79 additions & 0 deletions docs/en/nfd/install.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
weight: 20
---

# Install Node Feature Discovery

## Supported configurations

The `v0.18.3` Cluster Plugin package is published for:

| Requirement | Supported value |
| ------------------------- | ---------------------------------- |
| Alauda AI | v2.3 Stable |
| Alauda Container Platform | v4.0.x, v4.1.x, v4.2.x, and v4.3.x |
| Architecture | x86_64 (amd64) and ARM64 (arm64) |

## Download the Cluster Plugin

:::info

The **Alauda Build of Node Feature Discovery** Cluster Plugin can be retrieved
from Customer Portal. Contact Customer Support for more information.

:::

## Upload the Cluster Plugin

For more information, see the following documentation:

<ExternalSiteLink
name="acp"
href="extend/upload_package.html#upload-a-cluster-plugin"
children="Uploading Cluster Plugins"
/>

## Install the Cluster Plugin

1. Go to **Administrator** > **Marketplace** > **Cluster Plugins**.
2. Switch to the target cluster and deploy **Alauda Build of Node
Feature Discovery**.
3. Keep **gfd Extension** disabled unless NVIDIA GPU or CUDA-related discovery
labels are required.
4. Wait until the plugin status is **Installed**.

## Verify the installation

Verify that the NFD master, worker, and garbage collector workloads are ready:

```bash
kubectl -n kube-system get deployment node-feature-discovery-master node-feature-discovery-gc
kubectl -n kube-system get daemonset node-feature-discovery-worker
```

Verify that NFD is publishing discovered features:

```bash
kubectl get nodefeatures.nfd.k8s-sigs.io --all-namespaces
kubectl get node <node-name> --show-labels
```

The target node should contain labels with the
`feature.node.kubernetes.io/` prefix. If the GFD extension is enabled, the node
can also contain NVIDIA GPU or CUDA-related discovery labels.

## Upgrade to v0.18.3

Version `v0.18.3` updates the upstream NFD base and provides Alauda
multi-architecture images. The existing optional GFD integration is preserved,
and no user configuration migration is required.

1. Upload the `v0.18.3` plugin package.
2. Go to **Administrator** > **Clusters** > **Target Cluster** > **Functional
Components**.
3. Upgrade **Alauda Build of Node Feature Discovery** to `v0.18.3`.
4. Repeat the installation verification commands.

The NFD upgrade does not change separately managed accelerator driver or Device
Plugin versions. After the upgrade, verify that the node labels required by
your workloads remain present.
45 changes: 45 additions & 0 deletions docs/en/nfd/intro.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
weight: 10
---

# Introduction

**Alauda Build of Node Feature Discovery (NFD)** is based on the Kubernetes SIG
[Node Feature Discovery](https://kubernetes-sigs.github.io/node-feature-discovery/v0.18/)
project. It detects hardware features and system configuration on Kubernetes
nodes and publishes the discovered information as node labels and `NodeFeature`
resources.

NFD provides a shared node-discovery layer for AI workloads and other
components that select nodes by operating system, architecture, kernel, CPU,
PCI, or other hardware characteristics.

## Scope

NFD discovers and labels node capabilities. It does not:

- install or manage hardware drivers;
- expose GPU or NPU resources to Kubernetes;
- replace a vendor Device Plugin, DRA driver, or accelerator operator.

Install the plugin once in each target cluster that requires these node labels.
Other Cluster Plugins and workloads can consume NFD labels, while drivers and
resource-management components remain independently installed and managed.

## Optional GFD extension

The deployment form includes a **gfd Extension** switch, which is disabled by
default. Enable it only when workloads or other components need additional
NVIDIA GPU and CUDA-related labels.

The GFD extension adds discovery labels. It does not install the NVIDIA driver
or replace the NVIDIA GPU Device Plugin.

For platform installation and upgrade instructions, see
[Install Node Feature Discovery](./install).

## Related information

- [Supported configurations](../learn/supported-configurations-for-2.x)
- [Schedule inference services based on the CUDA version](../model_inference/inference_service/how_to/accurately_schedule)
- [Upstream Node Feature Discovery documentation](https://kubernetes-sigs.github.io/node-feature-discovery/v0.18/)