diff --git a/docs/en/learn/supported-configurations-for-2.x.md b/docs/en/learn/supported-configurations-for-2.x.md index 5aeceef5..df091b4f 100644 --- a/docs/en/learn/supported-configurations-for-2.x.md +++ b/docs/en/learn/supported-configurations-for-2.x.md @@ -22,7 +22,7 @@ This page lists the currently maintained Alauda AI versions in the component mat | Alauda Build of DCGM-Exporter | Cluster Plugin | v4.2.3-413-1 | v4.2.3-413-1 | | Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.8.3 | | Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.0 | -| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.17.4 | v0.17.4 | +| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.18.3 | v0.17.4 | | Alauda Build of Kueue | Cluster Plugin | v0.17.0 | v0.17.0 | | Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | v0.8.0-1 | | Alauda Build of JobSet (1) | Operator | - | v0.12.0 | @@ -63,7 +63,7 @@ This page lists the currently maintained Alauda AI versions in the component mat | Alauda Build of NPU Operator (3) | Cluster Plugin | v1.1.3 | v1.2.4 | | Alauda Build of HAMi | Cluster Plugin | v2.8.1 | v2.8.3 | | Alauda Build of HAMi-WebUI | Cluster Plugin | v1.10.0 | v1.10.0 | -| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.17.4 | v0.17.4 | +| Alauda Build of Node Feature Discovery | Cluster Plugin | v0.18.3 | v0.17.4 | | Alauda Build of Kueue | Cluster Plugin | v0.17.0 | v0.17.0 | | Alauda Build of LeaderWorkerSet | Cluster Plugin | v0.8.0-1 | v0.8.0-1 | | Alauda Build of JobSet (1) | Operator | - | v0.12.0 | diff --git a/docs/en/model_inference/inference_service/how_to/accurately_schedule.mdx b/docs/en/model_inference/inference_service/how_to/accurately_schedule.mdx index 3ea018d0..8e1378ba 100644 --- a/docs/en/model_inference/inference_service/how_to/accurately_schedule.mdx +++ b/docs/en/model_inference/inference_service/how_to/accurately_schedule.mdx @@ -18,57 +18,74 @@ In a Kubernetes cluster, inconsistencies in GPU models and CUDA driver versions 3. High maintenance overhead: Manually managing the CUDA version dependencies between nodes and applications increases operational complexity. -This document provides a step-by-step guide for scheduling inference services based on the CUDA runtime version and Nvidia Driver version. -With these settings, you can resolve the CUDA Runtime and CUDA Driver version mismatch at the Kubernetes scheduling level to ensure that applications are scheduled to compatible GPU nodes. - +This document provides a step-by-step guide for scheduling inference services +based on the CUDA compatibility version reported by the NVIDIA driver. With +these settings, you can reduce CUDA application and driver incompatibility by +scheduling workloads to compatible GPU nodes. ## Steps + ### Adding CUDA version in node labels -1. On each GPU node, run the following command to retrieve the supported CUDA runtime version: +1. On each GPU node, run the following command to retrieve the CUDA + compatibility version reported by the NVIDIA driver: + ```bash nvidia-smi | sed -n 's/.*CUDA Version: \([0-9.]\+\).*/\1/p' ``` - For example, the output might be 12.4. + + For example, the output might be 12.4. + + :::note + The **CUDA Version** displayed by `nvidia-smi` is the latest CUDA version + supported by the installed NVIDIA driver. It does not confirm that a CUDA + toolkit or runtime of that version is installed on the node. + ::: 2. On the control node, label the GPU node with the corresponding major and minor version: - ```bash - kubectl label node \ - nvidia.com/cuda.runtime.major=12 \ - nvidia.com/cuda.runtime.minor=4 - ``` + ```bash + kubectl label node \ + nvidia.com/cuda.runtime.major=12 \ + nvidia.com/cuda.runtime.minor=4 + ``` :::tip -If your cluster has many GPU nodes, it is difficult to label them manually. You can install the `Node Feature Discovery` cluster plugin. -By deploying the Node Feature Discovery(NFD) cluster plugin and turning on the GFD extension, GPU nodes will automatically be labeled with the CUDA version. -`Node Feature Discovery` cluster plugin can be retrieved from Customer Portal. Please contact Consumer Support for more information. +For clusters with many GPU nodes, install +[Alauda Build of Node Feature Discovery](../../../nfd/install) and enable its +**gfd Extension**. The GFD extension generates the +`nvidia.com/cuda.runtime.major` and `nvidia.com/cuda.runtime.minor` labels from +the CUDA compatibility version reported by the NVIDIA driver. ::: +### Schedule inference services based on the CUDA version -#### Scheduling Inference Services based on the CUDA version - Starting from Alauda AI 1.5, the product will automatically schedule pod of inference services by the CUDA version. For earlier versions, you can follow the following steps: - 1. Determine which ClusterServingRuntime you need to select when creating an inference service. - 2. Parse the ClusterServingRuntime label: - If `cpaas.io/accelerator-type` is nvidia, further parse `cpaas.io/cuda-version` (example 11.8). - 3. Add nodeAffinity field in the inference service, example: - ```yaml - apiVersion: serving.kserve.io/v1beta1 - kind: InferenceService - spec: - predictor: - affinity: - nodeAffinity: - preferredDuringSchedulingIgnoredDuringExecution: - - weight: 100 - preference: - matchExpressions: - - key: nvidia.com/cuda.runtime.major - operator: In - values: ["11"] - - key: nvidia.com/cuda.runtime.minor - operator: Gt - values: ["7"] # Since the k8s operator only supports Gt, which means greater than but not equal to, we use the rt version minus one to meet the requirements. - ``` +Alauda AI 1.5 and later automatically schedule inference service pods based on +the CUDA version. For earlier versions, complete the following steps: + +1. Determine which ClusterServingRuntime you need to select when creating an inference service. +2. Parse the ClusterServingRuntime label: + If `cpaas.io/accelerator-type` is `nvidia`, also read the + `cpaas.io/cuda-version` label, for example `11.8`. +3. Add a `nodeAffinity` field to the inference service. The following example + selects nodes reporting CUDA major version 11 and minor version 8 or later: + ```yaml + apiVersion: serving.kserve.io/v1beta1 + kind: InferenceService + spec: + predictor: + affinity: + nodeAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 100 + preference: + matchExpressions: + - key: nvidia.com/cuda.runtime.major + operator: In + values: ['11'] + - key: nvidia.com/cuda.runtime.minor + operator: Gt + values: ['7'] # Gt is strict, so 7 expresses a minimum minor version of 8. + ``` diff --git a/docs/en/nfd/index.mdx b/docs/en/nfd/index.mdx new file mode 100644 index 00000000..fd3c7e9a --- /dev/null +++ b/docs/en/nfd/index.mdx @@ -0,0 +1,7 @@ +--- +weight: 91 +--- + +# Alauda Build of Node Feature Discovery + + diff --git a/docs/en/nfd/install.mdx b/docs/en/nfd/install.mdx new file mode 100644 index 00000000..d52b78d5 --- /dev/null +++ b/docs/en/nfd/install.mdx @@ -0,0 +1,79 @@ +--- +weight: 20 +--- + +# Install Node Feature Discovery + +## Supported configurations + +The `v0.18.3` Cluster Plugin package is published for: + +| Requirement | Supported value | +| ------------------------- | ---------------------------------- | +| Alauda AI | v2.3 Stable | +| Alauda Container Platform | v4.0.x, v4.1.x, v4.2.x, and v4.3.x | +| Architecture | x86_64 (amd64) and ARM64 (arm64) | + +## Download the Cluster Plugin + +:::info + +The **Alauda Build of Node Feature Discovery** Cluster Plugin can be retrieved +from Customer Portal. Contact Customer Support for more information. + +::: + +## Upload the Cluster Plugin + +For more information, see the following documentation: + + + +## Install the Cluster Plugin + +1. Go to **Administrator** > **Marketplace** > **Cluster Plugins**. +2. Switch to the target cluster and deploy **Alauda Build of Node + Feature Discovery**. +3. Keep **gfd Extension** disabled unless NVIDIA GPU or CUDA-related discovery + labels are required. +4. Wait until the plugin status is **Installed**. + +## Verify the installation + +Verify that the NFD master, worker, and garbage collector workloads are ready: + +```bash +kubectl -n kube-system get deployment node-feature-discovery-master node-feature-discovery-gc +kubectl -n kube-system get daemonset node-feature-discovery-worker +``` + +Verify that NFD is publishing discovered features: + +```bash +kubectl get nodefeatures.nfd.k8s-sigs.io --all-namespaces +kubectl get node --show-labels +``` + +The target node should contain labels with the +`feature.node.kubernetes.io/` prefix. If the GFD extension is enabled, the node +can also contain NVIDIA GPU or CUDA-related discovery labels. + +## Upgrade to v0.18.3 + +Version `v0.18.3` updates the upstream NFD base and provides Alauda +multi-architecture images. The existing optional GFD integration is preserved, +and no user configuration migration is required. + +1. Upload the `v0.18.3` plugin package. +2. Go to **Administrator** > **Clusters** > **Target Cluster** > **Functional + Components**. +3. Upgrade **Alauda Build of Node Feature Discovery** to `v0.18.3`. +4. Repeat the installation verification commands. + +The NFD upgrade does not change separately managed accelerator driver or Device +Plugin versions. After the upgrade, verify that the node labels required by +your workloads remain present. diff --git a/docs/en/nfd/intro.mdx b/docs/en/nfd/intro.mdx new file mode 100644 index 00000000..c9f748b4 --- /dev/null +++ b/docs/en/nfd/intro.mdx @@ -0,0 +1,45 @@ +--- +weight: 10 +--- + +# Introduction + +**Alauda Build of Node Feature Discovery (NFD)** is based on the Kubernetes SIG +[Node Feature Discovery](https://kubernetes-sigs.github.io/node-feature-discovery/v0.18/) +project. It detects hardware features and system configuration on Kubernetes +nodes and publishes the discovered information as node labels and `NodeFeature` +resources. + +NFD provides a shared node-discovery layer for AI workloads and other +components that select nodes by operating system, architecture, kernel, CPU, +PCI, or other hardware characteristics. + +## Scope + +NFD discovers and labels node capabilities. It does not: + +- install or manage hardware drivers; +- expose GPU or NPU resources to Kubernetes; +- replace a vendor Device Plugin, DRA driver, or accelerator operator. + +Install the plugin once in each target cluster that requires these node labels. +Other Cluster Plugins and workloads can consume NFD labels, while drivers and +resource-management components remain independently installed and managed. + +## Optional GFD extension + +The deployment form includes a **gfd Extension** switch, which is disabled by +default. Enable it only when workloads or other components need additional +NVIDIA GPU and CUDA-related labels. + +The GFD extension adds discovery labels. It does not install the NVIDIA driver +or replace the NVIDIA GPU Device Plugin. + +For platform installation and upgrade instructions, see +[Install Node Feature Discovery](./install). + +## Related information + +- [Supported configurations](../learn/supported-configurations-for-2.x) +- [Schedule inference services based on the CUDA version](../model_inference/inference_service/how_to/accurately_schedule) +- [Upstream Node Feature Discovery documentation](https://kubernetes-sigs.github.io/node-feature-discovery/v0.18/)