diff --git a/README.md b/README.md index ee0c3dc..2ea16c9 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,8 @@ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH On Apple Silicon (e.g. Mac Mini M4), Metal is enabled by default and runs both the transformer and vision tower on the GPU. See [docs/backend/metal.md](docs/backend/metal.md) for building `vla.cpp` on macOS. +On Windows, build inside WSL2 (Ubuntu). See [docs/backend/windows-wsl.md](docs/backend/windows-wsl.md) for the WSL2 + CUDA setup, including the GPU/toolkit prerequisites. + ## Install simulators The eval scaffold under [`eval/`](eval/) supports two simulators end-to-end. Each setup script bootstraps an isolated Python 3.10 `uv` venv next to itself and clones the upstream sim repo. Both require [`uv`](https://github.com/astral-sh/uv) on `PATH`. diff --git a/docs/backend/windows-wsl.md b/docs/backend/windows-wsl.md new file mode 100644 index 0000000..68350b4 --- /dev/null +++ b/docs/backend/windows-wsl.md @@ -0,0 +1,117 @@ +# `vla.cpp` on Windows (WSL2 + CUDA) + +`vla.cpp` targets Linux and macOS. On Windows the supported path is **WSL2** +with an Ubuntu distribution: the toolchain (`libzmq`, `protobuf`, `pkg-config`, +the bash `patches/patch.sh` script) and the CUDA build all run natively inside +the Linux environment, while still using the host NVIDIA GPU through the +WSL CUDA driver. + +> **Status: draft.** Verified through toolchain + dependency setup on Ubuntu +> 24.04 (WSL2) with the CUDA 12.6 toolkit and an RTX 4050 Laptop GPU +> (`sm_89`). The end-to-end build and the SmolVLA run are being validated; +> the [Results](#results) section is filled in once that completes. + +## Prerequisites + +A WSL2 distribution (this guide uses **Ubuntu 24.04**) and a recent **NVIDIA +Windows driver** that exposes the GPU to WSL. Confirm the GPU is visible from +inside WSL before building: + +```bash +nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv +# e.g. NVIDIA GeForce RTX 4050 Laptop GPU, 6141 MiB, 556.29 +``` + +> Note: the Windows driver ships the WSL CUDA *driver*, but not the CUDA +> *toolkit* (`nvcc`). The toolkit is installed inside WSL, below. + +Install the build dependencies and a CUDA toolkit that matches your driver +(driver 556.29 supports CUDA ≤ 12.6; CUDA ≥ 12.4 is required for the GCC 13 +shipped on Ubuntu 24.04): + +```bash +sudo apt-get update +sudo apt-get install -y libzmq3-dev libprotobuf-dev protobuf-compiler \ + pkg-config build-essential cmake git wget + +# CUDA toolkit (nvcc) via the WSL-specific repo +cd /tmp +wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.1-1_all.deb +sudo dpkg -i cuda-keyring_1.1-1_all.deb +sudo apt-get update +sudo apt-get install -y cuda-toolkit-12-6 + +export PATH=/usr/local/cuda-12.6/bin:$PATH +export LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64:$LD_LIBRARY_PATH +nvcc --version # confirm the toolkit is on PATH +``` + +### Windows `PATH` translation errors + +WSL imports the Windows `PATH` into the Linux environment. A stale Windows +`PATH` entry that points at a drive no longer present produces, on every +command: + +``` +<3>WSL (NNN) ERROR: UtilTranslatePathList:2852: Failed to translate E:\Some\Dir +``` + +It is harmless, but to remove it delete the dead entry from the Windows user +`PATH` (PowerShell), then restart WSL: + +```powershell +$p = [Environment]::GetEnvironmentVariable('Path','User') -split ';' +$p = $p | Where-Object { $_ -and $_ -ne 'E:\Some\Dir' } +[Environment]::SetEnvironmentVariable('Path', ($p -join ';'), 'User') +``` + +```powershell +wsl --shutdown # reopen WSL afterwards so the new PATH is picked up +``` + +## Configure & build + +```bash +# Fetch llama.cpp at the pinned tag and apply the local patch +bash patches/patch.sh + +# CUDA build. Set CMAKE_CUDA_ARCHITECTURES for your GPU (see the table in the +# top-level README; RTX 40-series / Ada = 89). +cmake -B build \ + -DGGML_CUDA=ON \ + -DGGML_CUDA_GRAPHS=ON \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_CUDA_ARCHITECTURES=89 +cmake --build build -j"$(nproc)" +``` + +## GPU offload + +The VLA core selects its compute backend at load time. With a CUDA build it +picks the CUDA backend; confirm it from the SmolVLA startup banner: + +``` +vla: backend = CUDA (device 0) +``` + +If you instead see `vla: backend = CPU (4 threads)`, the build did not pick up +CUDA — rebuild from a clean `build/` and check `GGML_CUDA` is `ON` in the CMake +cache and that `nvcc` was on `PATH` at configure time. + +## Run SmolVLA + +SmolVLA ships a combined GGUF plus a separate `mmproj` vision tower (see +[Models](../../README.md#models); GGUF published at +[`vrfai/smolvla-libero-gguf`](https://huggingface.co/vrfai/smolvla-libero-gguf)). + +```bash +./build/vla-server "$VLA_MMPROJ" "$VLA_GGUF" +# vla-server: bound to tcp://*:5555. ready. +``` + +Drive it with the LIBERO client (`--arch smolvla`) as described in the +top-level README. + +## Results + +_Pending end-to-end validation on WSL2 + RTX 4050._