docs: Document Serverless network-volume warm cache (VolumeCache)#693
docs: Document Serverless network-volume warm cache (VolumeCache)#693promptless[bot] wants to merge 1 commit into
Conversation
Add a new page documenting the runpod-python volume cache: its built-in auto-hydrate/sync behavior, env-var configuration (RUNPOD_VOLUME_CACHE, RUNPOD_VOLUME_CACHE_MAX_GB, RUNPOD_CACHE_DIRS), the VolumeCache class and warm() context manager, and limitations. Wire it into the nav and the optimization checklist. Source: runpod/runpod-python#531 (SLS-367)
![promptless[bot]](https://avatars.githubusercontent.com/in/979718?s=60&v=4){kind=small}
|
|
||
| | Variable | Default | Description | | ||
| |----------|---------|-------------| | ||
| | `RUNPOD_VOLUME_CACHE` | Enabled | Controls whether the built-in volume cache runs. Set to `0`, `false`, or `no` to turn it off. | |
There was a problem hiding this comment.
Env-var names, defaults, and gating come from runpod/serverless/utils/rp_volume_cache.py in PR #531: RUNPOD_VOLUME_CACHE defaults on (disabled only for 0/false/no), RUNPOD_VOLUME_CACHE_MAX_GB defaults to 50, and RUNPOD_CACHE_DIRS is split on os.pathsep (: on Linux). Used to write the configuration table.
Source: https://github.com/runpod/runpod-python/pull/531/files
|
|
||
| ### Directories that are cached | ||
|
|
||
| By default, the volume cache targets the directories where Hugging Face and PyTorch store downloaded model weights: |
There was a problem hiding this comment.
The auto-discovered cache directories (HF_HOME or ~/.cache/huggingface, plus HF_HUB_CACHE and TORCH_HOME when set) come from _discover_model_dirs() in rp_volume_cache.py. Used to write the "Directories that are cached" section.
Source: https://github.com/runpod/runpod-python/pull/531/files
| `VolumeCache` accepts the following arguments: | ||
|
|
||
| | Argument | Default | Description | | ||
| |----------|---------|-------------| |
There was a problem hiding this comment.
The VolumeCache constructor signature and defaults (dirs, namespace defaulting to the endpoint ID, volume_path="/runpod-volume", max_size_gb=None, best_effort=True) and the public export runpod.serverless.VolumeCache come from rp_volume_cache.py and runpod/serverless/__init__.py in PR #531. Used to write the arguments table and code sample.
Source: https://github.com/runpod/runpod-python/pull/531/files
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Open this suggestion in Promptless to view citations and reasoning process
Adds a new Serverless page documenting the runpod-python volume cache introduced in runpod/runpod-python#531 (SLS-367): its built-in auto-hydrate/sync behavior, env-var configuration (RUNPOD_VOLUME_CACHE, RUNPOD_VOLUME_CACHE_MAX_GB, RUNPOD_CACHE_DIRS), the public VolumeCache class and warm() context manager, and its limitations. Adds the page to the nav and the optimization checklist. The source PR is still open (not merged), so some behavior (notably default-on vs opt-in) may change before release.
Trigger Events
Tip: Filter the Dashboard by labels or assignees to focus on what matters to you 🔎