Add SGLang Cosmos3 serving docs

[mickqian]

Summary

• add a Generator with SGLang section next to the existing Diffusers and vLLM-Omni serving paths
• document SGLang serve commands for Cosmos3-Nano and Cosmos3-Super
• list the current SGLang-supported Cosmos3 visual generation checkpoints and note follow-up gaps for sound, V2V, and action modes
• include OpenAI-compatible T2I and T2V request examples

Notes

This section is intended to mirror the vLLM-Omni style while keeping the current SGLang support boundary explicit.

[lfengad]

@yogeshbalaji Is this on the plan of the landing page?

[nv-dmajchrowski]

If I'm not mistaken

sglang serve \
  --model-path nvidia/Cosmos3-Super-Image2Video \
  --num-gpus 4

is equivalent to CFG + ulysses-deg 2 i.e.

sglang serve \
  --model-path nvidia/Cosmos3-Super-Image2Video \
  --num-gpus 4 --enable-cfg-parallel --ulysses-degree 2

which is indeed preferred way to serve multi-gpu inference, but only if the model fits into single GPU (>80GB). This it only best setup for performance, but it doesn't reduce memory requirements.

Safer option would be to use fsdp as an example for Cosmos3-Super checkpoint, as this setup actually does reduce memory requirement by sharding the weights across gpus, i.e.:

sglang serve \
  --model-path nvidia/Cosmos3-Super-Image2Video \
  --num-gpus 4 --use-fsdp-inference

[mickqian]

if we are looking for memory-friendly setups, yes we could do better, whether fsdp or offloading would do

[kediwu0331]

I think it's good that the default command is for better perf. But can we add a note saying something like if OOM error is hit, user can try add --use-fsdp-inference to save memory? @mickqian

[mickqian]

Updated in c1dc833. I kept the default command as the performance-mode setup and added a memory-mode fallback using --use-fsdp-inference for users who hit OOM.

[mickqian]

Updated again in fe557f1: switched the OOM fallback to SGLang Diffusion's higher-level --performance-mode memory preset, while keeping the default command as the performance-mode setup.

[kediwu0331]

looks good. thank you mick

[nv-dmajchrowski]

Can we make tag/stable release of the sglang repo and pin it here?
This command will always download top of tree sglang, which is not what we want as part of the README.

[mickqian]

good point. I added an optional checkout step plus a version note. the default keeps tracking upstream SGLang to pick up ongoing Cosmos 3 fixes/performance improvements, while production or reproducible deployments should pin a release tag or known-good commit before install.

[atharvajoshi10]

Probably good to specify we support other modalities such as sound and action.

[mickqian]

Updated the wording to mention that Cosmos 3 includes video-with-sound and action/policy models, while keeping this SGLang section scoped to the currently supported T2I/T2V/I2V generator serving paths.

[atharvajoshi10]

Probably best to support uv or venv

[mickqian]

Added a venv setup before the editable SGLang install.

[atharvajoshi10]

Can we add comments here to improve readability?

[mickqian]

Added comments for the submit, poll, and download steps in the video example.

[atharvajoshi10]

Is there a cleaner way to write this?
Maybe

until [ "$status" = completed ]; do
      status=$(curl -sS "http://localhost:30000/v1/videos/${job_id}" | jq -r .status)
      [ "$status" = failed ] && exit 1
      sleep 1
    done

[atharvajoshi10]

Also Generation usually take 3 mins on single GPU instance so we can try sleeping for a longer duration.

[mickqian]

Switched the polling snippet to a cleaner jq + until loop.

[mickqian]

Increased the polling interval to 5 seconds to better match longer video generation times.

[atharvajoshi10]

Thanks, Let some minor formatting comments, looks good otherwise

Not needed as Cosmos 3 transformer exists in SGlang

[atharvajoshi10]

@mickqian Can we point this directly to the SGlang cook book instead of maintaining the same doc here?

I also saw another PR from you here - https://huggingface.co/nvidia/Cosmos3-Super-Image2Video/discussions/5

[mickqian]

@atharvajoshi10
I agree the SGLang cookbook should be the source of truth for detailed and evolving usage.
that said, I would prefer not to replace the SGLang section here with only an external link, because this README already keeps detailed first-class generator instructions for vLLM-Omni. If SGLang only appears as a cookbook link, users may read that as a weaker or less official support path, even though SGLang supports the Cosmos 3 generator checkpoints natively.
I can trim this section to a minimal quickstart: supported checkpoints, install/start commands, one image/video example, and a link to the SGLang cookbook for advanced options.

[kediwu0331]

@atharvajoshi10 @vinjn
Could you review and give approval if everything looks ok to you?

[kediwu0331]

@mickqian @nv-dmajchrowski
I added examples for v2vs and action. Let me know if there is anything you want to change.

[lfengad]

Also, we could mimic the vllm-omni part, to add some content/mention about sglang in these sections of the README: #examples; #choosing-an-integration #key-capabilities?

[kediwu0331]

yeah sglang example is already there in #examples. I added a few sglang mention in #choosing-an-integration and #key-capabilities. Let me know if it looks ok.

[lfengad]

Is a \ missing here?

[kediwu0331]

yes fixed

[lfengad]

Is this 8001 consistent with the previous 30000 port?

[kediwu0331]

port should be 30000. fixed

[lfengad]

The VLLM API key would be wrong for sglang?

[kediwu0331]

fixed

[lfengad]

Should be sglang?

[kediwu0331]

yes fixed

[lfengad]

the vllm-omni name would be misleading?

[kediwu0331]

yes fixed

[lfengad]

Same here

[kediwu0331]

yes fixed

[lfengad]

Such sglang-omni also should remove -omni?

[kediwu0331]

yes fixed

[lfengad]

@kediwu0331 Since many details seems updated. Whether we need a further full run of all these added examples to make sure that they are still all directly runnable with this current version?
@qmiao-hub Whether we need the test reports regarding this change?
THX!