Add SGLang usage section to the Cosmos3-Super-Image2Video-4Step model card

README.md

@@ -9,6 +9,8 @@ tags:
   - cosmos
   - cosmos3
   - vllm-omni
+  - sglang
+  - sglang-diffusion
   - image-to-video
   - video-generation
 ---
@@ -196,6 +198,7 @@ Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated sys
 
 - [PyTorch](https://github.com/nvidia/cosmos3)
 - [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
+- [SGLang](https://github.com/sgl-project/sglang)
 
 **Supported Hardware Microarchitecture Compatibility:**
 
@@ -504,6 +507,64 @@ python scripts/upsample_prompt.py \
     --output-path scripts/upsampled.json
 ```
 
+## Usage: Run Inference with SGLang
+
+[SGLang Diffusion](https://docs.sglang.io/docs/sglang-diffusion/index) can serve `nvidia/Cosmos3-Super-Image2Video-4Step` through an OpenAI-compatible video generation endpoint. Install SGLang from the main branch with diffusion dependencies, then start the server:
+
+```bash
+git clone --branch main https://github.com/sgl-project/sglang.git
+cd sglang
+pip install -e "python[diffusion]"
+pip install "cosmos-guardrail==0.3.1"
+
+sglang serve \
+  --model-path nvidia/Cosmos3-Super-Image2Video-4Step \
+  --num-gpus 4
+```
+
+Cosmos 3 support in SGLang Diffusion currently requires the SGLang main branch. Switch to a stable SGLang release once Cosmos 3 support is included there.
+
+The 64B transformer is roughly 128 GB in BF16. Sequence and classifier-free-guidance parallelism divide activations rather than weights, so every GPU still holds a full copy of the model. On devices with less memory, such as H100-class GPUs, shard the weights across GPUs with `--use-fsdp-inference`, or stream them layer by layer with `--dit-layerwise-offload`:
+
+```bash
+sglang serve \
+  --model-path nvidia/Cosmos3-Super-Image2Video-4Step \
+  --num-gpus 4 \
+  --use-fsdp-inference true
+```
+
+Layerwise offload keeps the transformer in pinned host memory and prefetches each layer as it is needed, so it needs roughly 128 GB of free system RAM in addition to the GPU, and it also works on a single GPU.
+
+`Cosmos3-Super-Image2Video-4Step` uses a fixed 4-step distilled schedule. SGLang detects the distilled checkpoint from `scheduler/scheduler_config.json` and drives the sampler directly from `fixed_step_sampler_config.t_list`, so `num_inference_steps` and `flow_shift` are ignored for this model and `guidance_scale` is forced to `1.0`.
+
+Example image-to-video request. The video endpoint is asynchronous: submit the job, poll until it completes, then download the result. The example below uses 480p generation (`832x480` at 16:9), the recommended setting for this distilled I2V checkpoint.
+
+```bash
+job_id=$(curl -sS -X POST http://localhost:30000/v1/videos \
+  --form-string "prompt=A small warehouse robot moves a blue box across a clean floor." \
+  --form-string "negative_prompt=blurry, distorted, low quality" \
+  --form-string "size=832x480" \
+  --form-string "num_frames=189" \
+  --form-string "fps=24" \
+  --form-string "seed=42" \
+  --form-string 'extra_params={"guardrails":true,"use_resolution_template":false,"use_duration_template":false}' \
+  -F "input_reference=@input.png" \
+  | python -c 'import json, sys; print(json.load(sys.stdin)["id"])')
+
+while true; do
+  status=$(curl -sS "http://localhost:30000/v1/videos/${job_id}" \
+    | python -c 'import json, sys; print(json.load(sys.stdin)["status"])')
+  [ "$status" = "completed" ] && break
+  [ "$status" = "failed" ] && exit 1
+  sleep 1
+done
+
+curl -sS -L "http://localhost:30000/v1/videos/${job_id}/content" \
+  -o cosmos3_i2v_4step_sglang.mp4
+```
+
+For complete serving instructions and request examples, see the [Cosmos3 SGLang cookbook](https://docs.sglang.io/cookbook/diffusion/Cosmos/Cosmos3).
+
 ## Limitations
 
 Cosmos3 may produce imperfect outputs in challenging scenarios. Generation artifacts include temporal inconsistency, unstable camera or object motion, imprecise physical interactions, inaccurate audio-video synchronization, and action-state drift — especially in long-horizon or high-resolution outputs. Reasoning may also be incorrect: object states, causal relationships, spatial geometry, temporal ordering, agent intent, and future outcomes can be misinferred, and complex or long-context inputs may yield hallucinated entities, inconsistent interpretations, or implausible predictions. Because the model lacks an explicit physics simulator, 3D geometry, 4D space-time evolution, object permanence, contact dynamics, and physical laws are only approximated — producing artifacts such as disappearing or morphing objects, unrealistic collisions, and physically implausible motions. Quality further degrades in out-of-distribution environments, safety-critical edge cases, and domains underrepresented in training.
@@ -512,7 +573,7 @@ Cosmos3 outputs should not be treated as physically accurate simulation, reliabl
 
 ## Inference
 
-**Acceleration Engine:** [PyTorch](https://pytorch.org/), [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
+**Acceleration Engine:** [PyTorch](https://pytorch.org/), [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [SGLang](https://github.com/sgl-project/sglang), [SGLang Diffusion](https://docs.sglang.io/docs/sglang-diffusion/index)
 
 **Test Hardware:** RTX PRO 6000, H20, H100, H200, B200