Andyson commited on
Commit
e17416e
·
verified ·
1 Parent(s): e5a8b91

Add SCOPE Perflow comparison README

Browse files
SCOPE_fastvideo_perflow_compare/README.md ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SCOPE LoRA Comparison Workflow
2
+
3
+ This workflow runs SCOPE with the original SCOPE weights plus two external
4
+ Wan2.2 TI2V LoRA checkpoints:
5
+
6
+ - FastVideo DMD LoRA, used for 3-step inference.
7
+ - Perflow DMD step4 LoRA, used for 4-step inference.
8
+
9
+ The current Perflow comparison reuses the existing SCOPE/FastVideo example 0
10
+ clips and generates only the new Perflow 4-step clip before stitching a 2x2
11
+ comparison video.
12
+
13
+ ## Environment
14
+
15
+ On this cluster the Slurm scripts default to the existing `diffusion-pipe`
16
+ conda env:
17
+
18
+ ```bash
19
+ conda activate diffusion-pipe
20
+ python scripts/scope_fastvideo/check_env.py
21
+ ```
22
+
23
+ For a fresh environment, create one with FastVideo as the dependency baseline.
24
+ The script installs local `FastVideo` and `SCOPE` editable, installs
25
+ `fastvideo-kernel==0.2.6`, and requires a prebuilt `flash-attn` wheel instead
26
+ of compiling flash-attn locally:
27
+
28
+ ```bash
29
+ CONDA_ENV=scope_fastvideo bash scripts/scope_fastvideo/create_env.sh
30
+ ```
31
+
32
+ ## Checkpoint Paths
33
+
34
+ SCOPE weights:
35
+
36
+ ```text
37
+ models/SCOPE
38
+ ```
39
+
40
+ FastVideo source model:
41
+
42
+ ```text
43
+ FastVideo/FastWan2.2-TI2V-5B-FullAttn-Diffusers
44
+ ```
45
+
46
+ FastVideo extracted LoRA used by SCOPE:
47
+
48
+ ```text
49
+ models/fastvideo_dmd_lora/adapter_r32.safetensors
50
+ ```
51
+
52
+ FastVideo inference setting:
53
+
54
+ ```text
55
+ num_inference_steps=3
56
+ dmd_denoising_steps=1000,757,522
57
+ ```
58
+
59
+ Perflow source repo:
60
+
61
+ ```text
62
+ Perflow-Shuai/dmd_step4_rope_offset_checkpoint_model_002000
63
+ ```
64
+
65
+ Perflow local Hub cache observed on this machine:
66
+
67
+ ```text
68
+ /home/shyang/.cache/huggingface/hub/models--Perflow-Shuai--dmd_step4_rope_offset_checkpoint_model_002000/snapshots/c10942ab1e8498d2c39da3773b989ca7c9889b40/model.pt
69
+ ```
70
+
71
+ Perflow checkpoint format:
72
+
73
+ ```text
74
+ model.pt top-level keys: generator_lora, critic_lora, step
75
+ loaded for inference: generator_lora
76
+ ignored for inference: critic_lora
77
+ step: 2000
78
+ rank: 128
79
+ ```
80
+
81
+ Perflow inference setting:
82
+
83
+ ```text
84
+ num_inference_steps=4
85
+ perflow_denoising_steps=
86
+ ```
87
+
88
+ No explicit Perflow DMD timestep list was available in the checkpoint, so the
89
+ comparison uses SCOPE's normal 4-step scheduler path rather than reusing
90
+ FastVideo's 3-step DMD timesteps.
91
+
92
+ ## Direct Inference Commands
93
+
94
+ FastVideo LoRA on SCOPE example 0:
95
+
96
+ ```bash
97
+ python SCOPE/inference.py \
98
+ --model_dir models/SCOPE \
99
+ --input_image SCOPE/examples/example_0/image.png \
100
+ --action_path SCOPE/examples/example_0/action.parquet \
101
+ --prompt "$(cat SCOPE/examples/example_0/prompt.txt)" \
102
+ --output_dir outputs/scope_fastvideo/manual_fastvideo_example_0 \
103
+ --height 480 \
104
+ --width 832 \
105
+ --max_frames 81 \
106
+ --num_inference_steps 3 \
107
+ --dmd_denoising_steps 1000,757,522 \
108
+ --fastvideo_lora_path_or_repo models/fastvideo_dmd_lora/adapter_r32.safetensors \
109
+ --lora_strength 1.0 \
110
+ --fps 20 \
111
+ --comparison_mode
112
+ ```
113
+
114
+ Perflow LoRA on SCOPE example 0:
115
+
116
+ ```bash
117
+ python SCOPE/inference.py \
118
+ --model_dir models/SCOPE \
119
+ --input_image SCOPE/examples/example_0/image.png \
120
+ --action_path SCOPE/examples/example_0/action.parquet \
121
+ --prompt "$(cat SCOPE/examples/example_0/prompt.txt)" \
122
+ --output_dir outputs/scope_fastvideo/manual_perflow_example_0 \
123
+ --height 480 \
124
+ --width 832 \
125
+ --max_frames 81 \
126
+ --num_inference_steps 4 \
127
+ --dmd_denoising_steps "" \
128
+ --fastvideo_lora_path_or_repo Perflow-Shuai/dmd_step4_rope_offset_checkpoint_model_002000 \
129
+ --lora_strength 1.0 \
130
+ --fps 20 \
131
+ --comparison_mode
132
+ ```
133
+
134
+ ## Slurm Jobs
135
+
136
+ FastVideo full 3-case evaluation:
137
+
138
+ ```bash
139
+ mkdir -p logs/scope_fastvideo
140
+ UPLOAD=1 sbatch scripts/scope_fastvideo/slurm_full_eval.sbatch
141
+ ```
142
+
143
+ Perflow example 0 comparison against the existing FastVideo result:
144
+
145
+ ```bash
146
+ mkdir -p logs/scope_fastvideo
147
+ UPLOAD=1 sbatch scripts/scope_fastvideo/slurm_perflow_compare.sbatch
148
+ ```
149
+
150
+ This job uses partition `backfill`, matching the previous FastVideo full
151
+ evaluation job on this cluster.
152
+
153
+ The Perflow job writes:
154
+
155
+ ```text
156
+ outputs/scope_fastvideo/perflow_compare/clips/example_0_scope_perflow_lora_4step.mp4
157
+ outputs/scope_fastvideo/perflow_compare/comparisons/example_0_comparison.mp4
158
+ outputs/scope_fastvideo/perflow_compare/manifest.json
159
+ outputs/scope_fastvideo/perflow_compare/environment.json
160
+ ```
161
+
162
+ When `UPLOAD=1`, it uploads the folder to:
163
+
164
+ ```text
165
+ Andyson/lora/SCOPE_fastvideo_perflow_compare/
166
+ ```
167
+
168
+ ## Notes
169
+
170
+ - The FastVideo model card's DMD setting is 3-step with timesteps
171
+ `1000,757,522`; it is not a 4-step LoRA.
172
+ - The Perflow repo name and checkpoint are step4, so this comparison runs it
173
+ with 4 inference steps.
174
+ - The current VSA kernel path can fall back to full attention on this H100 /
175
+ torch / triton runtime. The Perflow comparison does not enable VSA.
176
+ - Hugging Face authentication is expected to come from `huggingface-cli login`
177
+ or `HF_TOKEN`; no token is stored in this README.