Am I doing something wrong? Very slow on 5070 ti 16GB even using fp8
Even using just fp16 it's painfully slow... running Sage attention on 5070 ti 16GB and it's taking 12 minutes just for a 2:30 song.
It's like 3 minutes to do a complete 3:30 song in Ace-Step using fp32 even. Am I doing something wrong?
Any way to speed it up? It's seems to barely use my GPU... My GPU temp is only at 37 degrees throughout the sampling but up to 64 degrees on secondary encoding which takes around 90 seconds.
It does sound great even with fp16 though from what I can tell.
Just I run a website that offers free generation and 12 minutes is way too long to add.
Is there maybe a distilled version that gets near the same quality using less steps? I notice the sampling is like 5000 vs I think 900 for Ace Step which is why it takes so long.
Would a convrot int8 version help speed up the sampling?
mine is slow too im using rtx 3060 32GB RAM, i think this really need high end GPU not for consumer grade GPU, id suggest you try acestep xl instead or wait for quantized version.
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16 [INFO] Requested to load MiniMaxMusic3TEModel [INFO] loaded completely; 9835.80 MB usable, 8759.61 MB loaded, full load: True AR sampling: 1%|▍ | 103/7501 [03:25<3:08:35, 1.53s/it]
I already have Ace-Step setup (Using SFT fp32 version) but my friend says the vocals are robotic (and says same thing for base). So I have been using this but it's so much slower but it does sound better from what I have heard so far. Have not done enough testing yet to form an opinion on other aspects like creativity between each model. I have been generating with Minimax Music 3 for 3 days now though and have quite a few samples but I have not listened to many of them yet. I created a jukebox on my website as well that has access to all the music I have created so far (around 240 songs so far but over 4000 songs based on different variations and genres so lots of variety).
I am trying to find a way to speed up to add to my website which has free image, audio, and video generation; AI Image Central. Because currently it's taking 13-18 minutes to generate depending on length of song.
The GPU is barely used during the sampling it seems as the temp stays low like 37 degrees versus up to 72 degrees for intense generation. Should be some way to make better use of the GPU during sampling.
mine is slow too im using rtx 3060 32GB RAM, i think this really need high end GPU not for consumer grade GPU, id suggest you try acestep xl instead or wait for quantized version.
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16 [INFO] Requested to load MiniMaxMusic3TEModel [INFO] loaded completely; 9835.80 MB usable, 8759.61 MB loaded, full load: True AR sampling: 1%|▍ | 103/7501 [03:25<3:08:35, 1.53s/it]
Yeah, is slow I stoped a 5minute generation after one hour and only had 35% of the task. 3060 -12gb vram - 32gb ram.
Ace step 1.5XL is more faster but lack of good quality compared with suno or other models.
I'll wait for a better optimization or the quantized version if there is available in the future.
I have the same GPU, from what I've seen, the model spends most of its time in the text encode phase.
The diffusion phase takes just a few minutes.
For example, I just generated a 3-minute song in about 24 minutes, 20:30 for text encode and 3:30 for diffusion (at 40steps)
For reference, these are times with a RTX 5070 ti 16GB using Sage Attention.
AR sampling: 88%|█████████████████████████████████████████████████████████▎ | 4627/5251 [11:32<01:33, 6.69it/s]
[INFO] Model MiniMaxMusic3 prepared for dynamic VRAM loading. 4685MB Staged. 0 patches attached. Force pre-loaded 73 weights: 580 KB.
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [01:59<00:00, 3.99s/it]
[INFO] Requested to load MiniMaxMusic3DAV
[INFO] loaded completely; 206.60 MB loaded, full load: True
[INFO] Prompt executed in 00:13:52
I just executed the comfyui standard workflow but i changed max length to 300sec. Everything else was unchanged.
[INFO] Prompt executed in 143.60 seconds
Song Max Length 300 sec
Song Actual Length 159 sec
GPU RTX 5090
CPU RAM 192GB
TE pruned_bf16
unet fp32
no sage attention
Its a some BUG in use of text_encoder model
[INFO] Model MiniMaxMusic3TEModel prepared for dynamic VRAM loading. 8758MB Staged. 0 patches attached. Force pre-loaded 154 weights: 674 KB.
AR sampling: 1%|▎ | 57/9001 [00:33<1:27:36, 1.70it/s][INFO] Interrupting prompt a64c7bc3-8c43-4c13-b4c5-7effa0c34f5a
1.70it/s - its awfully low speed.
in normal it mast be like
MiniMaxMusic3TEModel prepared for dynamic VRAM loading. 8758MB Staged. 0 patches attached. Force pre-loaded 154 weights: 674 KB.
AR sampling: 59%|██████████████████████████████████ | 4404/7501 [01:30<01:03, 48.93it/s]
about 50it/s
I can’t figure out the cause; the bug is very strange and reproduces randomly, and I don’t see any patterns. Restarting helps.
MultyGPU PC (3x5070ti) Linux.
Can you guys please just try using the pruned BF16 model or the FP8 one (from the link in another recent discussion here)?
The INT8 ConvRot is going to be extremely, EXTREMELY slow if you havent bothered to properly install the requirements for it or you dont have hardware that actually supports INT8 ConvRot.
I'm talking 50 it/s difference in some cases, with the INT8 ConvRot going at like 3it/s and the BF16 going at like 50-60.
You guys likely have fast internet, there is no reason not to at least give it a shot, even if you don't have enough VRAM to fit it all at once (even as little as half speed because of the model switching blocks between RAM and VRAM will still be miles ahead of what you currently have).
The issue is not the convrot text encoder (In fact I went from 7it/sec with convrot down to like 1it/sec using the bf16 text encoder). I found the issue.
Because of issues with comfyUI with memory leaks and not clearing memory correctly I had to disable pinned memory.
Then I read here that specifying --cuda-device 0 resolved the issues with the memory leaks (which are in pinned memory from my understanding)
After adding the '--cuda-device 0' command line option and removing the line to disable pinned memory and async My sampling speeds are much better in the 50it/sec range.
Here is my old config versus new config where I went from 7it/sec to 50it/sec
@rem C:\ComfyUI_windows_portable\python_embeded\python.exe -s C:\ComfyUI_windows_portable\ComfyUI\main.py --windows-standalone-build --port=8190 --use-sage-attention --enable-manager --reserve-vram 0.6 --disable-async-offload --disable-pinned-memory
And new config with amazing speed (Which I also need to say I have 5070 ti so I compiled and enabled Sage Attention which is crazy fast):
@C:\ComfyUI_windows_portable\python_embeded\python.exe -s C:\ComfyUI_windows_portable\ComfyUI\main.py --cuda-device 0 --windows-standalone-build --port=8190 --enable-manager --reserve-vram 0.6 --use-sage-attention
If you compile and use Sage Attention, speed up is like 6 times the standard with 5000 series hardware.
Generation now:
[INFO] Model MiniMaxMusic3TEModel prepared for dynamic VRAM loading. 8758MB Staged. 0 patches attached. Force pre-loaded 154 weights: 674 KB.
AR sampling: 27%|█████████████████▋ | 1486/5451 [00:29<01:18, 50.80it/s]
It seems dynamic VRAM still has issues. Constantly crashing with an error every 2-5 generations. Running latest aio does not resolve.
To work around this, can do --disable-pinned-memory and maintain the high speed and avoid the crashes until fixed and also still have dynamic VRAM.
So run with (Remove the Sage attention option if you have not installed):
@C:\ComfyUI_windows_portable\python_embeded\python.exe -s C:\ComfyUI_windows_portable\ComfyUI\main.py --cuda-device 0 --windows-standalone-build --port=8190 --enable-manager --reserve-vram 0.6 --use-sage-attention --disable-pinned-memory