Got it working but couldn't manage to generate video. Any tips?
RTX3090. I installed torch 2.5 because the original installation didn't work. (RuntimeError: Device type CUDA is not supported for torch.Generator() api.)
I'm trying to make the character walk.
Changed parameters:
Input and output
start_image_path = "assets/doc1.jpg"
end_image_path = "assets/doc1.jpg" # Can be None for start-image-to-video
output_video_path = "outputs/doc1.mp4"
Video settings
video_length = 120 # The max video length is 120 frames (24 frames per second)
base_resolution = 384 # # The pixels in the generated video are approximately 512 x 512. Values in the range of [384, 896] typically produce good video quality.
video_size = None # Override base_resolution. Format: [height, width], e.g., [384, 672]
Control settings
aspect_ratio = "16:9" # Do not change, currently "16:9" works better
motion = "auto" # Motion control, choose in ["1", "2", "3", "4", "auto"]
camera_direction = "static" # Camera control, choose in ["static", "left", "right", "up", "down", "auto"]
You may set the motion = "2" or "3" to have a try.
This model isn't trained for this kind of pictures I suppose.
Maybe. The background of this image is too clean. The training data usually have some kind of background.
I have tried several times, only this doesn't create new background (but generates three legs...
24G VRAM only to create this?
@happynear : This is still very good. Can you share which parameters you used? I can try other similar characters I have.
By the way what I'm trying to do is not to generate video, I want to generate a couple of frames and then use them to generate "sprite sheets" for 2D games. Editing out a third leg or artifacts is some problem but not a big one!
@mrw21j I didn't understand your comment? Too much? Too little?
It just seems like a lot of memory to create an image with artifact and poorly defined lines