Instructions to use ByteDance/LatentSync with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ByteDance/LatentSync with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("ByteDance/LatentSync", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
metadata
license: openrail++
library_name: diffusers
tags:
- video-to-video
The checkpoints of LatentSync
This repo not only stores the pretrained U-Net and SyncNet checkpoints of LatentSync, but also stores the whisper checkpoints, auxiliary checkpoints for detecting face, calculating syncnet confidence score and so on. They have covered all you need for both inference and training of LatentSync