Instructions to use timbrooks/instruct-pix2pix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use timbrooks/instruct-pix2pix with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("timbrooks/instruct-pix2pix", dtype=torch.bfloat16, device_map="cuda") prompt = "Turn this cat into a dog" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") image = pipe(image=input_image, prompt=prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Unable to select the model in Stable Diffusion
This is the message I am getting when selecting the model in Stable Diffusion. Can someone tell me what I am doing wrong?
Loading weights [db9dd001] from D:\Active Projects\AI\stable-diffusion-webui\models\Stable-diffusion\instruct-pix2pix-00-22000.safetensors
Failed to load checkpoint, restoring previous
Loading weights [c88e730a] from D:\Active Projects\AI\stable-diffusion-webui\models\Stable-diffusion\protogenX34Photoreal_1.safetensors
Applying cross attention optimization (Doggettx).
changing setting sd_model_checkpoint to instruct-pix2pix-00-22000.safetensors [db9dd001]: RuntimeError
Traceback (most recent call last):
File "D:\Active Projects\AI\stable-diffusion-webui\modules\shared.py", line 505, in set
self.data_labels[key].onchange()
File "D:\Active Projects\AI\stable-diffusion-webui\modules\call_queue.py", line 15, in f
res = func(*args, **kwargs)
File "D:\Active Projects\AI\stable-diffusion-webui\webui.py", line 73, in
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights()))
File "D:\Active Projects\AI\stable-diffusion-webui\modules\sd_models.py", line 369, in reload_model_weights
load_model_weights(sd_model, checkpoint_info)
File "D:\Active Projects\AI\stable-diffusion-webui\modules\sd_models.py", line 203, in load_model_weights
model.load_state_dict(sd, strict=False)
File "D:\Active Projects\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
size mismatch for model.diffusion_model.input_blocks.0.0.weight: copying a param with shape torch.Size([320, 8, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).
I had the same thing, but doing a git pull in the SD directory fixed the problem for me. Hope that helps.