Hugging Face's logo

Diffusers documentation

AutoencoderKLKVAEVideo

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.39.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

AutoencoderKLKVAEVideo

The 3D variational autoencoder (VAE) model with KL loss.

The model can be loaded with the following code snippet.

import torch
from diffusers import AutoencoderKLKVAEVideo

vae = AutoencoderKLKVAEVideo.from_pretrained("kandinskylab/KVAE-3D-1.0", subfolder="diffusers", torch_dtype=torch.float16)

AutoencoderKLKVAEVideo

class diffusers.AutoencoderKLKVAEVideo

< >

( ch: int = 128ch_mult: typing.Tuple[int, ...] = (1, 2, 4, 8)num_res_blocks: int = 2in_channels: int = 3out_ch: int = 3z_channels: int = 16temporal_compress_times: int = 4 )

Parameters

  • ch (int, optional, defaults to 128) — Base channel count.
  • ch_mult (Tuple[int], optional, defaults to (1, 2, 4, 8)) — Channel multipliers per level.
  • num_res_blocks (int, optional, defaults to 2) — Number of residual blocks per level.
  • in_channels (int, optional, defaults to 3) — Number of input channels.
  • out_ch (int, optional, defaults to 3) — Number of output channels.
  • z_channels (int, optional, defaults to 16) — Number of latent channels.
  • temporal_compress_times (int, optional, defaults to 4) — Temporal compression factor.

A VAE model with KL loss for encoding videos into latents and decoding latent representations into videos. Used in KVAE.

This model inherits from ModelMixin. Check the superclass documentation for its generic methods implemented for all models (such as downloading or saving).

decode

< >

( z: Tensorreturn_dict: bool = True ) → ~models.vae.DecoderOutput or tuple

Parameters

  • z (torch.Tensor) — Input batch of latent vectors with shape (B, C, T, H, W).
  • return_dict (bool, optional, defaults to True) — Whether to return a ~models.vae.DecoderOutput instead of a plain tuple.

Returns

~models.vae.DecoderOutput or tuple

Decoded video.

Decode a batch of videos.

disable_slicing

< >

( )

Disable sliced VAE decoding.

enable_slicing

< >

( )

Enable sliced VAE decoding.

encode

< >

( x: Tensorreturn_dict: bool = True )

Parameters

  • x (torch.Tensor) — Input batch of videos with shape (B, C, T, H, W).
  • return_dict (bool, optional, defaults to True) — Whether to return a ~models.autoencoder_kl.AutoencoderKLOutput instead of a plain tuple.

Encode a batch of videos into latents.

forward

< >

( sample: Tensorsample_posterior: bool = Falsereturn_dict: bool = Truegenerator: typing.Optional[torch.Generator] = None ) → ~models.vae.DecoderOutput or tuple

Parameters

  • sample (torch.Tensor) — Input sample.
  • sample_posterior (bool, optional, defaults to False) — Whether to sample from the posterior.
  • return_dict (bool, optional, defaults to True) — Whether or not to return a DecoderOutput instead of a plain tuple.
  • generator (torch.Generator, optional) — A torch.Generator to make sampling deterministic.

Returns

~models.vae.DecoderOutput or tuple

If return_dict is True, a ~models.vae.DecoderOutput is returned, otherwise a plain tuple is returned.

Update on GitHub

←AutoencoderKLKVAE AutoencoderKLLTX2Audio→