Hugging Face's logo

Viggle
/
Meridian

Video-to-Video
Diffusers
Safetensors
novel-view-synthesis
camera-control
re-camera
Model card Files
xet
 Community
Meridian / docs /installation.md
yycc's picture
yycc
Release Meridian teacher and DMD adapter weights
2083d05 verified
preview code
|
Raw
History Blame Contribute Delete
10.2 kB

Installation

← Meridian · Inference · Studio demo

Before downloading

  • Review the licenses and intended use. The weights are not Apache 2.0, the MiniMax-H3 license has territorial restrictions, and the VGGT-Omega dependency is licensed separately for noncommercial research.
  • Use a CUDA GPU with substantial memory. The released scripts run on one GPU and do not expose CPU inference, multi-GPU sharding, quantization, or CPU-offload options. The reported resident-service peak is approximately 88 GiB for 73 frames. The CLI logs approximately 82 GiB of peak PyTorch-allocated memory during denoising; this does not measure the whole-process peak or driver-level GPU usage. A 96 GB-class GPU is the reported configuration for takes up to 124 frames; 243-frame takes reach approximately 113 GiB in the service. Leave headroom for geometry caches, other processes, and differences between GB and GiB.
  • Allow disk space beyond the weights. The teacher and adapter total approximately 64 GiB; the H3 VAE, VGGT-Omega checkpoint, package caches, uploads, and generated videos are additional.
  • Reference environment: Python 3.12, CUDA 12.8, PyTorch 2.9.1, and torchvision 0.24.1. B200 is the reported benchmark GPU, not a claim that every CUDA GPU is validated.
  • Have Git, FFmpeg, and FFprobe on PATH. Git is needed for the pinned Diffusers install; the Python packages do not install the FFmpeg command-line executable.

1. Create an environment and download the code

Run these commands in a shell with Python 3.12 available. python below always means the Python in the activated environment. Sign in if repository access requires it. The command fetches code, runtime assets, guides, and sample clips, not the optional showcase videos or model weights.

python3.12 -m venv .venv-meridian
source .venv-meridian/bin/activate
python -m pip install --upgrade pip
python -m pip install huggingface_hub

hf auth login
hf download Viggle/Meridian --local-dir Meridian \
  --include "README.md" "LICENSE*" "NOTICE" "MODIFICATIONS.md" "requirements.txt" \
  "recam/*" "inference/*" "service/*" "assets/*" "examples/*" \
  "docs/installation.md" "docs/inference.md"
cd Meridian
python -m pip install -r requirements.txt
python -m pip install peft==0.18.0

# These must work before loading any model weights or starting the GPU service.
python inference/sample.py --help
python service/app.py --help

The PEFT package is needed by the student adapter loader and is not currently listed in requirements.txt; install it explicitly. Use a dedicated environment rather than upgrading a shared inference environment in place.

Keep the Diffusers commit pinned by requirements.txt (d6726f3). The scripts use MiniMax-H3 classes and modular-pipeline helpers that may not exist in another build, even if its version string includes dev. Do not replace that dependency with an arbitrary PyPI release.

Supply the teacher and LoRA weights

Download Meridian's teacher and DMD adapter from this repository. Run from the Meridian directory:

hf download Viggle/Meridian --include "transformer/*" "lora/*" --local-dir .

The teacher uses standard Diffusers safetensors shards; keep the configuration and index file alongside them. The teacher and adapter total approximately 64 GiB. VGGT-Omega and the H3 VAE are obtained separately below.

If you already have the Meridian checkpoints, place the complete Diffusers transformer directory (including its configuration, weight shards, and any index file) and the student adapter alongside the code:

Meridian/
  inference/sample.py
  assets/
  transformer/
    config.json
    ... checkpoint files ...
  lora/
    pytorch_lora_weights.safetensors

Alternatively, add --ckpt /absolute/path/to/transformer --lora /absolute/path/to/lora to the CLI or Studio command. Use Meridian's finetuned teacher, not the unmodified MiniMax-H3 transformer.

2. Obtain VGGT-Omega separately

VGGT-Omega code and weights are not redistributed here. Request access to facebook/VGGT-Omega, read its license, and authenticate with a Hugging Face account that has been granted access.

# Run from the Meridian release directory; the checkout is placed beside it.
git clone https://github.com/facebookresearch/vggt-omega ../vggt-omega
export VGGT_OMEGA_DIR="$(cd ../vggt-omega && pwd)"

hf auth login
hf download facebook/VGGT-Omega vggt_omega_1b_512.pt \
  --local-dir "$VGGT_OMEGA_DIR/checkpoints"

Follow the VGGT-Omega checkout's own dependency instructions if additional packages are needed. Its source directory is imported directly; this release does not install it as a Python package.

By default, Meridian looks for $VGGT_OMEGA_DIR/checkpoints/vggt_omega_1b_512.pt. If you already store the weight file elsewhere:

export VGGT_OMEGA_CKPT=/absolute/path/to/vggt_omega_1b_512.pt

Keep these exports in the shell that starts inference. The CLI and service also accept --vggt-repo /absolute/path/to/vggt-omega and --vggt /absolute/path/to/the/checkpoint.pt.

Meta's FAIR Noncommercial Research License v1 restricts commercial use of the research materials and their outputs or results. Here those results include the geometry used to make the reference render. The Apache license on Meridian's code does not remove that restriction. Commercial use requires an appropriately licensed geometry solution or permission from Meta; swapping the geometry front end is not a built-in CLI option and requires integration work.

3. Provide the MiniMax-H3 VAE

By default, inference loads vae/ from MiniMaxAI/MiniMax-H3. It does not need the base transformer or the text encoder. To download only the VAE for local use:

hf download MiniMaxAI/MiniMax-H3 --include "vae/*" --local-dir ../MiniMax-H3

Then add --model-dir ../MiniMax-H3 to your CLI or service command. This path is the directory containing vae/, not vae/ itself. Without the flag, the default Hub identifier is used and the VAE is loaded through the Hugging Face cache.

Do not put Meridian's LoRA on the base MiniMax-H3 transformer: it was distilled on Meridian's finetuned teacher.

4. Check the setup

These checks import the required components without loading their weights or starting inference:

ffmpeg -version
ffprobe -version
python -m pip check
python -c "import torch; print('torch:', torch.__version__, 'CUDA:', torch.version.cuda, 'available:', torch.cuda.is_available())"
python -c "import peft; from diffusers import AutoencoderKLMiniMaxH3, MiniMaxH3Transformer3DModel, MiniMaxH3Scheduler; from recam.h3 import pack; print('H3 and PEFT imports OK')"
python -c "import os, sys; sys.path.insert(0, os.environ['VGGT_OMEGA_DIR']); from vggt_omega.models import VGGTOmega; print('VGGT-Omega import OK')"

For a geometry-only check on the selected GPU:

CUDA_VISIBLE_DEVICES=0 python inference/sample.py \
  --video examples/media/sp_bouldering_hang.mp4 \
  --yaw 15 --sweep --gauge-only --out out/check

This loads VGGT-Omega and prints geometry diagnostics. It does not load the VAE or transformer, and does not write the normal output videos. It is not a full inference or model-memory test.

Next: run the first take, learn the camera controls, or start the Studio demo.

Setup problems

Symptom Check
hf or ffmpeg not found Activate the environment for hf; install the system FFmpeg tools separately and check PATH.
Hub access denied Confirm the account has accepted the model's terms and received access; authenticate with that account. A token alone does not grant gated access.
No module named vggt_omega VGGT_OMEGA_DIR must contain the vggt_omega/ package. Export it in the same shell that starts the process.
VGGT-Omega not found Check both the source checkout and checkpoint path; VGGT_OMEGA_CKPT must name the .pt file.
Cannot import a MiniMax-H3 class or layout helper Reinstall the pinned requirements in the active environment; inspect python -c "import diffusers; print(diffusers.__file__)" for a conflicting checkout.
Missing PEFT or adapter-loading error Install PEFT, use the finetuned teacher, and confirm the adapter filename and --lora directory.
CUDA or attention-backend failure Check the PyTorch/CUDA/driver combination against the reference environment. The service selects _native_cudnn; other hardware/backend combinations are not validated here.
Out of memory Start with 73 output frames, a short source span, and no other GPU workload. The scripts do not automatically offload to CPU. The Studio keeps its models and recent geometry caches resident.

Lower-memory community work

Meridian retains MiniMax-H3's transformer architecture and uses precomputed text embeddings, so inference does not load the text encoder. This is a starting point for adapting community memory-saving techniques—not evidence that the remaining transformer, activations, VAE, and geometry fit a smaller GPU.

We welcome work on quantization and CPU offloading toward consumer GPUs such as the RTX 4090. Diffusers documents quantization and memory reduction and offloading. These are general integration references, not a tested Meridian recipe or a reason to replace the pinned Diffusers build indiscriminately.

The current CLI and service move their models onto one CUDA device; neither exposes those optimizations. A contribution needs to integrate them into the custom inference path and validate adapter loading, reference conditioning, image quality, peak GPU/host memory, and end-to-end latency. There is no verified RTX 4090 configuration or performance claim for this release.