RenderFormer V2

This repository bundles the two RenderFormer V2 renderer checkpoints and the material-latent models used by the data pipeline. Source code, installation instructions, scene conversion, and inference examples live in RenderFormer Studio.

If this repository is access-protected during release staging, authenticate with Hugging Face before loading it. One repository revision selects every component below as an atomic, compatible set.

Components

Path	Class	Resolution	Purpose
`transformer_512`	`RenderFormerModel`	512	Native-512 V2 renderer
`transformer_2048`	`RenderFormerModel`	2048	Native-2048 V2 renderer
`material_autoencoder`	`MaterialAutoencoder`	256	Material image to/from 9-D latent
`diffspec_mapper`	`DiffuseSpecularToLatent`	—	Diffuse/specular BRDF to 9-D latent
`metallic_mapper`	`PrincipledBRDFToLatent`	—	Metallic/roughness BRDF to 9-D latent
`metallic_transmission_mapper`	`PrincipledBRDFToLatentWithTransmission`	—	Metallic/transmission BRDF to 9-D latent

The renderer pipeline loads the Qwen-Image VAE configured by each transformer for environment and raw-material encoding. Those auxiliary weights are an external dependency and are not duplicated in this bundle.

Load a renderer

resolution automatically selects transformer_512 or transformer_2048:

from renderformer import RenderFormerV2Pipeline

pipeline = RenderFormerV2Pipeline.from_pretrained(
    "RenderFormer/renderformer-v2",
    resolution=512,
    device="cuda",
)
result = pipeline("scene.h5", resolution=512, precision="fp16")

Advanced callers may select a component explicitly with subfolder="transformer_512". The CLI and pipeline documentation in RenderFormer Studio cover H5 preparation, precision, view batching, and tone mapping.

Load material components

from renderformer.models.material import (
    DiffuseSpecularToLatent,
    MaterialAutoencoder,
    PrincipledBRDFToLatent,
    PrincipledBRDFToLatentWithTransmission,
)

repo = "RenderFormer/renderformer-v2"
material_autoencoder = MaterialAutoencoder.from_pretrained(
    repo, subfolder="material_autoencoder", strict=True
).eval()
diffspec_mapper = DiffuseSpecularToLatent.from_pretrained(
    repo, subfolder="diffspec_mapper", strict=True
).eval()
metallic_mapper = PrincipledBRDFToLatent.from_pretrained(
    repo, subfolder="metallic_mapper", strict=True
).eval()
transmission_mapper = PrincipledBRDFToLatentWithTransmission.from_pretrained(
    repo, subfolder="metallic_transmission_mapper", strict=True
).eval()

The autoencoder's preprocessor_config.json records the physical-input contract. Prefer encode_hdr and decode_hdr for finite, nonnegative, alpha-premultiplied linear RGB.

Citation

Please cite both RenderFormer papers when using this bundle. Full BibTeX entries are provided in the RenderFormer Studio README.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support