MYRIAD (Envisioning the Future, One Step at a Time)

Paper and Abstract

The MYRIAD (Motion hYpothesis Reasoning via Iterative Autoregressive Diffusion) model was presented in the paper Envisioning the Future, One Step at a Time.

From a single image, MYRIAD predicts distributions over sparse point trajectories autoregressively. This allows to predict consistent futures in open-set environments and plan actions by exploring a large number of counterfacual interactions.

Project Page and Code

• Project Page: https://compvis.github.io/myriad
• GitHub Repository: https://github.com/CompVis/flow-poke-transformer

From a single image, our model envisions diverse, physically consistent futures by predicting sparse point trajectories step by step.

Its efficiency enables exploring thousands of counterfactual rollouts directly in motion space - here illustrated for billiards planning, where candidate shots are evaluated by simulating many possible outcomes.

Usage

For programmatic use, the simplest way to use MYRIAD is via torch.hub:

myriad_openset = torch.hub.load("CompVis/myriad", "myriad_openset")

myriad_billiard = torch.hub.load("CompVis/myriad", "myriad_billiard")

If you wish to integrate MYRIAD in your own codebase, you can copy model.py and dinov3.py from the GitHub repository. The MyriadStepByStep class contains a predict_simulate method for unrolling trajectories and a low-level forward method to predict distributions for previously observed trajectories.

Citation

If you find our model or code useful, please cite our paper:

@inproceedings{baumann2026envisioning,
  title={Envisioning the Future, One Step at a Time},
  author={Baumann, Stefan Andreas and Wiese, Jannik and Martorella, Tommaso and Kalayeh, Mahdi M. and Ommer, Bjorn},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2026}
}