Playable Environments: Video Manipulation in Space and Time

CVPR 2022

Willi Menapace, Stéphane Lathuilière*, Aliaksandr Siarohin,
Christian Theobalt*, Sergey Tulyakov*, Vladislav Golyanik*, Elisa Ricci*

Work partially done while interning at MPI for Informatics
* Equal senior contribution

Paper Github Supplementary PDF
Overview Datasets Interactive Videos Action Conditioning Reconstruction Camera Manipulation Style Manipulation

Camera Manipulation Results

The following videos show the camera manipulation capabilities of our method.

Minecraft

In the following we show reconstructed sequences from the Minecraft Camera dataset. We show our reconstructions on the top row and ground truth in the bottom row. A single initial frame is observed by the model and novel views are rendered following the camera trajectory in the ground truth sequence. Differently from the baselines, our method correctly models camera movement and synthesizes views from the novel camera perspective.

Baselines:

Tennis

In the following we show sequences from the Tennis dataset rendered under randomly sampled camera trajectories. Several camera manipulations are possible, making it possible to generate more dramatic views of the scene.


Rotations, Translations and Focal Length Manipulation

To better illustrate the possible manipulations of the camera we produce videos where we separately manipulate camera rotations, camera translations and focal length respectively.