Playable Environments: Video Manipulation in Space and Time

CVPR 2022

Willi Menapace^☨, Stéphane Lathuilière^, Aliaksandr Siarohin,
Christian Theobalt^, Sergey Tulyakov^, Vladislav Golyanik^, Elisa Ricci^*

^☨ Work partially done while interning at MPI for Informatics
^* Equal senior contribution

Overview Datasets Interactive Videos Action Conditioning Reconstruction Camera Manipulation Style Manipulation

Reconstruction Results

The following videos show reconstructed videos produced by our method (top row) and compare them to the ground truth (bottom row). Our method can successfully reconstruct the movement of the players and move the camera as in the ground truth sequence. We perform the same analysis against the baselines which produce player motions that are less precise and do not correctly model camera movement. In particular, on the Minecraft dataset the baselines fail to generate the image due to the wide changes in the camera.