OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

Mastering Atari with Discrete World Models

A comprehensive list of papers for the definition of World Models and using World Models for General Video Generation, Embodied AI, and Autonomous Driving, including papers, codes, and related websites.

[ICCV 2025 & ICCV 2025 RIWM Outstanding Paper] Aether: Geometric-Aware Unified World Modeling

You can easily calculate FVD, PSNR, SSIM, LPIPS for evaluating the quality of generated or predicted videos.

Official implementation for NIPS'17 paper: PredRNN: Recurrent Neural Networks for Predictive Learning Using Spatiotemporal LSTMs.

The pytorch implementation of our CVPR 2023 paper "Conditional Image-to-Video Generation with Latent Flow Diffusion Models"

Stochastic Adversarial Video Prediction

Code release for "PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning" (ICML 2018)

[ICLR 2024] MogaNet: Efficient Multi-order Gated Aggregation Network

e3d-lstm; Eidetic 3D LSTM A Model for Video Prediction and Beyond

Official repository for "iVideoGPT: Interactive VideoGPTs are Scalable World Models" (NeurIPS 2024), https://arxiv.org/abs/2405.15223

Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks

Code release for "Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics" (CVPR 2019)

Official repository for "RLVR-World: Training World Models with Reinforcement Learning" (NeurIPS 2025), https://arxiv.org/abs/2505.13934

Video Predicting using ConvLSTM and pytorch

Code release for ICLR 2023 paper: SlotFormer on object-centric dynamics models

Surveillance Perspective Human Action Recognition Dataset: 7759 Videos from 14 Action Classes, aggregated from multiple sources, all cropped spatio-temporally and filmed from a surveillance-camera like position.

Official PyTorch implementation of "Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning" (CVPR 2021 Oral)

Official implementation of the paper Stochastic Latent Residual Video Prediction