Introduction#

The Matrix is the first world model that supports generating high-quality, infinite-time interactive videos in real-time. Previous world models were limited to generating simple 2D games like DOOM or Minecraft in low resolution, and struggled with essential abilities like generalization or infinite-length video generation.

The success of The Matrix in these areas is driven by the Swin-DPM model and a training strategy that fuses real and simulated data, contributing significantly to the system’s ability to generalize and generate infinite content.

The Matrix Team is dedicated to advancing global research and creating a better world for the well-being of all humankind. We’re excited to announce the open release of all necessary components to reproduce the results reported in our paper, including:

  • Training and inference scripts

  • Model checkpoints from various phases

  • Collected SOURCE dataset

  • Methodology for collecting the SOURCE dataset

Feel free to leverage our work to further the development of fundamental world simulations, for agent learning, AI gaming, interactive video generation, metaverse applications, and beyond—under the Apache 2.0 License.

This Page