🌐 Awesome Generalizable World Models

Mechanisms, generalization, and interpretability of learned world models.

Representation structure, latent dynamics, physical inference, and planning.

Research Map · Reading Guide · BibTeX · Related Resources · Contributing

36 papers across six topics · 9 supplementary papers · 45 papers in total.

Entries describe the mechanism, evaluation setting, and evidence for each contribution. New papers extend these six topics through a distinct method, analysis, or test of generalization. Each paper has one primary topic and is listed alphabetically within it.

Unseen compositions, dynamics shifts, task transfer, and adaptation.

• Behavior-Invariant Task Representations — Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning
  
  Fuyuan Qian et al. · Learns task representations invariant to behavior policy for offline-to-online adaptation.

• Considering Context — When World Models Need Context Encoders
  
  Oleg Smirnov et al. · Measures the prediction information supplied by context beyond an agent's history.

• Context and Diversity Matter — The Emergence of In-Context Learning in World Models
  
  Fan Wang et al. · Distinguishes recognizing familiar environments from learning dynamics through current experience.

• CaDM — Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning
  
  Kimin Lee et al. · Infers environment context from interaction history to model changing dynamics.

• Curious Replay — Curious Replay for Model-based Adaptation
  
  Isaac Kauvar et al. · Prioritizes stored experience using novelty and prediction error to support adaptation.

• Dreamweaver — Learning Compositional World Models from Pixels
  
  Junyeob Baek et al. · Learns object and attribute representations for combinations beyond training configurations.

• Latent Dynamics Geometries — Dynamics Are Learned, Not Told: Semi-Supervised Discovery of Latent Dynamics Geometries For Zero-Shot Policy Adaptation
  
  Zhiming Xu et al. · Learns trajectory-based dynamics representations for zero-shot policy adaptation to changed dynamics.

• General agents need world models
  
  Jonathan Richens et al. · Theory · Relates generalization across goal-directed tasks to predictive models extractable from a policy.

• WM3C — Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learning
  
  Xinyue Wang et al. · Uses language-guided causal components to model and adapt to unseen environments.

• HOWM — Toward Compositional Generalization in Object-Oriented World Modeling
  
  Linfeng Zhao et al. · Formalizes object composition and learns a differentiable world model for unseen combinations.

• WOMBET — World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning
  
  Mintae Kim et al. · Transfers model-generated experience from a source task to target-task fine-tuning.

Topic overview and comparisons · Contents

Physical variables, causal interventions, and interpretable model design.

• Physically Interpretable World Models — Four Principles for Physically Interpretable World Models
  
  Jordan Peper et al. · Organizes latent representations and supervision around physical meaning.

• Interpreting Physics — Interpreting Physics in Video World Models
  
  Sonia Joseph et al. · Analyzes the organization of physical information through layer-wise probes, subspaces, and interventions.

• Probing RL World Models — What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators
  
  Xinyu Zhang · Probes and intervenes on game-state representations in IRIS and DIAMOND.

• What Has a Foundation Model Found? — What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models
  
  Keyon Vafa et al. · Probes learned world structure through inductive biases in new tasks.

Topic overview and comparisons · Contents

Objects, relations, and modular transition mechanisms.

• Causal-JEPA — Learning World Models through Object-Level Latent Masking
  
  Heejeong Nam et al. · Combines object-level latent masking with joint-embedding predictive learning.

• C-SWM — Contrastive Learning of Structured World Models
  
  Thomas Kipf et al. · Learns object-centric representations and relational dynamics with a contrastive objective.

• NRI — Neural Relational Inference for Interacting Systems
  
  Thomas Kipf et al. · Infers latent interaction graphs from trajectories with a graph-based dynamics decoder.

• SPARTAN — A Sparse Transformer World Model Attending to What Matters
  
  Anson Lei et al. · Uses sparse, context-dependent interactions to adapt to changes in mechanisms.

Topic overview and comparisons · Contents

Latent geometry, learned transformations, and symmetry-based dynamics.

• GRWM — Cloning Deterministic Worlds: The Critical Role of Latent Geometry in Long-Horizon World Models
  
  Zaishuo Xia et al. · Regularizes latent geometry and uses temporal context for long-horizon prediction in deterministic environments.

• FloWM — Flow Equivariant World Models: Structured Memory for Dynamic Environments
  
  Hansen Lillemark et al. · Updates spatial latent memory with self-motion and object-motion flows for prediction under partial observability.

• Group-Structured Latents — Learning Abstract World Models with a Group-Structured Latent Space
  
  Thomas Delliaux et al. · Encodes known symmetries, such as translation and rotation, as geometric inductive biases.

• Learning Group Structure — Learning Disentangled Representations and Group Structure of Dynamical Environments
  
  Robin Quessard et al. · Learns latent group transformations and disentangled subspaces from interaction sequences for long-horizon prediction.

• SEN — Learning Symmetric Embeddings for Equivariant World Models
  
  Jung Yeon Park et al. · Learns an observation embedding compatible with known latent symmetries for equivariant transition prediction.

Topic overview and comparisons · Contents

Goal-directed prediction, task abstraction, and model-based action selection.

• Denoised MDPs — Learning World Models Better Than the World Itself
  
  Tongzhou Wang et al. · Separates controllable and reward-relevant information from other factors in observations.

• DINO-WM — World Models on Pre-trained Visual Features enable Zero-shot Planning
  
  Gaoyue Zhou et al. · Combines frozen visual representations, action-conditioned latent prediction, and goal-directed planning.

• Plan2Explore — Planning to Explore via Self-Supervised World Models
  
  Ramanan Sekar et al. · Learns exploration behavior from model disagreement before downstream tasks are specified.

• Value Equivalence — The Value Equivalence Principle for Model-Based Reinforcement Learning
  
  Christopher Grimm et al. · Formalizes models that preserve decision-relevant Bellman updates without reconstructing every world detail.

• V-JEPA 2 — Self-Supervised Video Models Enable Understanding, Prediction and Planning
  
  Mido Assran et al. · Combines video representation pretraining with action-conditioned post-training for robot planning.

• JEPA-WMs — What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?
  
  Basile Terver et al. · Compares architecture, training objectives, and planners in JEPA world models.

Topic overview and comparisons · Contents

Compositional prediction, physical inference, and closed-loop validity.

• A Physical Law Perspective — How Far Is Video Generation from World Model: A Physical Law Perspective
  
  Bingyi Kang et al. · Tests physical prediction under extrapolation and compositional distribution shifts.

• SVIB — Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Models
  
  Yeongbin Kim et al. · Controls training-factor coverage to test visual prediction on unseen combinations.

• Physion++ — Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties
  
  Hsiao-Yu Tung et al. · Tests physical predictions that require inferring hidden object properties from observed interactions.

• Physion — Evaluating Physical Prediction from Vision in Humans and Machines
  
  Daniel Bear et al. · Tests future object-contact prediction from video across eight simulated physical scenarios.

• World-In-World — World Models in a Closed-Loop World
  
  Jiahan Zhang et al. · Tests action-conditioned world models in closed-loop tasks, varying post-training data and test-time computation.

• WorldGym — World Model as An Environment for Policy Evaluation
  
  Julian Quevedo et al. · Tests whether policy success rates and rankings in a learned video environment track real-robot evaluations.

Topic overview and comparisons · Contents

Supplementary Reading covers world-model foundations, related representation methods, and simulation or system-identification studies. Each entry states its connection to the main topics.

The 12-paper reading path connects evidence about learned structure to composition, geometry, adaptation, planning, and generalization tests. Short topic branches link related methods and evaluation protocols.

Candidate papers are listed separately from the main index and its paper counts.

• Awesome World Models — World-model methods and applications across robotics, games, driving, and science.
• Awesome World Models for Robotic Policy Learning — World models as policies, simulators, and video predictors for robot learning.
• Learning to Model the World — A survey repository spanning world-model representations, applications, interpretability, and benchmarks.
• Awesome World Model Evaluation — Evaluation papers and metrics organized by modality and evaluation dimension.
• A Survey of World Model Benchmarks — Benchmarks organized by physical, causal, control, and functional evaluation targets.

Paper suggestions and corrections are welcome through issues or pull requests. See Contributing for inclusion criteria and the entry format. Bibliographic records are available in BibTeX.

Original catalog text and illustrations: CC BY 4.0. Maintenance code: MIT. Linked papers and external resources retain their respective licenses. See license scope.