Pith. sign in

REVIEW 4 major objections 4 minor 38 references

HERA: Historical Evidence Routing Adapter for Physical Prediction in Latent World Models

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Routing stored historical evidence into a frozen V-JEPA 2 predictor lifts physical-prediction accuracy on IntPhys2 from 52.57% to 54.35%.

desk verdict A clean, parameter-efficient register-routing adapter for frozen video predictors, but the headline IntPhys2 gain is likely within noise and the physical-memory interpretation outruns the evidence. read the letter →

arxiv 2608.05523 v1 pith:GEESUNYY submitted 2026-08-06 cs.CV

Yuanruyi , Yue Cao , Haojia Gao , Guanqiu Guo , Ziyuezhang , Shangqin , Junbo Tan , Bokui Chen
show 2 more authors
Zhuo Zou Xueqian Wang
This is my paper · ORCID
classification cs.CV
keywords latentworldmodelsphysicalmemoryvideopredictionhistoricalevidenceroutingIntPhys2registertokensV-JEPA2occlusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a latent video world model's failures on physical events under occlusion are not primarily a storage problem: the history is in the context, but early evidence gets diluted before it can influence a later prediction. The proposed fix, HERA with Register-Routed Patch Memory (RRPM), leaves a pretrained V-JEPA 2 predictor completely frozen and adds a 3.00-million-parameter trainable interface that organizes past patch evidence into temporally structured memory and routes selected evidence back into prediction through dedicated registers. On the IntPhys2 possible-versus-impossible pairs, the adapter lifts pairwise AvgSurprise accuracy from 52.57% to 54.35%, with the largest gains on fixed-camera continuity (46.15% to 57.69%) and immutability (46.15% to 63.46%). If the claim holds, physical memory can be added to general-purpose predictive video models cheaply, without object supervision and without retraining the backbone.

What carries the argument

The carrying object is the Register-Routed Patch Memory (RRPM) adapter, which separates retention, retrieval, and integration. A Structured Memory Bank converts the projected context into 196 memory tokens grouped by temporal role: anchor memory preserves the earliest evidence such as identity, color, and initial position; middle memory compresses interaction history; recent memory holds the pre-prediction state; and global memory provides a coarse summary of the whole context. Memory Registers, a trainable set of 12 tokens appended to the predictor input, retrieve from the bank through gated cross-attention after the fourth and eighth frozen predictor blocks, with the gate initialized so that training begins near the frozen predictor's behavior. Workspace Registers, a separate trainable set of 8 tokens, participate in the normal self-attention of the frozen blocks and thereby carry the retrieved evidence into the prediction. Because only the registers are updated at the read layers, the context and mask tokens are never rewritten, so the frozen predictor operates in its native latent workspace while memory enters through a small set of dedicated carriers; the ablations indicate that this split of functions is load-bearing.

What would settle it

Take the trained HERA checkpoint and, at inference only, delete the anchor memory group from the Structured Memory Bank (or swap it with the recent group). If pairwise accuracy stays at 54.35% and, in particular, fixed-camera immutability stays at 63.46%, then the temporal-role structure the paper credits with recovering early evidence is not doing that work. A second check: train RRPM on Physion clips with the frame order reversed; if the IntPhys2 gain persists when temporal causality is destroyed, the improvement is a loss-reduction artifact rather than routed physical evidence.

Watch

Extended reading notes

Core claim

The central claim is that selective routing of retained historical evidence into a frozen latent predictor is a viable mechanism for physical memory -- the capacity to preserve and recover evidence about existence, attributes, motion, and interaction after it leaves the screen. Concretely, HERA with RRPM improves V-JEPA 2-G's pairwise AvgSurprise accuracy on the IntPhys2 Main split from 52.57% to 54.35% while adding only 3.00M trainable parameters and leaving the context encoder, target encoder, and all predictor blocks frozen under the unchanged future-latent prediction objective. The improvement concentrates in the subgroups whose decisive evidence appears early and then disappears: fixed-camera continuity rises from 46.15% to 57.69% and fixed-camera immutability from 46.15% to 63.46%, while hard examples and solidity each gain about 8-9 points. Against five alternative memory mechanisms -- clip-local patch memory, prefix cross-window memory, hierarchical patch memory, shared-register memory, and direct-to-target memory -- and against ablations that remove either register group, the split between Memory Registers, which retrieve stored evidence, and Workspace Registers, which integrate it through the frozen self-attention pathway, is what carries the advantage. The paper's conclusion is that a predictor needs not just preserved history but a route by which relevant evidence reaches the prediction when needed, and that this route can be added without disturbing the pretrained latent workspace.

Load-bearing premise

The whole result rests on the validity of the IntPhys2 AvgSurprise probe: that the pairwise margin in per-window future-latent $\ell^1$ prediction error between matched impossible and possible videos measures whether the model recovered hidden physical evidence, rather than merely how well the adapter optimized the same $\ell^1$ objective on its Physion training distribution and how that transfers to IntPhys2's low-level statistics.

Editorial extensions

If this is right

  • Storing more or longer history is not the binding constraint: all five alternative memory mechanisms that cache, compress, or hierarchically organize past tokens trail selective routing through registers under the same protocol.
  • Physical memory can be added to a frozen pretrained latent predictor with a 3.00M-parameter adapter, so improving occlusion-sensitive physical reasoning need not involve fine-tuning or retraining the backbone.
  • The gains are concentrated where evidence must be recovered after vanishing from view, which supports the paper's interpretation of the adapter as recovering physical evidence rather than improving prediction uniformly.
  • Separating retrieval from integration matters: a shared-register design underperforms the split design, and removing either register group from a trained checkpoint costs 0.39 accuracy points, so mixing workspace and memory roles in one token set appears to create interference.
  • Routing through a frozen pathway is also competitive on parameters: the direct-to-target variant, which unfreezes the final predictor blocks and tunes 5.96M parameters, still trails HERA.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report per-group ablations of the Structured Memory Bank; assigning the immutability gain specifically to anchor memory, the component it credits with preserving early identity and color, is a direct testable extension.
  • Because the adapter is trained with the same $\ell^1$ future-latent objective that defines the AvgSurprise probe (Eqs. 2 and 14), a stricter control would shuffle or drop the anchor memory group at inference, or evaluate on a violation benchmark with different low-level statistics, to confirm the margins track recovered physical evidence rather than generic loss reduction.
  • The split-register pattern is a generic adapter architecture for frozen predictors: the same routing idea could be carried to other settings where a later prediction depends on evidence that has left the input, such as audio event reasoning or embodied control, without object-centric supervision.
  • The temporal-role memory organization occupies a middle path between raw context caching and explicit object slots, avoiding both the dilution of early tokens and the requirement of stable object discovery from frozen features.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces HERA, a framework for routing retained historical evidence into a frozen latent video predictor, instantiated as Register-Routed Patch Memory (RRPM). RRPM adds a Structured Memory Bank, Memory Registers, and Workspace Registers to V-JEPA 2-G while keeping the encoder, target encoder, and predictor blocks frozen (3.00M trainable parameters). The method is trained on Physion with the base future-latent L1 objective and evaluated on the IntPhys2 Main split using the official AvgSurprise pairwise accuracy. The reported central result is an improvement from 52.57% to 54.35% pairwise accuracy over the frozen baseline, with larger subgroup gains on fixed-camera continuity and immutability. The paper also compares against five alternative memory mechanisms and reports register-removal ablations.

Significance. If the reported gains are statistically reliable, HERA would be a useful, parameter-efficient way to extend a frozen latent world model with historical evidence, and the separation of Memory Registers from Workspace Registers is a plausible design principle. The paper is also commendable on protocol hygiene: the same scorer is used for all methods, no IntPhys2 labels or pair identities are used in training, the backbone is frozen, and the comparison includes multiple architectural controls. However, the headline gain is small, and the significance of the result is not yet established because the paper lacks confidence intervals, significance tests, and a fixed, pre-registered context selection protocol. The subgroup claims are especially fragile because subset sizes and selected context lengths are not reported.

major comments (4)
  1. [Experiments, Table 1] The headline 1.78-point gain (V-JEPA 2-G 52.57% vs. HERA with RRPM 54.35% on 506 matched pairs) is not shown to exceed evaluation noise. For a single proportion near 0.53 with n=506, the standard error is about 2.2 percentage points, and the paired-difference standard error is not smaller; with no confidence intervals, no variance across seeds, and no paired significance test (e.g., McNemar or a paired bootstrap), the central claim that HERA improves pairwise accuracy is not statistically supported.
  2. [Problem Formulation, Eqs. (6)-(7) and Experiments] The reported accuracy is a maximum over an unspecified context set C, and Eq. (7) applies the maximum separately per subgroup. The paper never reports the candidate context lengths in C or the selected context length for each method and subgroup, so differences between rows in Tables 1 and 2 may reflect differential test-set selection rather than evidence routing. A fixed, pre-registered context protocol, or metrics averaged over C, is needed before the comparisons can be interpreted.
  3. [Table 2] The subgroup analysis reports no subset sizes. The 11.54-point gain on fixed-camera continuity (46.15% to 57.69%) and the 17.31-point gain on fixed-camera immutability (46.15% to 63.46%) could correspond to a handful of pair flips; without per-subgroup n, confidence intervals, or significance tests, these rows do not support the abstract's emphasis on strongly improved subgroup performance.
  4. [Method, Training and Inference, Eq. (14) vs. Eq. (2)] The adapter is trained with exactly the per-token L1 future-latent objective that defines AvgSurprise in Eq. (2), so an improvement on this probe can in principle reflect better optimization of the probe loss rather than specific retrieval of historical evidence. This is a shared property of the IntPhys2 evaluation protocol rather than a defect unique to HERA, but it is load-bearing for the physical-memory interpretation; a matched control (e.g., training RRPM with a shuffled or temporally reversed Structured Memory Bank under the same Eq. (14)) would isolate routing as the cause of the gain.
minor comments (4)
  1. [Abstract and text] The text contains many missing spaces between words (e.g., 'Yetthesemodels', 'Addressingthischallenge'), which should be fixed in a copyedit pass.
  2. [Table 2] The gain column is labeled with a percent sign, but the entries are percentage-point differences (e.g., 57.74 - 48.81 = 8.93 points); relabel the column to avoid ambiguity.
  3. [Experimental Setup] The method 'HERA + low-LR continuation' appears in Table 1 but is never described in the text; please define its training schedule and how it differs from the main HERA configuration.
  4. [Problem Formulation, Eq. (5)] The paper states that 'no ties occur in the reported evaluations,' but no tie counts are given; since ties are possible under Eq. (5), report the number of tied pairs per context length.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claim is an empirical transfer result against an external benchmark, with only minor non-circular robustness concerns.

full rationale

The paper's central claim is that HERA with RRPM improves pairwise AvgSurprise accuracy on the external IntPhys2 Main split. No load-bearing circular step is present. The adapter is trained on Physion with the future-latent L1 objective (Eq. 14), which is the same per-window loss used in the AvgSurprise probe (Eq. 2); however, the reported success metric is the pairwise margin (Eq. 4) on a different distribution, and the paper explicitly separates training from evaluation: 'The memory module is trained with the same future-latent objective as V-JEPA 2. Its physical value is judged only by whether it changes pairwise violation sensitivity.' The margin is not directly optimized, and IntPhys2 physical-validity labels are never used in training, so the improvement is not forced by construction. The best-context summary in Eqs. (6)-(7) is a maximum over a predefined context set and may inflate reported accuracies, but this is a statistical selection issue, not a circular reduction; the comparison is still made under the same protocol for all methods. No self-citation chain or imported uniqueness theorem is load-bearing, and the benchmark is external and falsifiable. The main risks are statistical power (506 pairs, no confidence intervals) and the construct validity of AvgSurprise as a physical-consistency measure, which are correctness concerns rather than circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 3 invented entities

The central claim depends on hand-chosen architectural hyperparameters (memory token counts, register counts, read layers, gate init) and on the assumption that the frozen V-JEPA 2 latents already contain recoverable evidence. The invented entities are architectural components, not physical postulates; they lack independent evidence beyond in-paper ablations. The probe-metric identity (training loss equals evaluation surprise) is a benchmark property, accounted under circularity.

free parameters (5)
  • Memory token counts (anchor, middle, recent, global) = 96/28/64/8
    Hand-chosen capacities for the Structured Memory Bank; they determine how much temporal evidence is retained and influence the accuracy gains.
  • Number of Memory Registers and Workspace Registers = 12 and 8
    Hand-chosen token counts for the two register groups; capacity affects retrieval and integration.
  • Register read layers L_read = {3, 7}
    Hand-chosen placement of gated cross-attention reads after the fourth and eighth predictor blocks; influences how much integration remains.
  • Gate initialization = not reported (described as conservative)
    The gate scales the cross-attention update; its initial value controls deviation from frozen behavior but is not specified numerically.
  • Evaluation context length selection = best over predefined set C
    Accbest(C) takes the maximum over context lengths, chosen once per method; this selection can inflate reported accuracy relative to a fixed context.
assumptions (4)
  • domain assumption V-JEPA 2-G frozen latents contain sufficient early object evidence to make retrieval useful
    The whole wiring of HERA presumes the evidence (identity, color, trajectory) exists somewhere in the frozen patch latents; if it is absent, routing cannot recover it. Invoked in Method, Structured Memory Bank.
  • domain assumption Future-latent L1 prediction error is a valid measure of physical surprise
    Eqs. (2)-(5) adopt the official IntPhys2 AvgSurprise scorer as a physical-consistency probe; the paper does not independently validate this assumption.
  • domain assumption Training on Physion transfers to IntPhys2 evaluation
    The adapter is trained on the 14,000-video Physion split and evaluated on IntPhys2 Main; transfer without domain shift is assumed (Experiments, Dataset).
  • domain assumption Temporal grouping (anchor/middle/recent) captures meaningful physical evidence without object annotations
    The Structured Memory Bank assigns temporal roles to memory tokens; the paper assumes these roles are physically informative and that the context mask yields a temporally regular tubelet view (Algorithm 1 fallback aside).
invented entities (3)
  • Structured Memory Bank
    purpose: Stores compressed patch history organized by temporal role (anchor, middle, recent, global) for later retrieval.
    The bank's behavior is supported only by in-paper ablations and comparisons; no external evidence confirms the tokens encode the claimed temporal roles.
  • Memory Registers
    purpose: Trainable tokens that query the Structured Memory Bank via gated cross-attention at read layers and carry evidence through predictor self-attention.
    Their contribution is shown by an internal ablation (removing them drops accuracy by 0.39 points); there is no out-of-paper evidence.
  • Workspace Registers
    purpose: Trainable tokens that integrate retrieved evidence through the frozen predictor's native self-attention without altering context tokens.
    Their contribution is shown by an internal ablation; no independent evidence exists outside this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HERA: Historical Evidence Routing Adapter for Physical Prediction in Latent World Models." pith.science (2026). https://pith.science/paper/GEESUNYY

@misc{pith2026260805523,
  author       = {Pith},
  title        = {Pith review of: HERA: Historical Evidence Routing Adapter for Physical Prediction in Latent World Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GEESUNYY}},
  note         = {Machine review of arXiv:2608.05523}
}
read the original abstract

Predictive video models have emerged as promising world models by learning latent visual dynamics from large-scale video. Yet these models remain challenged by physical events under occlusion, where later predictions may depend on object evidence that is no longer available in the current view. Addressing this challenge requires historical evidence not only to be preserved but also to remain accessible when it becomes relevant to a subsequent prediction. Existing approaches mainly enlarge the temporal context, cache generic video features, or impose explicit object-centric states, thereby improving the capacity or structure of retained history. However, they do not directly address how relevant historical evidence can be selectively retrieved and integrated into a pretrained predictor without interfering with its native latent workspace. Accordingly, we introduce HERA (Historical Evidence Routing Adapter), a framework for routing retained historical evidence into a frozen latent predictor, and instantiate it with Register-Routed Patch Memory (RRPM), a lightweight adapter comprising a Structured Memory Bank, Memory Registers, and Workspace Registers. On the IntPhys2 Main split, HERA with RRPM improves the pairwise AvgSurprise accuracy of V-JEPA 2-G from 52.57% to 54.35%. Subgroup analysis shows particularly strong improvements on fixed-camera continuity, from 46.15% to 57.69%, and fixed-camera immutability, from 46.15% to 63.46%. These results support historical evidence routing as a practical adaptation strategy for physical prediction in latent world models.

Figures

Figures reproduced from arXiv: 2608.05523 by the authors.

Figure 1
Figure 1. Architecture of HERA with RRPM. A frozen V-JEPA 2 context encoder produces tokens that are organized into temporally structured memory. Memory Registers retrieve this evidence through gated cross-attention, while Workspace Registers support its integration through the frozen predictor’s self-attention pathway. This formulation separates diagnosis from augmentation. The diagnostic output is a pairwise surprise margin… view at source ↗
Figure 2
Figure 2. Structured Memory Bank in RRPM. The Structured Memory Bank preserves historical patch evidence according to temporal role. Anchor memory stores early object evidence, middle memory summarizes motion and interaction history, recent memory stores the state immediately before prediction, and global memory provides a coarse context summary. The bank gives the predictor access to physically relevant history without requi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 23 canonical work pages

  1. [1]

    Agarwal, N.; Ali, A.; Bala, M.; Balaji, Y.; Barker, E.; Cai, T.; Chattopadhyay, P.; et al. 2025. Cosmos World Foundation Model Platform for Physical AI . arXiv:2501.03575

    arXiv 2025

  2. [2]

    Latent Video Prediction Learns Better World Models

    Alrasheed, A. J.; Parast, A. Y.; Azam, B.; Bailey, J.; and Akhtar, N. 2026. Latent Video Prediction Learns Better World Models. arXiv:2605.15618

    work page Pith review arXiv 2026

  3. [3]

    Arnab, A.; Dehghani, M.; Heigold, G.; Sun, C.; Lu c i \'c , M.; and Schmid, C. 2021. ViViT : A Video Vision Transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6836--6846

    work page 2021

  4. [4]

    Assran, M.; Duval, Q.; Misra, I.; Bojanowski, P.; Vincent, P.; Rabbat, M.; LeCun, Y.; and Ballas, N. 2023. Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15619--15629

    work page 2023

  5. [5]

    S.; and Wasserman, S

    Baillargeon, R.; Spelke, E. S.; and Wasserman, S. 1985. Object Permanence in Five-Month-Old Infants. Cognition, 20(3): 191--208

    work page 1985

  6. [6]

    Bao, H.; Dong, L.; and Wei, F. 2022. BEiT : BERT Pre-Training of Image Transformers. In International Conference on Learning Representations

    work page 2022

  7. [7]

    Bardes, A.; Garrido, Q.; Ponce, J.; Chen, X.; Rabbat, M.; LeCun, Y.; Assran, M.; and Ballas, N. 2024. V-JEPA : Latent Video Prediction for Visual Representation Learning. arXiv:2404.08471

    arXiv 2024

  8. [8]

    Bardes, A.; et al. 2025. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv:2506.09985

    arXiv 2025

Show all 38 references
  1. [9]

    E.; and Cohan, A

    Beltagy, I.; Peters, M. E.; and Cohan, A. 2020. Longformer: The Long-Document Transformer. arXiv:2004.05150

    2020 arXiv

  2. [10]

    Bertasius, G.; Wang, H.; and Torresani, L. 2021. Is Space-Time Attention All You Need for Video Understanding? In Proceedings of the International Conference on Machine Learning

    2021

  3. [11]

    Bordes, F.; et al. 2025. IntPhys 2: Benchmarking Intuitive Physics Understanding in Video Models. arXiv:2503.20883

    2025

  4. [12]

    Borgeaud, S.; Mensch, A.; Hoffmann, J.; Cai, T.; Rutherford, E.; Millican, K.; van den Driessche, G. B.; Lespiau, J.-B.; Damoc, B.; Clark, A.; de Las Casas, D.; Guy, A.; Menick, J.; Ring, R.; Hennigan, T.; Huang, S.; Maggiore, L.; Jones, C.; Cassirer, A.; Brock, A.; Paganini, ...

    2022

  5. [13]

    Bruce, J.; Dennis, M.; Edwards, A.; Parker-Holder, J.; Shi, Y.; Hughes, E.; Lai, M.; Mavalankar, A.; Steigerwald, R.; Apps, C.; Aytar, Y.; Bechtle, S.; Behbahani, F.; Chan, S. C. Y.; Heess, N.; Gonzalez, L. C.; Osindero, S.; Ozair, S.; Reed, S.; Zhang, J.; Zolna, K.; Clune, J....

    2024 arXiv

  6. [14]

    Bulatov, A.; Kuratov, Y.; and Burtsev, M. 2022. Recurrent Memory Transformer. arXiv:2207.06881

    2022 arXiv

  7. [15]

    V.; and Salakhutdinov, R

    Dai, Z.; Yang, Z.; Yang, Y.; Carbonell, J.; Le, Q. V.; and Salakhutdinov, R. 2019. Transformer- XL : Attentive Language Models Beyond a Fixed-Length Context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2978--2988

    2019

  8. [16]

    Darcet, T.; Oquab, M.; Mairal, J.; and Bojanowski, P. 2024. Vision Transformers Need Registers. In International Conference on Learning Representations

    2024

  9. [17]

    F.; Mahendran, A.; van Steenkiste, S.; Greff, K.; Mozer, M

    Elsayed, G. F.; Mahendran, A.; van Steenkiste, S.; Greff, K.; Mozer, M. C.; and Kipf, T. 2022. SAVi ++: Towards End-to-End Object-Centric Learning from Real-World Videos. arXiv:2206.07764

    2022 arXiv

  10. [18]

    Feichtenhofer, C.; Fan, H.; Li, Y.; and He, K. 2022. Masked Autoencoders as Spatiotemporal Learners. In Advances in Neural Information Processing Systems

    2022

  11. [19]

    Ha, D.; and Schmidhuber, J. 2018. World Models. arXiv:1803.10122

    2018 arXiv

  12. [20]

    Hafner, D.; Lillicrap, T.; Ba, J.; and Norouzi, M. 2020. Dream to Control: Learning Behaviors by Latent Imagination. In International Conference on Learning Representations

    2020

  13. [21]

    Hafner, D.; Lillicrap, T.; Fischer, I.; Villegas, R.; Ha, D.; Lee, H.; and Davidson, J. 2019. Learning Latent Dynamics for Planning from Pixels. In Proceedings of the 36th International Conference on Machine Learning, 2555--2565

    2019

  14. [22]

    Hafner, D.; Pasukonis, J.; Ba, J.; and Lillicrap, T. 2023. Mastering Diverse Domains through World Models. arXiv:2301.04104

    2023 arXiv

  15. [23]

    Hansen, N.; Su, H.; and Wang, X. 2022. Temporal Difference Learning for Model Predictive Control. In Proceedings of the 39th International Conference on Machine Learning

    2022

  16. [24]

    He, K.; Chen, X.; Xie, S.; Li, Y.; Doll \'a r, P.; and Girshick, R. 2022. Masked Autoencoders Are Scalable Vision Learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16000--16009

    2022

  17. [25]

    Hong, Y.; Mei, Y.; Ge, C.; Xu, Y.; Zhou, Y.; Bi, S.; Hold-Geoffroy, Y.; Roberts, M.; Fisher, M.; Shechtman, E.; Sunkavalli, K.; Liu, F.; Li, Z.; and Tan, H. 2025. RELIC : Interactive Video World Model with Long-Horizon Memory. arXiv:2512.04040

    2025

  18. [26]

    Joseph, S.; Garrido, Q.; Balestriero, R.; Kowal, M.; Fel, T.; Bakhtiari, S.; Richards, B.; and Rabbat, M. 2026. Interpreting Physics in Video World Models. arXiv:2602.07050

    2026

  19. [27]

    F.; Mahendran, A.; Stone, A.; Sabour, S.; Heigold, G.; Jonschkowski, R.; Dosovitskiy, A.; and Greff, K

    Kipf, T.; Elsayed, G. F.; Mahendran, A.; Stone, A.; Sabour, S.; Heigold, G.; Jonschkowski, R.; Dosovitskiy, A.; and Greff, K. 2022. Conditional Object-Centric Learning from Video. In International Conference on Learning Representations

    2022

  20. [28]

    Locatello, F.; Weissenborn, D.; Unterthiner, T.; Mahendran, A.; Heigold, G.; Uszkoreit, J.; Dosovitskiy, A.; and Kipf, T. 2020. Object-Centric Learning with Slot Attention. In Advances in Neural Information Processing Systems

    2020

  21. [29]

    Oshima, Y.; Iwasawa, Y.; Suzuki, M.; Matsuo, Y.; and Furuta, H. 2025. WorldPack : Dynamic Frame Compression for Long-context Video World Modeling. arXiv:2512.02473

    2025 arXiv

  22. [30]

    S.; Weinstein, A.; Battaglia, P.; and Botvinick, M

    Piloto, L. S.; Weinstein, A.; Battaglia, P.; and Botvinick, M. 2018. Probing Physics Knowledge Using Tools from Developmental Psychology. In Proceedings of the Annual Meeting of the Cognitive Science Society

    2018

  23. [31]

    W.; Potapenko, A.; Jayakumar, S

    Rae, J. W.; Potapenko, A.; Jayakumar, S. M.; Hillier, C.; and Lillicrap, T. P. 2020. Compressive Transformers for Long-Range Sequence Modelling. In International Conference on Learning Representations

    2020

  24. [32]

    Riochet, R.; Castro, M.; Bernard, M.; Lerer, A.; Fergus, R.; Izard, V.; and Dupoux, E. 2018. IntPhys : A Framework and Benchmark for Visual Intuitive Physics Reasoning. In Advances in Neural Information Processing Systems

    2018

  25. [33]

    Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T.; Lillicrap, T.; and Silver, D. 2020. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. Nature, 588: 604--609

    2020

  26. [34]

    Tong, Z.; Song, Y.; Wang, J.; and Wang, L. 2022. VideoMAE : Masked Autoencoders Are Data-Efficient Learners for Self-Supervised Video Pre-Training. In Advances in Neural Information Processing Systems

    2022

  27. [35]

    Wu, T.; Yang, S.; Po, R.; Xu, Y.; Liu, Z.; Lin, D.; and Wetzstein, G. 2025. Video World Models with Long-term Spatial Memory. arXiv:2506.05284

    2025 arXiv

  28. [36]

    N.; Hutchins, D.; and Szegedy, C

    Wu, Y.; Rabe, M. N.; Hutchins, D.; and Szegedy, C. 2022. Memorizing Transformers. arXiv:2203.08913

    2022 arXiv

  29. [37]

    Xiang, F.; et al. 2021. Physion : Evaluating Physical Prediction from Vision in Humans and Machines. In Advances in Neural Information Processing Systems

    2021

  30. [38]

    Yu, J.; Gao, J.; Bai, J.; Qin, Y.; Huang, K.; Liu, Q.; Wang, X.; Wan, P.; Gai, K.; and Liu, X. 2026. MemLearner : Learning to Query Context Memory for Video World Models. arXiv:2606.31734

    2026 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.