REVIEW 4 major objections 7 minor 35 references
SlotMem claims that a character's identity in long generated videos can be carried by a compact, continuously updated slot memory extracted from diffusion-transformer features, and that this beats keyframe-based memory on consistency benchm
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 22:22 UTC pith:YG7MEQZT
load-bearing objection Genuinely new slot-based character memory, but the empirical claims outrun the experimental design until a fine-tuning-only control is added. the 4 major comments →
SlotMem: Character-Addressable Internal Memory for Narrative Long Video Generation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
SlotMem represents each recurring character as a compact memory slot. At generation time, it reads cross-attention responses to the character's name, selects the visual tokens that respond to that name, compresses them with a learned encoder, and stores one slot per role. When the character reappears, Character-Wise Cross-Attention injects the slot only into the localized tokens of that same character. A Memory Writer then merges reliable new observations into the stored slot through a gated residual update. The paper reports that this slot-based memory improves long-range character similarity and subject consistency over baselines that retrieve whole keyframes, while keeping video quality c
What carries the argument
The central mechanism is the Character-Semantic Probe, which reads text-to-visual cross-attention responses for a character's name and turns them into a mask of that character's visual tokens. Those masked tokens are compressed by a Memory Encoder into a small set of slot queries, updated by a Memory Writer whose cosine-similarity gate admits only reliable residual corrections, and re-injected via Character-Wise Cross-Attention, which restricts the memory to tokens inside the character's mask. The probe makes the memory addressable by name, the slots keep it compact, and the writer makes it evolve across chunks.
Load-bearing premise
The whole pipeline rests on assuming that each character gets a stable, visually distinctive name in the captions and that cross-attention to that name reliably points at that character's pixels; when names are ambiguous, the probe mixes regions and the memory degrades.
What would settle it
Run SlotMem on a test set where one named character disappears for three or more chunks and reappears, with captions generated under varying levels of name ambiguity; if character similarity is not consistently higher than a keyframe-based memory baseline, or if probe masks on ambiguous names select more than a small fraction of non-character tokens, the central claim fails. A second check: strip out the Memory Writer and see whether long-gap consistency scores drop; the paper predicts they will.
If this is right
- Long-range character consistency can be maintained without storing full keyframes, so memory cost stays fixed as video length grows.
- Memory retrieval becomes character-addressable: a chunk about a particular character pulls only that character's slot, not generic context.
- Characters that vanish for several chunks and then reappear can be refreshed with new observations instead of being replaced by a new keyframe.
- The same role-wise slot design can be applied to any persistent visual entity that can be named in the prompt, not just people.
Where Pith is reading between the lines
- I would expect the method's ceiling to track caption quality: with richer attribute-level anchors, the probe's masks should be cleaner and the memory more disentangled; the paper points in this direction but leaves it untested.
- The conservative gated update suggests a natural stress test: delete the Memory Writer on videos with long gaps and measure how quickly identity similarity decays; the paper's ablations hint the writer matters most on exactly those cases.
- A broader testable extension would apply the same slot machinery to object permanence in world-model-style generation, since the probe only needs a name anchor to localize the entity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SlotMem, an internal memory framework for autoregressive long video generation built on Wan2.2-I2V. Recurring characters are represented by compact role-wise slot memory instead of keyframe-level memory. A Character-Semantic Probe localizes character-relevant visual tokens from cross-attention responses; a Memory Encoder compresses those tokens into slot memory; a Memory Writer updates the slots conservatively; and Character-Wise Cross-Attention injects the memory only into tokens of the corresponding character. The method is evaluated on VBench, ViStoryBench, and NarraStream-Bench, plus a human preference study, reporting improved long-range character consistency over several baselines while maintaining comparable video quality. The paper also releases code.
Significance. The core idea—replacing discrete keyframe memory with compressed, per-character slot memory that is updated over time—is technically coherent and a plausible step forward for long narrative video generation. The manuscript has clear strengths: the method is modular, the memory extraction/update/injection design is well specified, code is promised, and the internal ablations in Table 2 support the contribution of both the Memory Encoder and Memory Writer. However, the headline empirical claim against baselines is confounded by an unmatched training setup, and the evaluation is based on only 60 test samples without statistical precision. If the authors add a proper fine-tune-only control and report uncertainty/robustness, the core contribution could be solid.
major comments (4)
- [Sec. 4.1.2 / Table 1] The central claim that SlotMem improves character consistency over existing baselines is not cleanly supported because SlotMem is fine-tuned with rank-128 LoRA on 15,350 curated samples, whereas Wan2.2-I2V, StoryDiffusion, and IAMFlow are training-free (StoryMem uses its own fine-tuning). The ablations in Table 2 all include the same LoRA fine-tuning, so they cannot isolate the effect of fine-tuning itself. A Wan2.2+LoRA control trained on the same data but without the slot-memory modules is necessary to attribute the 0.7701→0.8603 Character Similarity and 0.8427→0.8771 Subject Consistency gains to the memory mechanism rather than to extra training signal. This confound is not listed in the limitations in Appendix E.
- [Sec. 4.1.3 / Table 3 / Appendix B] The evaluation set is only 60 test samples, with no error bars, confidence intervals, or statistical significance tests reported. The headline differences are modest (e.g. Character Similarity 0.8603 vs 0.8446 for IAMFlow; Subject Consistency 0.8771 vs 0.8524). Table 3 further shows that SlotMem does not consistently win on all VLM-sensitive metrics under all judges (e.g. Entity Grounding 0.7504 under Qwen3-VL-30B is below Wan2.2's 0.7993). Without repeated runs or variance estimates, it is impossible to tell which of the reported differences are meaningful and which are sampling noise.
- [Sec. 3.1 / Appendix E / Fig. 6] The Character-Semantic Probe is the sole addressing mechanism for memory extraction, update, and injection, but its reliability is inherited from the automatic captioning pipeline. The paper acknowledges the 'guitarist' failure case in Appendix E, where ambiguous captions produce mixed probe regions and degrade memory. However, the manuscript does not quantify how often such ambiguous naming occurs or how much it affects performance across the test set. Because the probe's masks are load-bearing for every memory operation, the paper should report robustness statistics—for example, agreement of probe masks with human-annotated character regions on a subset, or the rate of consistent character-name reuse across chunks.
- [Appendix D / Fig. 7] The human preference study reports only win/tie/lose percentages. No number of evaluators, total number of pairwise comparisons, or inter-rater agreement is given, and no statistical test is performed. Given that the study is used to support the claim of preferred quality, this lack of methodological detail weakens the evidence. Please add sample sizes, evaluator demographics (or at least recruitment information), and a significance test or confidence interval for the win rates.
minor comments (7)
- [Eq. (8)] The notation h^m_l(t_s) is used for hidden states of the memory chunk without being explicitly defined in the main text; please clarify that it denotes the DiT hidden states at layer l for the memory chunk at timestep t_s.
- [Algorithm 1] In line 9, online slots are accumulated as U^k ← U^k ∪ V^k_s across denoising steps. It would be helpful to specify how slots from different timesteps are combined or whether only a particular timestep is used.
- [Table 1 caption] The caption says 'All models use Wan2.2-I2V-A14B as the backbone,' but Appendix B states that StoryMem follows its original Wan2.2-based setting and IAMFlow was adapted from Wan2.1-T2V. Please align the caption with the actual backbone versions and clarify whether StoryMem uses the same Wan2.2 checkpoint.
- [Sec. 4.3] The statement that removing Character-Wise Cross-Attention 'would disable memory injection and reduce the model to the native Wan2.2 baseline' is inaccurate, since the model would still be LoRA-fine-tuned. It would be a Wan2.2+LoRA baseline, not the original native model.
- [Eq. (14)] The indicator function I[·] and the sigmoid σ(·) are used but not defined at first use; please add a brief explanation.
- [Appendix B, hyperparameters] The 'noise-domain boundary ratio' is listed as a hyperparameter but never defined. Please specify what this ratio controls in the Wan2.2 sampling process.
- [Fig. 2] The qualitative comparison would benefit from explicit mention of which video/chunk is shown and how representative the selected frames are; currently the selection criteria are unclear.
Circularity Check
No significant circularity: the central claim is supported by external benchmarks and ablations, and the probe's name-based addressing is a design choice rather than a fitted-prediction loop.
full rationale
The paper's core empirical claim—that SlotMem improves long-range character consistency—is evaluated on external benchmarks (VBench, ViStoryBench, NarraStream-Bench) and a human study, with no metric fitted from the model's own outputs. The Character-Semantic Probe uses cross-attention to character names, which is a self-referential definition of 'character-addressable' memory, but this is an architectural choice, not a derivation that reduces the benchmark results to the method's inputs. The ablations in Table 2 compare variants within the same fine-tuned setup, and while the absence of a fine-tune-only control is a legitimate experimental confound, it is not a circularity: the gains are not guaranteed by construction. Appendix E openly acknowledges a limitation with ambiguous character anchors ('the guitarist'), which is a failure mode, not a circular step. There are no self-citations or imported uniqueness theorems that carry the argument. The empirical improvements, if valid, come from the memory modules interacting with the pretrained backbone and external evaluation, so the derivation chain is not equivalent to its inputs.
Axiom & Free-Parameter Ledger
free parameters (6)
- Memory top-p ratio (p) =
0.1
- Contrastive slot loss weight (lambda_slot) =
0.02
- Memory update similarity threshold (tau_prec)
- Number of slot queries (R)
- Noise-domain boundary ratio =
0.9
- LoRA rank =
128
axioms (5)
- domain assumption Text-to-visual cross-attention responses in the DiT localize character-relevant tokens.
- domain assumption Chunk-level prompts consistently reuse visually discriminative character names.
- domain assumption Latent-space tokens preserve spatial correspondence to pixel regions.
- domain assumption The pretrained WAN2.2-I2V model provides a valid, character-sensitive backbone.
- standard math Autoregressive factorization of the video distribution into chunks with local context and retrieved memory.
invented entities (1)
-
Role-wise character memory slots
no independent evidence
read the original abstract
Maintaining recurring character identities across scene transitions and long temporal gaps is a central challenge in narrative long video generation. Methods targeting global consistency often retrieve memory using cues that are not aligned with character identity preservation, while recent character-centric variants still rely on coarse frame-level kv memory that entangles identity with incidental visual factors and lacks a continuous update mechanism under limited memory capacity. To address these limitations, we propose \textbf{SlotMem}, a character-addressable internal memory framework for multi-character narrative long video generation. Specifically, SlotMem uses a Character-Semantic Probe to localize character-relevant visual tokens from cross-attention responses, and a Memory Encoder to compress DiT tokens into compact role-wise slot memory. As generation proceeds, a Memory Writer conservatively updates each character's memory with new observations, while Character-Wise Cross-Attention retrieves the role memory and injects it only into localized tokens of the same character. Experiments on multiple narrative long video generation benchmarks show that SlotMem improves long-range character consistency over existing baselines, while maintaining comparable video quality. Our code is available at https://github.com/YilaiLiu-HKU/SlotMem.
Figures
Reference graph
Works this paper leans on
-
[1]
Onestory: Coherent multi-shot video generation with adaptive memory,
Zhaochong An, Menglin Jia, Haonan Qiu, Zijian Zhou, Xiaoke Huang, Zhiheng Liu, Weiming Ren, Kumara Kahatapitiya, Ding Liu, Sen He, Chenyang Zhang, Tao Xiang, Fanny Yang, Serge Belongie, and Tian Xie. Onestory: Coherent multi-shot video generation with adaptive memory,
-
[2]
Mixture of contexts for long video generation
Shengqu Cai, Ceyuan Yang, Lvmin Zhang, Yuwei Guo, Junfei Xiao, Ziyan Yang, Yinghao Xu, Zhenheng Yang, Alan Yuille, Leonidas Guibas, Maneesh Agrawala, Lu Jiang, and Gordon Wetzstein. Mixture of contexts for long video generation. InICLR, 2026
2026
-
[3]
Self-forcing++: Towards minute-scale high-quality video generation, 2025
Justin Cui, Jie Wu, Ming Li, Tao Yang, Xiaojie Li, Rui Wang, Andrew Bai, Yuanhao Ban, and Cho-Jui Hsieh. Self-forcing++: Towards minute-scale high-quality video generation, 2025. URLhttps://arxiv.org/abs/2510.02283
Pith/arXiv arXiv 2025
-
[4]
Gemini 3 pro model card
Google DeepMind. Gemini 3 pro model card. https://deepmind.google/models/ model-cards/gemini-3-pro , May 2026. Model released in November 2025; model card updated in May 2026
2026
-
[5]
Long context tuning for video generation, 2025
Yuwei Guo, Ceyuan Yang, Ziyan Yang, Zhibei Ma, Zhijie Lin, Zhenheng Yang, Dahua Lin, and Lu Jiang. Long context tuning for video generation, 2025. URL https://arxiv.org/abs/ 2503.10589
Pith/arXiv arXiv 2025
-
[6]
Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tade- vosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Consistent, dynamic, and extendable long video generation from text.arXiv preprint arXiv:2403.14773, 2024
Pith/arXiv arXiv 2024
-
[7]
Self forcing: Bridging the train-test gap in autoregressive video diffusion, 2025
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion, 2025. URL https://arxiv. org/abs/2506.08009
Pith/arXiv arXiv 2025
-
[8]
VBench: Comprehensive benchmark suite for video generative models
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024
2024
-
[9]
Memflow: Flowing adaptive memory for consistent and efficient long video narratives, 2025
Sihui Ji, Xi Chen, Shuai Yang, Xin Tao, Pengfei Wan, and Hengshuang Zhao. Memflow: Flowing adaptive memory for consistent and efficient long video narratives, 2025. URL https://arxiv.org/abs/2512.14699. 10
arXiv 2025
-
[10]
Kling-omni technical report, 2025
Kling Team et al. Kling-omni technical report, 2025. URL https://arxiv.org/abs/2512. 16776
2025
-
[11]
Stable video infinity: Infinite-length video generation with error recycling, 2025
Wuyang Li, Wentao Pan, Po-Chien Luan, Yang Gao, and Alexandre Alahi. Stable video infinity: Infinite-length video generation with error recycling, 2025. URL https://arxiv.org/abs/ 2510.09212
arXiv 2025
-
[12]
Advancing narrative long video generation via training-free identity-aware memory, 2026
Jinzhuo Liu, Jiangning Zhang, Wencan Jiang, Yabiao Wang, Dingkang Liang, Zhucun Xue, Ran Yi, and Yong Liu. Advancing narrative long video generation via training-free identity-aware memory, 2026
2026
-
[13]
Rolling forcing: Autoregressive long video diffusion in real time, 2025
Kunhao Liu, Wenbo Hu, Jiale Xu, Ying Shan, and Shijian Lu. Rolling forcing: Autoregressive long video diffusion in real time, 2025. URLhttps://arxiv.org/abs/2509.25161
Pith/arXiv arXiv 2025
-
[14]
Holocine: Holistic generation of cinematic multi-shot long video narratives, 2025
Yihao Meng, Hao Ouyang, Yue Yu, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Hanlin Wang, Yixuan Li, Cheng Chen, Yanhong Zeng, Yujun Shen, and Huamin Qu. Holocine: Holistic generation of cinematic multi-shot long video narratives, 2025. URL https://arxiv.org/ abs/2510.20822
arXiv 2025
-
[15]
Introducing gpt-4.1 in the api
OpenAI. Introducing gpt-4.1 in the api. https://openai.com/index/gpt-4-1/ , April 2025
2025
-
[16]
Animaker: Multi-agent animated storytelling with mcts-driven clip generation, 2025
Haoyuan Shi, Yunxin Li, Xinyu Chen, Longyue Wang, Baotian Hu, and Min Zhang. Animaker: Multi-agent animated storytelling with mcts-driven clip generation, 2025. URL https:// arxiv.org/abs/2506.10540
arXiv 2025
-
[17]
Seedance 2.0: Advancing video generation for world complexity, 2026
Team Seedance et al. Seedance 2.0: Advancing video generation for world complexity, 2026
2026
-
[18]
Wan: Open and advanced large-scale video generative models, 2025
Wan Team et al. Wan: Open and advanced large-scale video generative models, 2025. URL https://arxiv.org/abs/2503.20314
Pith/arXiv arXiv 2025
-
[19]
Multishotmaster: A controllable multi-shot video generation framework, 2025
Qinghe Wang, Xiaoyu Shi, Baolu Li, Weikang Bian, Quande Liu, Huchuan Lu, Xintao Wang, Pengfei Wan, Kun Gai, and Xu Jia. Multishotmaster: A controllable multi-shot video generation framework, 2025. URLhttps://arxiv.org/abs/2512.03041
arXiv 2025
-
[20]
Automated movie generation via multi-agent cot planning, 2025
Weijia Wu, Zeyu Zhu, and Mike Zheng Shou. Automated movie generation via multi-agent cot planning, 2025. URLhttps://arxiv.org/abs/2503.07314
Pith/arXiv arXiv 2025
-
[21]
Bissyand, and Saad Ezzini
Zhifei Xie, Daniel Tang, Dingwei Tan, Jacques Klein, Tegawend F. Bissyand, and Saad Ezzini. Dreamfactory: Pioneering multi-scene long video generation with a multi-agent framework,
-
[22]
Boxun Xu, Yuming Du, Zichang Liu, Siyu Yang, Ziyang Jiang, Siqi Yan, Rajasi Saha, Albert Pumarola, Wenchen Wang, and Peng Li. Sparse forcing: Native trainable sparse attention for real-time autoregressive diffusion video generation, 2026. URL https://arxiv.org/abs/ 2604.21221
Pith/arXiv arXiv 2026
-
[23]
Jiwen Yu, Jianhong Bai, Yiran Qin, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. Context as memory: Scene-consistent interactive long video generation with memory retrieval.arXiv preprint arXiv:2506.03141, 2025
Pith/arXiv arXiv 2025
-
[24]
Videossm: Autoregressive long video generation with hybrid state-space memory, 2025
Yifei Yu, Xiaoshan Wu, Xinting Hu, Tao Hu, Yangtian Sun, Xiaoyang Lyu, Bo Wang, Lin Ma, Yuewen Ma, Zhongrui Wang, and Xiaojuan Qi. Videossm: Autoregressive long video generation with hybrid state-space memory, 2025. URLhttps://arxiv.org/abs/2512.04519
arXiv 2025
-
[25]
Storymem: Multi-shot long video storytelling with memory, 2025
Kaiwen Zhang, Liming Jiang, Angtian Wang, Jacob Zhiyuan Fang, Tiancheng Zhi, Qing Yan, Hao Kang, Xin Lu, and Xingang Pan. Storymem: Multi-shot long video storytelling with memory, 2025. URLhttps://arxiv.org/abs/2512.19539
arXiv 2025
-
[26]
Frame context packing and drift prevention in next-frame-prediction video diffusion models, 2025
Lvmin Zhang, Shengqu Cai, Muyang Li, Gordon Wetzstein, and Maneesh Agrawala. Frame context packing and drift prevention in next-frame-prediction video diffusion models, 2025. URLhttps://arxiv.org/abs/2504.12626. 11
arXiv 2025
-
[27]
Pretraining frame preservation in autoregressive video memory compression, 2025
Lvmin Zhang, Shengqu Cai, Muyang Li, Chong Zeng, Beijia Lu, Anyi Rao, Song Han, Gordon Wetzstein, and Maneesh Agrawala. Pretraining frame preservation in autoregressive video memory compression, 2025. URLhttps://arxiv.org/abs/2512.23851
Pith/arXiv arXiv 2025
-
[28]
Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. VBench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025
Pith/arXiv arXiv 2025
-
[29]
Videomemory: Toward consistent video generation via memory integration, 2026
Jinsong Zhou, Yihua Du, Xinli Xu, Luozhou Wang, Zijie Zhuang, Yehang Zhang, Shuaibo Li, Xiaojun Hu, Bolan Su, and Ying-cong Chen. Videomemory: Toward consistent video generation via memory integration, 2026. URLhttps://arxiv.org/abs/2601.03655
arXiv 2026
-
[30]
StoryDiffusion: Consistent self-attention for long-range image and video generation
Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. StoryDiffusion: Consistent self-attention for long-range image and video generation. InAdvances in Neural Information Processing Systems, 2024
2024
-
[31]
Vistorybench: Comprehensive benchmark suite for story visualization,
Cailin Zhuang, Ailin Huang, Yaoqi Hu, Jingwei Wu, Wei Cheng, Jiaqi Liao, Hongyuan Wang, Xinyao Liao, Weiwei Cai, Hengyuan Xu, Xuanyang Zhang, Xianfang Zeng, Zhewei Huang, Gang Yu, and Chi Zhang. Vistorybench: Comprehensive benchmark suite for story visualization,
-
[35]
{Current Chunk} Curated Video TransNet v2 + Optical Flow Group Group Group Chunk Chunk Chunk young man: A clean-shaven white man with neatly parted dark hair, wearing a striped gray prison shirt; old man: ... young man continues speaking to old man, lowering his gaze toward the game before looking up again with a thoughtful ... Curated Captions & ChunksHu...
arXiv 2042
-
[2024]
URLhttps://arxiv.org/abs/2408.11788
-
[2025]
URLhttps://arxiv.org/abs/2512.07802
-
[2026]
12 A Inference Phase Pipeline Algorithm 1: Inference of SlotMem Require: Chunk prompts P= [p 1,
URLhttps://arxiv.org/abs/2505.24862. 12 A Inference Phase Pipeline Algorithm 1: Inference of SlotMem Require: Chunk prompts P= [p 1, . . . ,pK], initial image condition I0, denoising timesteps {ts}S s=1, selected layer set Lp used for probing, memory extraction, and injection, layer-group mapg(ℓ p)forℓ p ∈ Lp, group-specific memory encoders{E g}, Memory W...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.