{"id":"b210b38f-7a36-4d19-a3c8-a6b0ce98e4ba","arxiv_id":"2607.18436","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A training-free 'surprise' controller decides which old frames to keep in memory and which chunks need fewer denoising steps, improving long-video consistency at real-time speed.","lead":"This paper adds two training-free controllers to streaming video generation: an external memory that keeps surprising old frames, and an adaptive denoising schedule that skips steps on easy chunks. It improves long-video consistency on standard benchmarks while staying near real-time throughput.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Mean-pooled value-token descriptor (Eq. 5) is invariant to spatial layout, so the memory bank may treat pose/viewpoint-distinct frames as redundant; this directly threatens the long-horizon consistency claim.","rationale":"The central claim of the paper is that Surprise Forcing improves long-horizon consistency and visual quality without sacrificing real-time throughput. The strongest empirical evidence for the consistency part is the VBench-2.0 gains in Human Identity, Human Clothes, Instance Preservation, and Multi-View Consistency (Table 2). All of these depend on the memory bank retaining the right historical frames. The bank's write/evict/routing decisions are driven entirely by the descriptor in Eq. 5, which is a global mean of value tokens. A global mean is invariant to spatial permutation, so the bank cannot distinguish two frames with the same global statistics but different layout or pose. This is not a peripheral design choice: it is the only mechanism by which the bank can preserve 'identity, clothing, geometry, or scene layout' (as claimed in the introduction). The authors acknowledge this limitation in Section 5, making it an explicit internal admission that the core representation may be insufficient for the very task the method is designed to solve. The reader's weakest assumption identifies the same issue, and I agree. The concern is concrete and testable: if a spatially-aware descriptor (e.g., per-cell averaging) substantially improves the VBench-2.0 metrics on a pose/layout stress set, then the mean-pooled descriptor is the limiting factor and the current ablation results may overstate the method's ability to handle realistic long-horizon consistency. If it does not improve the metrics, then the concern, while theoretically valid, does not land in practice. Because the reader already returns a CONDITIONAL verdict, and this concern reinforces that conditionality rather than overturning it, the verdict remains UNCHANGED. I do not see a stronger or more load-bearing technical concern than this: the missing error bars and lack of released code are real but are evidentiary limitations, not flaws in the method's logic. The descriptor issue, by contrast, targets the actual mechanism that is claimed to produce the consistency gains.","tokens_in":12702,"tokens_out":5883,"duration_ms":60047,"concrete_test":"Construct a targeted stress set of 30 one-minute prompts in which the subject undergoes an explicit pose/viewpoint/object-layout change around second 30 (e.g., person turns, object rotates, two objects swap positions). Run the current Surprise Forcing pipeline and a variant whose descriptor is spatially aware (e.g., divide the L value tokens into K×K spatial cells, compute per-cell mean descriptors, and use the max/average of per-cell similarities for s_nov and s_pred), keeping all other components fixed. Measure VBench-2.0 Human Identity and Multi-View Consistency on both variants, and instrument how often the memory bank admits/retrieves a pre-change vs post-change frame. If the spatial-aware variant improves these scores by more than ~3 points, the mean-pooled descriptor is the bottleneck; if the scores are within noise, the concern is minor and the current descriptor is sufficient in","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Surprise-Gated Memory Bank's admission and routing decisions rest on the frame descriptor d = mean(V_l)/||mean(V_l)|| (Eq. 5). Mean pooling over all spatial value tokens destroys spatial arrangement: two frames with identical global color/texture but different object pose, layout, or identity will have near-identical descriptors. Sections 3.3.1 and 3.3.2 then treat them as redundant: s_nov (nearest-neighbor) will be low, so the second frame may fail admission or be evicted. This is precisely the situation that matters for the central claim of improved long-horizon consistency, especially for VBench-2.0 metrics like Multi-View Consistency (Table 2), where a later viewpoint may require a frame that was skipped because its mean descriptor resembled an earlier one. The authors themselves flag this in Section 5: 'the descriptor is a compact mean-pooled summary and may not distinguish spatial arrangements that share similar global content.' If this limitation is not merely a qualitative caveat but a practical failure mode, then the memory bank's improvement over LongLive could be smaller or even negative in long rollouts with repeated poses/scenes, undermining the headline claim. The concern is not that the descriptor is imperfect in the abstract, but that the mechanism's only representation of historical identity and layout is permutation-invariant, so the bank cannot, by construction, distinguish the very spatial/identity changes that consistency metrics are designed to detect.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Surprise Forcing, a training-free inference-time framework for streaming autoregressive video diffusion. It adds two controllers on top of a pretrained streaming generator: (1) a Surprise-Gated Memory Bank that decides which evicted frames to store using a mean-pooled value-token descriptor, a dual-component surprise score, a feedback-controlled admission threshold, priority-based eviction, and query-dependent routing; and (2) a Surprise-Aware Denoising scheduler that estimates chunk difficulty from the maximum adjacent-frame cosine distance after the first denoising pass and skips intermediate steps for chunks ranked below a local percentile. The authors evaluate on VBench, VBench-Long, and selected VBench-2.0 consistency metrics, reporting improved scores over CausVid, Self Forcing, Rolling Forcing, and LongLive while retaining real-time throughput (17.18 FPS on 60-second 832x480 rollouts).","tokens_in":12962,"tokens_out":10706,"duration_ms":92366,"significance":"If the reported results hold, the paper makes a useful contribution to efficient long-video generation. The core idea—extracting control signals from the generator's own intermediate representations—is interesting and goes beyond common cache/eviction heuristics. The explicit formulation of budget-norm gating, the component ablations, and the use of public benchmarks are strengths. The paper also provides a falsifiable check in Figure 4, where the intra-chunk surprise score is correlated with residual denoising error (PLCC 0.695, SRCC 0.730). However, the central claims are currently supported only by single-point benchmark scores without variance or significance testing, and the memory descriptor that drives admission and routing is a mean-pooled summary whose discriminative limits are acknowledged by the authors but not experimentally probed. These issues leave the strength of the central consistency claim somewhat uncertain.","major_comments":[{"comment":"The frame descriptor d = mean(V_l)/||mean(V_l)|| is a permutation-invariant summary of spatial value tokens. Since admission (Eq. 6), priority-based eviction (Eq. 11), and dynamic routing (§3.3.2) all operate on this descriptor, the memory bank cannot distinguish frames that share similar global statistics but differ in spatial layout, object pose, or identity. This is directly relevant to the claimed improvements on VBench-2.0 Human Identity and Multi-View Consistency in Table 2. The authors acknowledge the issue in Section 5, but no experiment quantifies its impact. Please either compare against a spatially-aware descriptor (e.g., token-wise max-pooling or a small set of spatial-region descriptors) under the same pipeline, or provide a targeted failure analysis on prompts with repeated poses/viewpoints. Without such evidence, the central consistency claim is not fully established.","section":"§3.3, Eq. (5)"},{"comment":"All quantitative results are single point estimates with no confidence intervals, standard deviations, or significance tests. Some of the headline improvements over LongLive are small on individual metrics (e.g., Table 1: VBench Subject Consistency 96.28 vs 96.19; VBench-Long Subject Consistency 98.51 vs 98.42). Because video generation is stochastic, these differences may be within run-to-run noise. Please report at least 3 seeds with standard deviations (or paired bootstrap/permutation tests on the fixed prompt sets) for the main benchmark comparisons and the key ablations, especially Tables 1, 2, 5, and 9. This is necessary to support the claim that the improvements are reliable.","section":"Tables 1-11"},{"comment":"The intra-chunk surprise predictor is validated on only 50 MovieGen prompts with moderate correlation (PLCC 0.695, SRCC 0.730). The scheduler then uses this predictor to skip denoising work, with the default r_skip=0.4 reported as reducing 'denoising work by 15.8%'. However, the reduced schedule {0,3} has half the transformer passes of {0,1,2,3}, so the relationship between the 'Acc. Ratio' and the fraction of actually skipped chunks is unclear and should be stated explicitly. Please (a) clarify what 'Acc. Ratio' measures, (b) report the scheduler-only effect on quality by disabling the memory bank or holding it constant, and (c) show, if possible, the distribution of residual errors for skipped versus non-skipped chunks to confirm that the skipped chunks are indeed the easy ones. This is load-bearing for the speed-quality trade-off claim.","section":"§3.4, Table 5"}],"minor_comments":[{"comment":"The row labeled 'rskip = 084.03' appears to be a formatting error; it should presumably read 'rskip = 0.0' followed by the score 84.03.","section":"Table 5"},{"comment":"The query descriptor d_q is used in cosine similarity for routing but is never explicitly defined. It should be stated whether d_q is the same mean-pooled value-token descriptor as Eq. (5) computed for the current generation query, and over which tokens.","section":"§3.3.2"},{"comment":"Please specify which transformer layer(s) provide the value tokens V_l, the index l (e.g., spatial position, head, or token), and how L is defined. This is needed for reproducibility.","section":"Eq. (5)"},{"comment":"The phrase 'neighboring-chunk throttling' (used to explain why final commit rate can be lower than r) is never defined. Please clarify whether it refers to a separate mechanism or to the warmup/eviction behavior.","section":"§3.3"},{"comment":"The caption claims 'the speedup becomes more pronounced when generating longer videos', but Table 1 shows Surprise Forcing at 16.45 FPS (5s) and 17.18 FPS (60s), both below LongLive's 18.01 FPS. This statement is not supported by the reported numbers. Please rephrase to 'throughput remains real-time and comparable' or provide the intended baseline for the claimed speedup.","section":"Table 1 caption and §4.2"},{"comment":"The phrase 'state-of-the-art performance across all metrics' is too broad for a comparison against five streaming baselines. It should be restricted to 'among the compared streaming alternatives'.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is well-written and the proposed training-free framework is interesting, with a clear methodological contribution and useful ablations. My main concern is experimental rigor: the absence of error bars/significance tests makes the small margins on several metrics difficult to interpret, and the mean-pooled descriptor is a known discriminative weakness that the authors acknowledge but do not address empirically. These are addressable with additional experiments rather than fundamental errors, so I recommend major revision rather than rejection. The paper would also benefit from a more careful discussion of what the FPS numbers actually show relative to the baselines."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Surprise Forcing is a solid, training-free patch for streaming video generators. It improves long-range consistency on VBench-Long and VBench-2.0 while skipping roughly 16% of denoising steps, and the proposed memory bank plus adaptive scheduling are well specified and properly ablated.\n\nWhat is actually new: a unified, training-free controller that combines surprise-gated external memory with feedback-controlled admission, priority-based eviction, dynamic routing, and chunk-level step skipping. Individual pieces exist in prior work; the integration and the budget-norm gating rule are a legitimate practical contribution. The ablations are unusually complete for this kind of paper — each component, capacity, routing strategy, admission ratio, eviction policy, and two surprise-mixture variants are tested. The intra-chunk difficulty predictor is validated against residual denoising error (PLCC 0.695, SRCC 0.730), which is decent evidence that the signal means something.\n\nSoft spots, in proportion. First, every quantitative result is a single run without error bars or significance tests. For a system with this many hyperparameters, that is a real limitation. Second, the hyperparameters (r_skip, α, capacities, priority weights) appear to be chosen on the same benchmark family used for the headline tables, so there is a mild selection-on-evaluation risk. Third, no code or data are available, so independent verification is not possible. Fourth, 'state-of-the-art across all metrics' overstates the case: throughput is lower than LongLive (17.18 vs 18.01 FPS), so the speedup claim only applies to denoising work, not wall-clock FPS. Fifth, the mean-pooled descriptor of Eq. 5 is indeed permutation-invariant, so it cannot distinguish frames with similar global statistics but different spatial layout. The authors acknowledge this in Section 5. The stress-test concern is theoretically valid, but I do not think it sinks the paper: the descriptor is used only for admission and routing, while the bank stores the actual frame tokens, and the empirical gains on Multi-View Consistency and Human Identity are consistent with the mechanism working despite the descriptor's coarseness. A more serious failure might appear in longer or more pose-repetitive rollouts, but the existing evidence does not show it.\n\nBottom line: this is a well-scoped, honest engineering paper. It deserves a serious referee. The fixes are straightforward — error bars, a held-out validation split for hyperparameters, and code release. I would read it and cite it if I worked on long video generation.","headline":"A practical training-free method that improves long-video consistency and compute allocation; needs error bars and code, but the central claim holds.","tokens_in":13575,"tokens_out":3570,"would_cite":true,"duration_ms":28845,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Long video generation can stay consistent at real-time speed by treating memory and denoising compute as online allocation problems driven by the model's own 'surprise' signals.","keywords":["long video generation","streaming autoregressive diffusion","memory management","surprise gating","denoising step scheduling","training-free inference","video consistency","real-time generation"],"falsifier":"Generate a video where a subject's average color and texture stay constant while its pose or layout changes drastically (e.g., a person walking across a uniformly colored room); if the Surprise-Gated Memory Bank fails to retain the earlier view and subject identity drifts on VBench-2.0 Human Identity, that confirms the descriptor collapses spatial distinctions. A cleaner test: replace the mean-pooled descriptor with spatially-partitioned patch descriptors and check whether consistency scores rise — if they do, the mean pooling is the limiting factor.","tokens_in":12458,"feed_emoji":"🎬","tokens_out":3291,"duration_ms":25488,"temperature":0.7,"pith_summary":"This paper asks how a streaming text-to-video model should decide which past frames to keep and how many denoising steps each new chunk deserves. It claims that both decisions can be made online, without retraining, by reading 'surprise' directly from the model's own intermediate representations. The proposed Surprise Forcing framework keeps a small external memory of surprising frames and skips intermediate denoising steps on easy chunks, and the authors report that this improves long-horizon consistency and visual quality while keeping generation real-time. A sympathetic reader would care because current streaming generators forget early visual evidence and spend equal compute on every chunk, and this work offers a training-free way to fix both.","feed_headline":"Surprise gating keeps long videos consistent at 17 FPS","feed_subtitle":"A training-free method reallocates memory and denoising compute by surprise, improving consistency while staying real-time.","key_machinery":"The load-bearing mechanism is the surprise score: for memory, a mixture of global deviation and nearest-neighbor novelty computed on L2-normalized mean-pooled value tokens (Equation 6), gated by Budget-Norm Gating, an online controller that adjusts the admission threshold to hold a target write ratio; for denoising, the maximum adjacent-frame cosine distance in the first-step latent (Equation 13), ranked within a sliding window to decide step skipping. These signals are causal, require no auxiliary network, and convert 'what is surprising' into concrete resource-allocation decisions.","core_discovery":"The central claim is that selective memory and selective computation, driven by surprise signals within the inference trajectory, improve long-horizon consistency and visual quality without changing the trained generator. Concretely, a Surprise-Gated Memory Bank scores evicted frames by the deviation of their mean-pooled value-token descriptor from the bank, admits them through a feedback-controlled budget in normalized score space, evicts by a composite priority of surprise, usage, and age, and routes only top-k relevant entries to attention. In parallel, Surprise-Aware Denoising estimates chunk difficulty from the maximum adjacent-frame cosine distance after the first denoising pass and as","pith_inferences":["The mean-pooled descriptor is the critical bottleneck: if a scene contains two states with identical global color/texture but different spatial layout, the bank may treat them as redundant and evict the discriminative frame, undermining the consistency claim; a multi-resolution or spatially-partitioned descriptor would be a natural test extension.","The feedback-controlled admission ratio r and the priority weights are fixed hyperparameters; a higher-level controller that adapts these to prompt structure or scene dynamics could further improve the stability–freshness trade-off the paper identifies.","Because the surprise signal is self-referential, it should be tested on longer-than-one-minute rollouts and interactive trajectories where error accumulation may change the statistics of surprise; the paper explicitly flags this as unestablished.","The binary {0,3} versus full schedule could be generalized to intermediate step counts or to variable step allocations across chunks, which the paper lists as future work."],"forward_implications":["If the central claim holds, streaming video generators can improve long-horizon consistency without retraining, by reallocating memory and compute at inference time.","The reported speedup of ~15.8% denoising work at r_skip=0.4 with minimal quality loss suggests that difficulty-adaptive step skipping is a viable real-time technique.","The method's strong Multi-View Consistency gain implies that preserving early establishing views in an external bank can recover viewpoint consistency that rolling caches lose.","Since the framework is training-free and complementary to distillation-based streaming generators, it can be layered on top of future base models."],"fun_headline_variants":["Surprise-driven memory and compute allocation for long videos","Skipping easy video chunks by surprise keeps consistency","Surprise gating trims memory and denoising steps in video gen","Real-time video gen with surprise-based memory and denoising"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole memory pipeline assumes that a single mean-pooled, L2-normalized value-token vector per frame is enough to tell which frames hold information worth keeping; if two frames have similar global content but different layout or pose, the bank will treat them as the same and may discard the frame that later matters.","fun_headline_variants_meta":{"raw":{"variants":["Surprise-driven memory and compute allocation for long videos","Skipping easy video chunks by surprise keeps consistency","Surprise gating trims memory and denoising steps in video gen","Real-time video gen with surprise-based memory and denoising"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000863,"raw_usage":{"total_tokens":3571,"prompt_tokens":729,"completion_tokens":2842,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":473,"completion_tokens_details":{"reasoning_tokens":2772}},"tokens_in":473,"tokens_out":2842,"duration_ms":19869,"temperature":1.0,"reasoning_tokens":2772,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T15:25:10.002391+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a video where a subject's average color and texture stay constant while its pose or layout changes drastically (e.g., a person walking across a uniformly colored room); if the Surprise-Gated Memory Bank fails to retain the earlier view and subject identity drifts on VBench-2.0 Human Identity, that confirms the descriptor collapses spatial distinctions. A cleaner test: replace the mean-pooled descriptor with spatially-partitioned patch descriptors and check whether consistency scores rise — if they do, the mean pooling is the limiting factor.","supporting_citations":[],"review_version":1}