{"id":"1952850e-b863-4397-8872-7a8002ab28cd","arxiv_id":"2607.08770","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":7,"one_line_summary":"Fine-tuning CogVideoX with autoregressive context management and bidirectional alignment enables a single model to perform event-based video reconstruction, prediction, and zero-shot interpolation with superior temporal stability.","lead":"This paper fine-tunes a pre-trained video diffusion model to reconstruct, predict, and interpolate video from event camera data within a single architecture. It introduces mechanisms to prevent quality degradation over long sequences and achieves zero-shot frame interpolation, outperforming specialized baselines on standard benchmarks.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Adaptive Context Switch contributes only +0.03 PSNR in ablation (Table 3), and 3-step AR Unrolling training is extrapolated to 50+ inference chunks, leaving long-horizon stability largely attributable to the pretrained backbone rather than the proposed mechanisms.","rationale":"The reader correctly identified the τ=0.05 threshold as unjustified, but the more load-bearing concern is the marginal quantitative impact of the Adaptive Context Switch mechanism itself (+0.03 PSNR in ablation) combined with the large training-inference horizon mismatch (3 unrolling steps vs. 50+ inference chunks). Even if the threshold were perfectly calibrated, the mechanism's contribution to the central long-horizon stability claim appears quantitatively negligible on the metrics reported.\n\nHowever, this concern does not invalidate the paper's overall contributions. The method does achieve strong results across all three tasks, the zero-shot interpolation result is genuinely impressive, and the unification of three tasks in a single architecture is valuable. The concern is about attribution — whether the long-horizon stability comes from the proposed mechanisms or from CogVideoX's pretrained representations. This is addressable with additional ablations on longer sequences and temporal metrics, which is why the verdict remains CONDITIONAL rather than moving to REJECT. The paper would be strengthened by (1) reporting ablation metrics on the longest sequences rather than only HQF, (2) providing VBench consistency for ablated variants, and (3) justifying why 3 unrolling iterations suffice for 50+ chunk inference.","tokens_in":22543,"tokens_out":5492,"duration_ms":204833,"concrete_test":"Run the full method vs. the ablated variant without Adaptive Context Switch (Table 3, Row 4) specifically on the longest MVSEC sequence (2,740 frames, ~56 chunks), reporting both frame-level metrics (PSNR/LPIPS) and VBench subject consistency. If the gap remains ≤0.1 PSNR and ≤0.02 VBench on this longest sequence, the mechanism does not contribute meaningfully to long-horizon stability and the claim should be attributed primarily to the pretrained backbone. Additionally, increase AR Unrolling iterations from 3 to 10 on HQF; if performance improves beyond the current 16.45 PSNR, the current 3-iteration training is under-specified for the inference horizon.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is long-horizon stability via Autoregressive Unrolling and Adaptive Context Switching. However, Table 3 on HQF (2,430 frames ≈ 50 chunks) reveals that Adaptive Context Switch adds only +0.03 PSNR (16.42→16.45) and +0.007 LPIPS. AR Unrolling adds +0.79 PSNR — meaningful but modest. By contrast, the pretrained prior contributes +3.55 PSNR and context contributes +1.82 PSNR. This means the proposed long-horizon mechanisms together account for less than 1 PSNR point, while the pretrained backbone and context account for over 5.\n\nMore critically, the AR Unrolling is trained for only 3 iterations (3,000 steps each), but inference on MVSEC requires ~56 autoregressive chunks (2,740 frames / 49 frames per chunk). The model adapts to its own prediction errors for 3 steps during training but must maintain stability for 50+ steps at inference — a substantial extrapolation gap. The VBench subject consistency results (Table 6) show larger gaps vs. baselines (0.7204 vs 0.5413 for E2VID+), but these compare against entirely different architectures rather than ablated versions of the proposed method, so they cannot isolate the contribution of the stability mechanisms themselves.\n\nThe reader's concern about the unjustified τ=0.05 threshold is valid but secondary: even with perfect threshold calibration, the quantitative impact of Adaptive Context Switch is within noise range on the metrics reported. The qualitative ablation (Fig. 8) shows grid artifacts without it, but these visual differences are not reflected in the quantitative metrics, raising the question of whether the mechanism addresses a real failure mode or a cosmetic one.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The paper proposes LongE2V, a unified framework that fine-tunes a pre-trained video diffusion model (CogVideoX I2V) to jointly address event-based video reconstruction, prediction, and frame interpolation. The key technical contributions are: (1) Autoregressive Unrolling and Adaptive Context Switching to mitigate temporal drift in long sequences; (2) Reencoding Alignment with Cross Residual Correction to ensure bidirectional consistency during frame interpolation; and (3) Event Voxel Density Augmentation for cross-sensor robustness. The method is evaluated on ECD, MVSEC, HQF, and BS-ERGB benchmarks, outperforming specialized baselines on all three tasks, including zero-shot interpolation where the model was not explicitly trained. The approach is well-motivated, and the unification of three tasks under a single architecture is a practical strength.","tokens_in":23546,"tokens_out":1288,"duration_ms":178157,"significance":"The paper makes a solid contribution to event-based vision by demonstrating that a single fine-tuned video diffusion model can handle reconstruction, prediction, and zero-shot interpolation, outperforming specialized architectures. The quantitative results across four real-world benchmarks (Tables 1, 2) are comprehensive, and the zero-shot interpolation capability is a notable practical strength. The ablation studies (Tables 3, 4) attempt to isolate component contributions. The inclusion of VBench subject consistency (Table 6) and inference speed comparisons (Table 7) adds value. The approach of leveraging pre-trained video diffusion priors for event-based tasks is timely and the results are generally convincing.","major_comments":[{"comment":"Section 3.3, Eq. (2): The Adaptive Context Switching mechanism uses the average attention weight μ_attn between current and context tokens as a proxy for context relevance. The threshold τ=0.05 is stated without justification or sensitivity analysis. Given that Table 3 shows the Adaptive Context Switch contributes only +0.03 PSNR (16.42→16.45) and +0.007 SSIM, it is unclear whether this mechanism provides a statistically meaningful improvement or whether the threshold was selected post hoc. A sensitivity analysis over a range of τ values, or at minimum a justification for τ=0.05, would strengthen the claim that this mechanism is load-bearing for long-horizon stability.","section":null},{"comment":"Table 3 and the associated ablation: The pretrained prior contributes +3.55 PSNR and context contributes +1.82 PSNR, while the two proposed long-horizon mechanisms (AR Unrolling + Adaptive Context Switch) together contribute less than 1 PSNR point. The paper's central framing emphasizes long-horizon stability as the key contribution, but the quantitative ablation suggests the pretrained backbone and context conditioning are the primary drivers of performance. The authors should more carefully contextualize the relative contributions — perhaps through a long-horizon-specific metric (e.g., drift as a function of chunk index) rather than aggregate PSNR, which may not capture the drift artifacts that the qualitative results (Fig. 8) show.","section":null},{"comment":"Section 3.2 (Autoregressive Unrolling) and Appendix D: The model is trained with 3 unrolling iterations (3,000 steps each), but inference on MVSEC requires ~56 autoregressive chunks (2,740 frames / 49 frames per chunk). This is a substantial extrapolation gap between the training distribution (3 steps of self-generated context) and inference (50+ steps). While the VBench subject consistency results (Table 6) show improvements over baselines, these baselines are entirely different architectures, not ablated versions of the proposed method. The paper would benefit from an ablation that varies the number of unrolling iterations at training time and evaluates long-horizon drift at inference, to demonstrate that the unrolling mechanism (rather than the pretrained backbone) is responsible for the observed stability.","section":null}],"minor_comments":[{"comment":"Table 1 caption: 'Red: best; blue: second' — the blue second-best values are not visually distinguishable in the rendered table. Consider using bold/underline conventions.","section":null},{"comment":"Section 4.2, first paragraph: 'the 2th row' and 'the 3th row' should be 'the 2nd row' and 'the 3rd row'.","section":null},{"comment":"Section 3.1: The denoising objective is written with non-standard notation (the expectation brackets). A brief clarification of the notation would improve readability.","section":null},{"comment":"Figure 5: The diagram is dense and the flow of information through the decode-flip-encode loop could be clarified with more explicit labels or a step-by-step caption for the latent variables.","section":null},{"comment":"Appendix E, Table 7: The inference speed comparison is useful but the computational cost of the Adaptive Context Switch retry mechanism is not included. The authors should contextualize this.","section":null},{"comment":"Section 3.3, Reencoding Alignment: The claim that latent-space and pixel-space flipping are non-commutative due to 3D VAE temporal compression is intuitive but could be made more concrete with a brief illustrative example or measurement of the misalignment magnitude.","section":null},{"comment":"References: Several arXiv preprints are cited (e.g., [Blattmann et al. 2023a], [Chen et al. 2025c], [Guo et al. 2025]). Where published versions exist, they should be updated.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The stress-test concern about the modest quantitative impact of the proposed long-horizon mechanisms (Adaptive Context Switch +0.03 PSNR) is valid and worth raising. However, the aggregate PSNR metric may not fully capture drift artifacts that accumulate over long sequences — the qualitative ablation in Fig. 8 does show visible artifacts (grid patterns, point artifacts) when these mechanisms are removed. The paper's claims are defensible but would be substantially strengthened by a drift-specific evaluation (e.g., PSNR/SSIM as a function of chunk index) rather than only aggregate metrics. I rate this minor revision because the core framework is sound, the zero-shot interpolation results are genuinely impressive, and the concerns are about quantifying the contribution of specific components rather than correctness of the overall approach."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"The main thing to know: this paper fine-tunes CogVideoX on events and gets a single model that handles reconstruction, prediction, and frame interpolation. The zero-shot interpolation result is the strongest finding — they train only for reconstruction and prediction, then apply the same weights to interpolation and beat supervised baselines (CBMNet-Large, TLXNet+) on LPIPS. That's a genuine and practical result worth reading for anyone in event vision. The Reencoding Alignment fix for 3D VAE temporal flipping is a clean, well-motivated technical contribution — the observation that latent-space and pixel-space flipping don't commute under temporal compression is correct and the decode-flip-encode workaround is sensible. Cross Residual Correction to compensate for the re-encoding loss is a nice touch. The ablation in Table 4 isolates these components convincingly. The text-guided colorization bonus (Sec. 4.5) is a neat demonstration of decoupling structure from appearance. Now the soft spots. The stress-test concern about Adaptive Context Switch is largely correct. Table 3 shows it adds +0.03 PSNR and +0.007 LPIPS on HQF — within noise on those metrics. AR Unrolling adds +0.79 PSNR, which is real but modest. The pretrained prior and context framing do the heavy lifting (+5.35 PSNR combined). However, I'd push back on the stress-test's framing that this undermines the paper. The qualitative ablation (Fig. 8) shows visible grid artifacts without Adaptive Context Switch, suggesting it addresses a real failure mode that PSNR doesn't capture well. The VBench subject consistency gap (0.72 vs 0.54 for E2VID+) is substantial, though the stress-test is right that this compares across architectures rather than isolating the stability mechanisms. The extrapolation gap is fair: 3 unrolling iterations training vs. 50+ chunks at inference is a real concern, but the long-sequence results on MVSEC (2,740 frames) and HQF (2,430 frames) hold up visually in the figures, which is the relevant test. The τ=0.05 threshold lacking sensitivity analysis is a minor gap — worth noting but not load-bearing. Reconstruction PSNR gains over baselines are marginal; superiority rests on LPIPS, which is the right metric for generative quality but reviewers may want both. No code released. Overall: the paper delivers a useful unified framework with a genuinely surprising zero-shot interpolation result. The long-horizon mechanisms are incremental but defensible. It deserves a serious referee.","headline":"Solid practical paper. The zero-shot interpolation result is the real finding; the long-horizon stability mechanisms are modest in quantitative impact but visually meaningful.","tokens_in":23456,"tokens_out":607,"would_cite":true,"duration_ms":94039,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Diffusion model beats specialized methods at three event-camera tasks","keywords":["event camera","video diffusion model","video reconstruction","video prediction","frame interpolation","autoregressive generation","temporal drift","zero-shot transfer"],"falsifier":"Replace the attention-weight-based Adaptive Context Switching with a random or fixed-schedule context refresh of equal frequency; if long-horizon stability is comparable, the attention proxy is not doing the work the paper attributes to it.","tokens_in":22644,"feed_emoji":"📹","tokens_out":8171,"duration_ms":291967,"temperature":0.7,"pith_summary":"The paper claims that a single fine-tuned video diffusion model can handle three event-camera video tasks — reconstruction, prediction, and frame interpolation — that previously required separate specialized architectures. The authors take a pre-trained video diffusion transformer and condition it on event voxels, which are sparse brightness-change signals from neuromorphic sensors. The central technical contribution is a set of mechanisms that prevent temporal drift during long-sequence generation: Autoregressive Unrolling, which iteratively substitutes the model's own predictions for ground-truth context during fine-tuning to close the train-inference gap; and Adaptive Context Switching, which monitors attention weights between current and context tokens to decide when to refresh historical context during inference. For frame interpolation, the authors identify that temporal flipping in the 3D VAE's latent space is not commutative with flipping in pixel space, and propose Reencoding Alignment to fix this, along with Cross Residual Correction to recover information lost in the decode-encode loop. Trained on only 7,636 frames, the model outperforms specialized methods on real-world benchmarks across all three tasks, including zero-shot interpolation where no task-specific training was performed.","feed_headline":"Diffusion model beats specialized methods at three event-camera tasks","feed_subtitle":"Trained on 7,600 frames with self-correcting context, it handles reconstruction, prediction, zero-shot interpolation.","key_machinery":"Autoregressive Unrolling, Adaptive Context Switching, Reencoding Alignment, Cross Residual Correction","core_discovery":"The key finding is that the primary obstacle to long-horizon event-based video generation is not the quality of individual frames but the accumulation of errors across autoregressive chunks, and that this accumulation can be controlled through two complementary mechanisms. First, the train-inference gap — where models condition on ground-truth context during training but on their own predictions during inference — can be closed by iteratively fine-tuning on the model's own outputs (Autoregressive Unrolling). Second, the decision of when to refresh context during generation can be made dynamically by measuring how much attention current tokens pay to context tokens, using a fixed threshold as","pith_inferences":["The success of attention-weight-based drift detection suggests that attention patterns in diffusion transformers may encode implicit quality signals beyond what is visible in the output, which could be exploited for inference-time quality control in other generation tasks.","If Autoregressive Unrolling is the primary mechanism closing the train-inference gap, then scheduled sampling strategies from the RNN literature may be broadly applicable to diffusion-based autoregressive generation, not just event-camera tasks.","The fact that a single architecture handles reconstruction, prediction, and interpolation by varying only input conditions suggests these may all be instances of conditional video generation with different boundary conditions rather than fundamentally distinct problems.","The sensitivity to event sparsity and hot-pixel noise (noted in the limitations) implies that event-based generation pipelines may benefit from dedicated event denoising or density normalization as a separate preprocessing stage."],"forward_implications":["If attention-based context switching reliably detects drift, the same mechanism could stabilize other autoregressive generation pipelines (text-to-video, audio, 3D) where long-horizon error accumulation is a problem.","The zero-shot interpolation result — where a model trained only for reconstruction and prediction transfers to interpolation without fine-tuning — suggests these tasks share a common latent structure that diffusion priors can exploit without explicit task boundaries.","The finding that latent-space flipping diverges from pixel-space flipping under 3D VAE compression affects any bidirectional video generation method that operates in latent space.","Training on fewer than 8,000 frames and generalizing across multiple real-world benchmarks suggests that pre-trained video diffusion priors are highly sample-efficient for sensor-specific conditioning tasks."],"fun_headline_variants":["Self-correcting diffusion turns event camera streams into stable long video","Event-based video diffusion stops error drift with adaptive context refresh","Closing the train-inference gap in long-horizon event video generation","One video diffusion model handles event reconstruction, prediction, and interpolation","Measuring token attention to decide when to refresh context in event video diffusion"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method's long-horizon stability rests on the assumption that the average attention weight between current and context tokens reliably indicates whether the model is drifting. The threshold for triggering a context refresh is set to 0.05 without sensitivity analysis, so if attention patterns do not actually correlate with drift risk, the mechanism either fires too often or too rarely.","fun_headline_variants_meta":{"raw":{"variants":["Self-correcting diffusion turns event camera streams into stable long video","Event-based video diffusion stops error drift with adaptive context refresh","Closing the train-inference gap in long-horizon event video generation","One video diffusion model handles event reconstruction, prediction, and interpolation","Measuring token attention to decide when to refresh context in event video diffusion"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":622,"prompt_tokens":533,"completion_tokens":89,"prompt_tokens_details":null},"tokens_in":533,"tokens_out":89,"duration_ms":47553,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T01:27:55.805878+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Replace the attention-weight-based Adaptive Context Switching with a random or fixed-schedule context refresh of equal frequency; if long-horizon stability is comparable, the attention proxy is not doing the work the paper attributes to it.","supporting_citations":[],"review_version":1}