Pith. sign in

REVIEW 3 major objections 7 minor 104 references

LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models

T0 review · 3 major / 7 minor · reviewed 2026-07-10 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Diffusion model beats specialized methods at three event-camera tasks

desk verdict Solid practical paper. The zero-shot interpolation result is the real finding; the long-horizon stability mechanisms are modest in quantitative impact but visually meaningful. read the letter →

arxiv 2607.08770 v1 pith:IAWVFIWH submitted 2026-07-09 cs.CV

Cheng-De Fan , Chun-Wei Tuan Mu , Chen-Wei Chang , Chin-Yang Lin , Kun-Ru Wu , Yu-Chee Tseng , Yu-Lun Liu This is my paper · ORCID
classification cs.CV
keywords eventcameravideodiffusionmodelreconstructionpredictionframeinterpolationautoregressivegenerationtemporaldriftzero-shottransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a single fine-tuned video diffusion model can handle three event-camera video tasks — reconstruction, prediction, and frame interpolation — that previously required separate specialized architectures. The authors take a pre-trained video diffusion transformer and condition it on event voxels, which are sparse brightness-change signals from neuromorphic sensors. The central technical contribution is a set of mechanisms that prevent temporal drift during long-sequence generation: Autoregressive Unrolling, which iteratively substitutes the model's own predictions for ground-truth context during fine-tuning to close the train-inference gap; and Adaptive Context Switching, which monitors attention weights between current and context tokens to decide when to refresh historical context during inference. For frame interpolation, the authors identify that temporal flipping in the 3D VAE's latent space is not commutative with flipping in pixel space, and propose Reencoding Alignment to fix this, along with Cross Residual Correction to recover information lost in the decode-encode loop. Trained on only 7,636 frames, the model outperforms specialized methods on real-world benchmarks across all three tasks, including zero-shot interpolation where no task-specific training was performed.

What carries the argument

Autoregressive Unrolling, Adaptive Context Switching, Reencoding Alignment, Cross Residual Correction

What would settle it

Replace the attention-weight-based Adaptive Context Switching with a random or fixed-schedule context refresh of equal frequency; if long-horizon stability is comparable, the attention proxy is not doing the work the paper attributes to it.

Watch

Extended reading notes

Core claim

The key finding is that the primary obstacle to long-horizon event-based video generation is not the quality of individual frames but the accumulation of errors across autoregressive chunks, and that this accumulation can be controlled through two complementary mechanisms. First, the train-inference gap — where models condition on ground-truth context during training but on their own predictions during inference — can be closed by iteratively fine-tuning on the model's own outputs (Autoregressive Unrolling). Second, the decision of when to refresh context during generation can be made dynamically by measuring how much attention current tokens pay to context tokens, using a fixed threshold as

Load-bearing premise

The method's long-horizon stability rests on the assumption that the average attention weight between current and context tokens reliably indicates whether the model is drifting. The threshold for triggering a context refresh is set to 0.05 without sensitivity analysis, so if attention patterns do not actually correlate with drift risk, the mechanism either fires too often or too rarely.

Editorial extensions

If this is right

  • If attention-based context switching reliably detects drift, the same mechanism could stabilize other autoregressive generation pipelines (text-to-video, audio, 3D) where long-horizon error accumulation is a problem.
  • The zero-shot interpolation result — where a model trained only for reconstruction and prediction transfers to interpolation without fine-tuning — suggests these tasks share a common latent structure that diffusion priors can exploit without explicit task boundaries.
  • The finding that latent-space flipping diverges from pixel-space flipping under 3D VAE compression affects any bidirectional video generation method that operates in latent space.
  • Training on fewer than 8,000 frames and generalizing across multiple real-world benchmarks suggests that pre-trained video diffusion priors are highly sample-efficient for sensor-specific conditioning tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The success of attention-weight-based drift detection suggests that attention patterns in diffusion transformers may encode implicit quality signals beyond what is visible in the output, which could be exploited for inference-time quality control in other generation tasks.
  • If Autoregressive Unrolling is the primary mechanism closing the train-inference gap, then scheduled sampling strategies from the RNN literature may be broadly applicable to diffusion-based autoregressive generation, not just event-camera tasks.
  • The fact that a single architecture handles reconstruction, prediction, and interpolation by varying only input conditions suggests these may all be instances of conditional video generation with different boundary conditions rather than fundamentally distinct problems.
  • The sensitivity to event sparsity and hot-pixel noise (noted in the limitations) implies that event-based generation pipelines may benefit from dedicated event denoising or density normalization as a separate preprocessing stage.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes LongE2V, a unified framework that fine-tunes a pre-trained video diffusion model (CogVideoX I2V) to jointly address event-based video reconstruction, prediction, and frame interpolation. The key technical contributions are: (1) Autoregressive Unrolling and Adaptive Context Switching to mitigate temporal drift in long sequences; (2) Reencoding Alignment with Cross Residual Correction to ensure bidirectional consistency during frame interpolation; and (3) Event Voxel Density Augmentation for cross-sensor robustness. The method is evaluated on ECD, MVSEC, HQF, and BS-ERGB benchmarks, outperforming specialized baselines on all three tasks, including zero-shot interpolation where the model was not explicitly trained. The approach is well-motivated, and the unification of three tasks under a single architecture is a practical strength.

Significance. The paper makes a solid contribution to event-based vision by demonstrating that a single fine-tuned video diffusion model can handle reconstruction, prediction, and zero-shot interpolation, outperforming specialized architectures. The quantitative results across four real-world benchmarks (Tables 1, 2) are comprehensive, and the zero-shot interpolation capability is a notable practical strength. The ablation studies (Tables 3, 4) attempt to isolate component contributions. The inclusion of VBench subject consistency (Table 6) and inference speed comparisons (Table 7) adds value. The approach of leveraging pre-trained video diffusion priors for event-based tasks is timely and the results are generally convincing.

major comments (3)
  1. Section 3.3, Eq. (2): The Adaptive Context Switching mechanism uses the average attention weight μ_attn between current and context tokens as a proxy for context relevance. The threshold τ=0.05 is stated without justification or sensitivity analysis. Given that Table 3 shows the Adaptive Context Switch contributes only +0.03 PSNR (16.42→16.45) and +0.007 SSIM, it is unclear whether this mechanism provides a statistically meaningful improvement or whether the threshold was selected post hoc. A sensitivity analysis over a range of τ values, or at minimum a justification for τ=0.05, would strengthen the claim that this mechanism is load-bearing for long-horizon stability.
  2. Table 3 and the associated ablation: The pretrained prior contributes +3.55 PSNR and context contributes +1.82 PSNR, while the two proposed long-horizon mechanisms (AR Unrolling + Adaptive Context Switch) together contribute less than 1 PSNR point. The paper's central framing emphasizes long-horizon stability as the key contribution, but the quantitative ablation suggests the pretrained backbone and context conditioning are the primary drivers of performance. The authors should more carefully contextualize the relative contributions — perhaps through a long-horizon-specific metric (e.g., drift as a function of chunk index) rather than aggregate PSNR, which may not capture the drift artifacts that the qualitative results (Fig. 8) show.
  3. Section 3.2 (Autoregressive Unrolling) and Appendix D: The model is trained with 3 unrolling iterations (3,000 steps each), but inference on MVSEC requires ~56 autoregressive chunks (2,740 frames / 49 frames per chunk). This is a substantial extrapolation gap between the training distribution (3 steps of self-generated context) and inference (50+ steps). While the VBench subject consistency results (Table 6) show improvements over baselines, these baselines are entirely different architectures, not ablated versions of the proposed method. The paper would benefit from an ablation that varies the number of unrolling iterations at training time and evaluates long-horizon drift at inference, to demonstrate that the unrolling mechanism (rather than the pretrained backbone) is responsible for the observed stability.
minor comments (7)
  1. Table 1 caption: 'Red: best; blue: second' — the blue second-best values are not visually distinguishable in the rendered table. Consider using bold/underline conventions.
  2. Section 4.2, first paragraph: 'the 2th row' and 'the 3th row' should be 'the 2nd row' and 'the 3rd row'.
  3. Section 3.1: The denoising objective is written with non-standard notation (the expectation brackets). A brief clarification of the notation would improve readability.
  4. Figure 5: The diagram is dense and the flow of information through the decode-flip-encode loop could be clarified with more explicit labels or a step-by-step caption for the latent variables.
  5. Appendix E, Table 7: The inference speed comparison is useful but the computational cost of the Adaptive Context Switch retry mechanism is not included. The authors should contextualize this.
  6. Section 3.3, Reencoding Alignment: The claim that latent-space and pixel-space flipping are non-commutative due to 3D VAE temporal compression is intuitive but could be made more concrete with a brief illustrative example or measurement of the misalignment magnitude.
  7. References: Several arXiv preprints are cited (e.g., [Blattmann et al. 2023a], [Chen et al. 2025c], [Guo et al. 2025]). Where published versions exist, they should be updated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper's contributions are training strategies and architectural modifications evaluated against external benchmarks, not derivations that reduce to their own inputs.

full rationale

The paper's derivation chain is straightforward and non-circular. (1) Autoregressive Unrolling (Sec. 3.2) is a scheduled-sampling-style training strategy where the model fine-tunes on its own predictions — this is a standard technique to bridge the train-test gap, not a circular derivation. (2) Adaptive Context Switching (Sec. 3.3) uses attention weights as a heuristic proxy for context relevance with an empirically set threshold τ=0.05; while the threshold is unjustified, this is a correctness/robustness concern, not circularity. (3) Reencoding Alignment (Eq. 3-4) is justified by the mathematical observation that latent-space and pixel-space flipping are non-commutative under 3D VAE compression — an independent argument, not a self-referential one. (4) Cross Residual Correction (Eq. 5-6) injects computed residuals across branches; the residual is defined as the difference between original and re-encoded latents, which is a genuine information-restoration mechanism, not a tautology. (5) The inspiration from LookingGlass [Chang et al. 2025] is a citation to external work by different authors, not a self-citation. All claims are validated against external benchmarks (ECD, MVSEC, HQF, BS-ERGB) with standard metrics (PSNR, SSIM, LPIPS, VBench). The skeptic's concerns about modest ablation gains and the train-to-inference extrapolation gap are legitimate correctness risks but do not constitute circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 3 invented entities

The paper introduces several task-specific mechanisms with associated free parameters. The most structurally important axiom is that attention weight serves as a drift proxy, which is ad hoc and unverified beyond the ablation showing overall benefit.

free parameters (7)
  • τ (Adaptive Context Switch threshold) = 0.05
    Set without sensitivity analysis; determines when context is refreshed during long generation.
  • B (event voxel temporal bins) = 3
    Chosen to match VAE 3-channel input; not justified against alternatives.
  • LoRA rank r = 64
    Standard choice but still a free parameter affecting capacity.
  • Context length = 20 frames
    Fixed for training and inference; not ablated.
  • Unrolling iterations T = 3
    Number of autoregressive unrolling cycles; not justified against alternatives.
  • Z_x0 dropout rate = 0.05
    Applied to enhance reconstruction robustness; chosen ad hoc.
  • Text prompt probability = 0.20
    Probability of using text prompts during training for colorization capability.
assumptions (4)
  • domain assumption Pre-trained video diffusion models encode useful visual priors transferable to event-based generation.
    Sec. 1, Sec. 3.2: The entire approach depends on CogVideoX priors being applicable to event-conditioned generation.
  • ad hoc to paper Average attention weight between current and context tokens correlates with context relevance and temporal stability.
    Sec. 3.3, Eq. 2: The Adaptive Context Switch mechanism assumes μ_attn is a valid proxy for drift risk, which is unverified.
  • domain assumption Event voxel grids with B=3 bins provide sufficient temporal resolution for video generation.
    Sec. 3.1: B=3 is chosen for VAE compatibility but temporal adequacy is assumed.
  • domain assumption Global brightness alignment is a fair evaluation protocol for event-based reconstruction.
    Sec. 4.2: All methods are brightness-aligned with ground truth, which assumes intensity recovery is not part of the task.
invented entities (3)
  • Adaptive Context Switch independent evidence
    purpose: Dynamic context update mechanism to prevent temporal drift in long video generation.
    Ablation in Tab. 3 (Row 4 vs 5) shows its contribution to PSNR/SSIM/LPIPS on HQF.
  • Reencoding Alignment independent evidence
    purpose: Resolving temporal misalignment between latent-space and pixel-space flipping for bidirectional interpolation.
    Ablation in Tab. 4 and Fig. 9 demonstrate its necessity for structural fidelity.
  • Cross Residual Correction independent evidence
    purpose: Compensating for information loss during the decode-flip-encode loop via cross-injection of residuals.
    Ablation in Tab. 4 shows LPIPS improvement from 0.161 to 0.124 when added.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models." pith.science (2026). https://pith.science/paper/IAWVFIWH

@misc{pith2026260708770,
  author       = {Pith},
  title        = {Pith review of: LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IAWVFIWH}},
  note         = {Machine review of arXiv:2607.08770}
}
read the original abstract

Recovering high-quality video from sparse event streams is a challenging task. Regression methods often blur textures, while existing generative models struggle with long-term stability. We propose LongE2V, a novel approach that leverages pre-trained video diffusion priors to jointly handle event-based video reconstruction, prediction, and frame interpolation. By fine-tuning a foundational video model, our approach achieves high data efficiency and superior perceptual quality. We introduce Autoregressive Unrolling and Adaptive Context Switching to mitigate temporal drift in extremely long sequences. We also propose Reencoding Alignment with Cross Residual Correction to ensure precise bidirectional consistency during frame interpolation. Furthermore, Event Voxel Density Augmentation ensures robustness across varying sensor resolutions. Extensive experiments on real-world benchmarks demonstrate that LongE2V outperforms state-of-the-art methods across all three tasks, exhibiting exceptional temporal coherence and zero-shot generalization. Project page: https://cdfan0627.github.io/LongE2V-page/

Figures

Figures reproduced from arXiv: 2607.08770 by the authors.

Figure 1
Figure 1. Event-based video generation. We leverage pre-trained video diffusion priors to address three distinct inverse problems within a single architecture. Depending on the input condition, our model performs: (a) Video Reconstruction, recovering high-fidelity textures from sparse event streams, (b) Video Prediction, generating long-term sequences from a single start frame with minimal drift via our autoregressive unrolli… view at source ↗
Figure 2
Figure 2. Challenges in event-based video generation. We highlight failure cases in state-of-the-art methods: (a) Reconstruction: Regression-based methods (e.g., E2VID+[Stoffregen et al. 2020]) suffer from “regression-to￾the-mean,” causing blurry textures and detail loss. (b) Prediction: Direct video diffusion (e.g., VDM-EVFI[Chen et al. 2025a]) on long sequences suffers from error accumulation, leading to severe color tempor… view at source ↗
Figure 3
Figure 3. Autoregressive Unrolling. To bridge the domain gap between training and inference, we employ an iterative training strategy. Initially, the model is trained with Ground Truth (GT) context frames for convergence (left). Subsequently, we activate the unrolling mechanism by performing an inference pass to generate predictions, which then replace the GT context frames for fine-tuning (right). This iterative feedback loo… view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Reencoding Alignment and Cross Residual Correction. To address temporal misalignment caused by the discrepancy between latent-space and pixel-space flipping, we propose Reencoding Alignment. The denoised latents, 𝑍ˆ 𝑓 𝑤𝑑 0 and 𝑍ˆ𝑏𝑤𝑑 0 , are decoded into pixel space, fl…
Figure 6
Figure 6. Figure 6: Qualitative comparisons on ECD [Mueggler et al. 2017], MVSEC [Zhu et al. 2018], and HQF [Stoffregen et al. 2020] datasets. Our LongE2V recovers high-frequency textures where regression baselines (E2VID+, HyperE2VID) suffer from blurring (Row 1). In prediction tasks, we…
Figure 7
Figure 7. Figure 7: Zero-shot interpolation on BS-ERGB and HQF. Baselines (TLXNet+, CBMNet-Large) suffer from structural collapse or blur under large motion (Top), whereas our LongE2V captures accurate dynamics. On fine text (Bottom), our Reencoding Alignment eliminates the ghosting seen …
Figure 9
Figure 9. Figure 9: Visual ablation on interpolation. w/o Reencoding Alignment causes ghosting due to latents misalignment; w/o Cross Residual Correction blurs fine details due to VAE loss; and w/o Event Voxel Density Augmentation yields artifacts from density mismatch. Our Full Method re…
Figure 11
Figure 11. Figure 11: Additional qualitative comparisons on ECD [Mueggler et al. 2017], MVSEC [Zhu et al. 2018], and HQF datasets [Stoffregen et al. 2020]. SIGGRAPH Conference Papers ’26, July 19–23, 2026, Los Angeles, CA, USA [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Additional zero-shot interpolation results on BS-ERGB [Tulyakov et al. 2021] and HQF [Stoffregen et al. 2020] datasets. SIGGRAPH Conference Papers ’26, July 19–23, 2026, Los Angeles, CA, USA [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Limitations. (a) Failure cases under sparse or low-quality event streams. (b) Sensitivity to noise where "hot pixel" is preserved or amplified in the reconstructed frames [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

104 extracted references · 104 canonical work pages

  1. [1]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Repurposing pre-trained video diffusion models for event-based video interpolation , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    work page

  2. [2]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Event-based video frame interpolation with cross-modal asymmetric bidirectional motion fields , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    work page

  3. [3]

    European Conference on Computer Vision , pages=

    Reducing the sim-to-real gap for event cameras , author=. European Conference on Computer Vision , pages=. 2020 , organization=

    work page 2020

  4. [4]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Simultaneous optical flow and intensity estimation from an event camera , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    work page

  5. [5]

    International Journal of Computer Vision , volume=

    Real-time intensity-image reconstruction for event cameras using manifold regularisation , author=. International Journal of Computer Vision , volume=. 2018 , publisher=

    work page 2018

  6. [6]

    Asian Conference on Computer Vision , pages=

    Continuous-time intensity estimation using event cameras , author=. Asian Conference on Computer Vision , pages=. 2018 , organization=

    work page 2018

  7. [7]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    High speed and high dynamic range video with an event camera , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2019 , publisher=

    work page 2019

  8. [8]

    Conference on robot learning , pages=

    Esim: an open event camera simulator , author=. Conference on robot learning , pages=. 2018 , organization=

    work page 2018

Show all 104 references
  1. [9]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Evreal: Towards a comprehensive benchmark and analysis suite for event-based video reconstruction , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  2. [10]

    The International journal of robotics research , volume=

    The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM , author=. The International journal of robotics research , volume=. 2017 , publisher=

    2017

  3. [11]

    IEEE Robotics and Automation Letters , volume=

    The multivehicle stereo event camera dataset: An event camera dataset for 3D perception , author=. IEEE Robotics and Automation Letters , volume=. 2018 , publisher=

    2018

  4. [12]

    Proceedings of the European Conference on Computer Vision (ECCV) Workshops , pages=

    Unsupervised event-based optical flow using motion compensation , author=. Proceedings of the European Conference on Computer Vision (ECCV) Workshops , pages=

  5. [13]

    arXiv preprint arXiv:2311.15127 , year=

    Stable video diffusion: Scaling latent video diffusion models to large datasets , author=. arXiv preprint arXiv:2311.15127 , year=

    arXiv

  6. [14]

    2025 , url=

    Yuhan Liu and LingHui Fu and Zhen Yang and Hao Chen and Youfu Li and Yongjian Deng , booktitle=. 2025 , url=

    2025

  7. [15]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    LookingGlass: Generative Anamorphoses via Laplacian Pyramid Warping , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  8. [16]

    arXiv preprint arXiv:2503.20314 , year=

    Wan: Open and advanced large-scale video generative models , author=. arXiv preprint arXiv:2503.20314 , year=

    arXiv

  9. [17]

    European Conference on Computer Vision , pages=

    Explorative inbetweening of time and space , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  10. [18]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Time lens: Event-based video frame interpolation , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  11. [19]

    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

    Fast image reconstruction with an event camera , author=. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

  12. [20]

    IEEE Transactions on Image Processing , volume=

    Spade-e2vid: Spatially-adaptive denormalization for event-based video reconstruction , author=. IEEE Transactions on Image Processing , volume=. 2021 , publisher=

    2021

  13. [21]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Event-based video reconstruction using transformer , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  14. [22]

    IEEE Transactions on Image Processing , volume=

    Hypere2vid: Improving event-based video reconstruction via hypernetworks , author=. IEEE Transactions on Image Processing , volume=. 2024 , publisher=

    2024

  15. [23]

    Advances in Neural Information Processing Systems , volume=

    Lase-e2v: Towards language-guided semantic-aware event-to-video reconstruction , author=. Advances in Neural Information Processing Systems , volume=

  16. [24]

    arXiv preprint arXiv:2407.08231 , year=

    E2VIDiff: Perceptual Events-to-Video Reconstruction using Diffusion Priors , author=. arXiv preprint arXiv:2407.08231 , year=

    arXiv

  17. [25]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Time lens++: Event-based frame interpolation with parametric non-linear flow and multi-scale fusion , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  18. [26]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    A unified framework for event-based frame interpolation with ad-hoc deblurring in the wild , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  19. [27]

    European Conference on Computer Vision , pages=

    Timelens-xl: Real-time event-based video frame interpolation with large motion , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  20. [28]

    Advances in neural information processing systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=

  21. [29]

    arXiv preprint arXiv:2011.13456 , year=

    Score-based generative modeling through stochastic differential equations , author=. arXiv preprint arXiv:2011.13456 , year=

    2011 arXiv

  22. [30]

    Advances in neural information processing systems , volume=

    Video diffusion models , author=. Advances in neural information processing systems , volume=

  23. [31]

    arXiv preprint arXiv:2210.02303 , year=

    Imagen video: High definition video generation with diffusion models , author=. arXiv preprint arXiv:2210.02303 , year=

    arXiv

  24. [32]

    arXiv preprint arXiv:2209.14792 , year=

    Make-a-video: Text-to-video generation without text-video data , author=. arXiv preprint arXiv:2209.14792 , year=

    arXiv

  25. [33]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Align your latents: High-resolution video synthesis with latent diffusion models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  26. [34]

    arXiv preprint arXiv:2408.06072 , year=

    Cogvideox: Text-to-video diffusion models with an expert transformer , author=. arXiv preprint arXiv:2408.06072 , year=

    arXiv

  27. [35]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Adding conditional control to text-to-image diffusion models , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  28. [36]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  29. [37]

    arXiv e-prints , pages=

    Control-a-video: Controllable text-to-video generation with diffusion models , author=. arXiv e-prints , pages=

  30. [38]

    Advances in Neural Information Processing Systems , volume=

    Videocomposer: Compositional video synthesis with motion controllability , author=. Advances in Neural Information Processing Systems , volume=

  31. [39]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Structure and content-guided video synthesis with diffusion models , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  32. [40]

    arXiv preprint arXiv:2408.13005 , year=

    Easycontrol: Transfer controlnet to video diffusion for controllable generation and interpolation , author=. arXiv preprint arXiv:2408.13005 , year=

    arXiv

  33. [41]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Streamingt2v: Consistent, dynamic, and extendable long video generation from text , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  34. [42]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Nuwa-xl: Diffusion over diffusion for extremely long video generation , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  35. [43]

    arXiv preprint arXiv:2310.15169 , year=

    Freenoise: Tuning-free longer video diffusion via noise rescheduling , author=. arXiv preprint arXiv:2310.15169 , year=

    arXiv

  36. [44]

    arXiv preprint arXiv:2506.08009 , year=

    Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion , author=. arXiv preprint arXiv:2506.08009 , year=

    arXiv

  37. [45]

    arXiv preprint arXiv:2512.15702 , year=

    End-to-End Training for Autoregressive Video Diffusion via Self-Resampling , author=. arXiv preprint arXiv:2512.15702 , year=

    arXiv

  38. [46]

    arXiv preprint arXiv:2504.12626 , year=

    Packing input frame context in next-frame prediction models for video generation , author=. arXiv preprint arXiv:2504.12626 , year=

  39. [47]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Events-to-video: Bringing modern computer vision to event cameras , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  40. [48]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Event-based frame interpolation with ad-hoc deblurring , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  41. [49]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    Event-based vision: A survey , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2020 , publisher=

    2020

  42. [50]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Back to event basics: Self-supervised learning of image reconstruction for event cameras via photometric constancy , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  43. [51]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Formulating event-based image reconstruction as a linear inverse problem with deep regularization using optical flow , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2022 , publisher=

    2022

  44. [52]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Sparse-e2vid: A sparse convolutional model for event-based video reconstruction trained with real event noise , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  45. [53]

    European Conference on Computer Vision , pages=

    Revisit event generation model: Self-supervised learning of event-to-video reconstruction with implicit neural representations , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  46. [54]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Event-based video reconstruction via potential-assisted spiking neural network , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  47. [55]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Retina-like visual image reconstruction via spiking neural model , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  48. [56]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Event-based high dynamic range image and very high frame rate video generation using conditional generative adversarial networks , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  49. [57]

    European Conference on Computer Vision , pages=

    Temporal Residual Guided Diffusion Framework for Event-Driven Video Reconstruction , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  50. [58]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Efi-net: Video frame interpolation from fusion of events and frames , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  51. [59]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    TTA-EVF: test-time adaptation for event-based video frame interpolation via reliable pixel and sample estimation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  52. [60]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Timereplayer: Unlocking the potential of event cameras for video interpolation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  53. [61]

    European Conference on Computer Vision , pages=

    Video interpolation by event-driven anisotropic adjustment of optical flow , author=. European Conference on Computer Vision , pages=. 2022 , organization=

    2022

  54. [62]

    arXiv preprint arXiv:2305.10198 , year=

    IDO-VFI: Identifying Dynamics via Optical Flow Guidance for Video Frame Interpolation with Events , author=. arXiv preprint arXiv:2305.10198 , year=

    arXiv

  55. [63]

    arXiv preprint arXiv:2404.18156 , year=

    Event-based video frame interpolation with edge guided motion refinement , author=. arXiv preprint arXiv:2404.18156 , year=

    arXiv

  56. [64]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Unifying motion deblurring and frame interpolation with events , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  57. [65]

    arXiv preprint arXiv:2509.08260 , year=

    EVDI++: Event-based Video Deblurring and Interpolation via Self-Supervised Learning , author=. arXiv preprint arXiv:2509.08260 , year=

    arXiv

  58. [66]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Video frame interpolation via direct synthesis with the event-based reference , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  59. [67]

    SIGGRAPH Asia 2024 Conference Papers , pages=

    Lumiere: A space-time diffusion model for video generation , author=. SIGGRAPH Asia 2024 Conference Papers , pages=

    2024

  60. [68]

    arXiv preprint arXiv:2401.03048 , year=

    Latte: Latent diffusion transformer for video generation , author=. arXiv preprint arXiv:2401.03048 , year=

    arXiv

  61. [69]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Gentron: Diffusion transformers for image and video generation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  62. [70]

    OpenAI Blog , volume=

    Video generation models as world simulators , author=. OpenAI Blog , volume=

  63. [71]

    arXiv preprint arXiv:2412.20404 , year=

    Open-sora: Democratizing efficient video production for all , author=. arXiv preprint arXiv:2412.20404 , year=

    arXiv

  64. [72]

    arXiv preprint arXiv:2412.03603 , year=

    Hunyuanvideo: A systematic framework for large video generative models , author=. arXiv preprint arXiv:2412.03603 , year=

    arXiv

  65. [73]

    GitHub repository , howpublished=

    Mochi 1 , author=. GitHub repository , howpublished=. 2024 , publisher =

    2024

  66. [74]

    Forty-first international conference on machine learning , year=

    Scaling rectified flow transformers for high-resolution image synthesis , author=. Forty-first international conference on machine learning , year=

  67. [75]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Goku: Flow based video generative foundation models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  68. [76]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    From slow bidirectional to fast autoregressive video diffusion models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  69. [77]

    arXiv preprint arXiv:2307.04725 , year=

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning , author=. arXiv preprint arXiv:2307.04725 , year=

    arXiv

  70. [78]

    ACM SIGGRAPH 2024 Conference Papers , pages=

    Motionctrl: A unified and flexible motion controller for video generation , author=. ACM SIGGRAPH 2024 Conference Papers , pages=

    2024

  71. [79]

    European Conference on Computer Vision , pages=

    Draganything: Motion control for anything using entity representation , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  72. [80]

    SIGGRAPH Asia 2024 Conference Papers , pages=

    Trailblazer: Trajectory control for diffusion-based video generation , author=. SIGGRAPH Asia 2024 Conference Papers , pages=

    2024

  73. [81]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Tora: Trajectory-oriented diffusion transformer for video generation , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  74. [82]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Motion prompting: Controlling video generation with motion trajectories , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  75. [83]

    arXiv preprint arXiv:2411.04989 , year=

    Sg-i2v: Self-guided trajectory control in image-to-video generation , author=. arXiv preprint arXiv:2411.04989 , year=

    arXiv

  76. [84]

    arXiv preprint arXiv:2404.02101 , year=

    Cameractrl: Enabling camera control for text-to-video generation , author=. arXiv preprint arXiv:2404.02101 , year=

    arXiv

  77. [85]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Flovd: Optical flow meets video diffusion model for enhanced camera-controlled video synthesis , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  78. [86]

    arXiv preprint arXiv:2402.01566 , year=

    Boximator: Generating rich and controllable motions for video synthesis , author=. arXiv preprint arXiv:2402.01566 , year=

    arXiv

  79. [87]

    European Conference on Computer Vision , pages=

    Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  80. [88]

    Advances in Neural Information Processing Systems , volume=

    Video diffusion models are training-free motion interpreter and controller , author=. Advances in Neural Information Processing Systems , volume=

  81. [89]

    Advances in neural information processing systems , volume=

    Scheduled sampling for sequence prediction with recurrent neural networks , author=. Advances in neural information processing systems , volume=

  82. [90]

    Advances in neural information processing systems , volume=

    Professor forcing: A new algorithm for training recurrent networks , author=. Advances in neural information processing systems , volume=

  83. [91]

    arXiv preprint arXiv:2407.16655 , year=

    Moviedreamer: Hierarchical generation for coherent long visual sequence , author=. arXiv preprint arXiv:2407.16655 , year=

  84. [92]

    Advances in Neural Information Processing Systems , volume=

    Fifo-diffusion: Generating infinite videos from text without training , author=. Advances in Neural Information Processing Systems , volume=

  85. [93]

    Advances in Neural Information Processing Systems , volume=

    Freelong: Training-free long video generation with spectralblend temporal attention , author=. Advances in Neural Information Processing Systems , volume=

  86. [94]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Ouroboros-diffusion: Exploring consistent content generation in tuning-free long video diffusion , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  87. [95]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Upscale-a-video: Temporal-consistent diffusion model for real-world video super-resolution , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  88. [96]

    The Twelfth International Conference on Learning Representations , year=

    Seine: Short-to-long video diffusion model for generative transition and prediction , author=. The Twelfth International Conference on Learning Representations , year=

  89. [97]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Towards end-to-end generative modeling of long videos with memory-efficient bidirectional transformers , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  90. [98]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Deep video frame interpolation using cyclic frame generation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  91. [99]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Pantheon360: Taming Digital Twin Generation via 3D-Aware 360deg Video Diffusion , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  92. [100]

    arXiv preprint arXiv:2604.02329 , year=

    Generative World Renderer , author=. arXiv preprint arXiv:2604.02329 , year=

  93. [101]

    arXiv preprint arXiv:2512.23709 , year=

    Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion , author=. arXiv preprint arXiv:2512.23709 , year=

    arXiv

  94. [102]

    Advances in Neural Information Processing Systems , volume=

    Narcan: Natural refined canonical image with integration of diffusion prior for video editing , author=. Advances in Neural Information Processing Systems , volume=

  95. [103]

    arXiv preprint arXiv:2407.01519 , year=

    Diffir2vr-zero: Zero-shot video restoration with diffusion-based image restoration models , author=. arXiv preprint arXiv:2407.01519 , year=

  96. [104]

    arXiv preprint arXiv:2203.14206 , year=

    Denoising likelihood score matching for conditional score-based data generation , author=. arXiv preprint arXiv:2203.14206 , year=

    arXiv

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.