Pith. sign in

REVIEW 4 major objections 4 minor 60 references

TLB-VFI: Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame Interpolation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TLB-VFI claims a new perceptual-quality bar for diffusion-based video frame interpolation while cutting training cost by orders of magnitude.

desk verdict A genuinely clever temporal-aware Brownian bridge design with a credible efficiency story, but the SOTA claim rests on adopted baselines from the authors' own prior paper and is not independently verifiable from the manuscript. read the letter →

arxiv 2507.04984 v1 pith:ZLVCJJ5I submitted 2025-07-07 cs.CV

Zonglin Lyu , Chen Chen This is my paper · ORCID
classification cs.CV
keywords videoframeinterpolationBrownianbridgediffusionlatentmodelstemporal-awareautoencoder3Dwavelettransformopticalflowguidanceperceptualmetricsefficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes TLB-VFI, a diffusion model for video frame interpolation that aims to combine the temporal awareness of video-based diffusion models with the small training cost of image-based ones. Its central claim is that by bridging between the latent encoding of a full three-frame clip and the encoding of the same clip with the unknown middle frame replaced by zeros, the Brownian bridge restores missing temporal information instead of collapsing into an identity map. On perceptual metrics (FID, LPIPS, FloLPIPS) the method reports the best scores across SNU-FILM, Xiph, and DAVIS under its stated fair comparison, with roughly 20% FID improvement over the second-best method on the hardest subsets. Alongside accuracy it claims an order-of-magnitude reduction in training data, parameters, and inference time relative to video diffusion baselines, which matters because good interpolation underpins video compression and novel-view synthesis.

What carries the argument

The load-bearing object is the latent Brownian bridge with deliberately separated endpoints. The diffusion process is $q(x_t|x_0,x_T) = N(t/T x_0 + (1 - t/T) x_T, t(T-t)/T I)$ with $T=2$, and the denoising U-Net is trained to predict $x_t - x_0$; the method's Proposition 1 states the bridge is not an identity map only if $\mathbb{E}(x_0 - x_T) \neq 0$, which the zero-replaced middle frame creates. Around that bridge sits a temporal-aware autoencoder: an image encoder shared across frames, encoder/decoder temporal blocks made of 3D convolution and spatiotemporal self-/cross-attention, multi-level feature sharing of $I_0, I_1$ into the decoder, and a 3D-wavelet feature gating $\sigma(f_w) \odot f_i + f_i$ that multiplies pixel-space high-frequency information into the latent features. Optical flow warping plus mask/residual prediction gives the pixel-level guidance that lets the diffusion model run in latent space with only 10 sampling steps.

What would settle it

Re-run Consec. BB [27], LDMVFI [9], and PerVFI [46] on the exact test frames and sample sizes used to compute TLB-VFI's Table 1 FID/LPIPS values; if the about-20% margins on SNU-FILM-extreme and Xiph-4K shrink to within noise once protocols are matched, or PerVFI's numbers beat them when it is included in the ranking, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The paper's discovery is that the efficiency problem of video-based diffusion interpolation can be solved without giving up temporal modelling. The method first trains an autoencoder whose encoder sees the full clip [I0, In, I1] and whose decoder outputs a mask and residual that warp the two known frames; temporal blocks inside the autoencoder use 3D convolutions and spatiotemporal attention in latent space, while a 3D-wavelet gating mechanism injects pixel-level high-frequency temporal information. For inference the unknown middle frame is replaced by zeros, and a Brownian bridge diffusion process -- with endpoints $x_0 = E([I_0, I_n, I_1])$ and $x_T = E([I_0, 0, I_1])$ -- recovers the missing latent features. The paper argues, via Proposition 1, that this only works when the endpoint distributions differ significantly in expectation, and it verifies that its zero-replacement scheme produces a 40-55% MAPE gap whereas the earlier Consecutive Brownian Bridge setup produces less than 1%, explaining why the earlier bridge behaved as identity. The result is a claim of best perceptual quality, measured by FID/LPIPS/FloLPIPS, at 3x fewer parameters and 2.3x faster inference than image-based diffusion baselines and over 20x fewer parameters with 9000x less training data than video diffusion baselines.

Load-bearing premise

The central claim rests on the assumption that the Table 1 baseline numbers -- copied from an earlier paper by the same authors -- were measured under the same protocol, and that leaving PerVFI out of the ranking because of its extra training data is justified.

Editorial extensions

If this is right

  • On the paper's own comparison, TLB-VFI is the top-scoring method on every dataset and perceptual metric, beating the second-best image-based diffusion models by about 20% FID and FloLPIPS on SNU-FILM-extreme and Xiph-4K.
  • At 46.7M parameters it has roughly 3x fewer parameters and 2.3x faster inference than Consec. BB under matched 10-step sampling, and over 2x faster than LDMVFI.
  • Compared with video-based diffusion methods such as VIDIM, Dreammover, and ViBiDSampler, it needs about 9000x less training data, over 20x fewer parameters, and runs over 10x faster, because optical-flow guidance supplies pixel-level structure.
  • The method can take more than three frames as input, predicting two target frames in a single sampling run where image-based diffusion baselines must sample twice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the mean-shift condition in Proposition 1 is a general design rule for Brownian-bridge latent translation -- if endpoints are too similar the bridge degenerates into identity -- and could be applied to other conditional generation tasks beyond video frame interpolation.
  • Editorial extension: if the reported efficiency gains hold in independent re-implementations, flow-guided latent diffusion becomes a viable template for real-time perceptual-quality video processing, since the heavy video diffusion stacks are replaced by a small 3D U-Net and a wavelet gate.
  • Editorial extension: the paper's documented PSNR/SSIM versus LPIPS disagreement suggests that pixel metrics may favour blurry solutions, so perceptual-metric evaluation of VFI could change which methods are adopted in practice; TLB-VFI's gap over PerVFI on FID but not on PSNR is consistent with that reading.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes TLB-VFI, a latent Brownian Bridge diffusion model for video frame interpolation. The method combines a temporal-aware autoencoder that extracts temporal information in latent space with 3D convolutions and spatiotemporal attention, a 3D-wavelet feature gating mechanism that captures pixel-level temporal frequencies, and optical-flow-based prediction of a mask and residual. The diffusion model operates between the latent features of the original video clip and a clip where the intermediate frame is zeroed out. The paper reports state-of-the-art LPIPS, FloLPIPS, and FID results on Xiph, DAVIS, and SNU-FILM, including roughly 20% FID improvements on Xiph-4K and SNU-FILM-extreme, with lower inference cost than prior diffusion-based VFI methods.

Significance. If the empirical results hold under a consistent and independently verifiable evaluation protocol, the paper is a meaningful contribution: it shows that a diffusion-based VFI model can achieve perceptual quality competitive with or better than existing methods while being substantially lighter and faster than video-diffusion baselines. The architectural ideas are clearly presented, the ablation study gives a partial decomposition of the contributions, and the authors state that code and results will be released. However, the current evaluation is self-referential: all baseline numbers in Table 1 are adopted from the authors' own Consec. BB paper, the strongest perceptual competitor PerVFI is excluded from ranking, and no error bars or sample sizes are reported. The theoretical Proposition 1 is also close to tautological. The significance is therefore conditional on a rigorous re-evaluation.

major comments (4)
  1. [Table 1 and Sec. 4.2] The central claim of state-of-the-art performance rests on baseline numbers 'adopted from Consec.BB [27]', a paper with the same first author, rather than on re-running those baselines under the authors' own evaluation protocol. FID, LPIPS, and FloLPIPS depend on the number of evaluated images and on the sampling procedure (e.g., number of diffusion steps), so the reported roughly 20% FID improvements over the second-best method (e.g., Xiph-4K: 24.424 to 19.114; SNU-FILM-extreme: 36.631 to 29.868) could be an artifact of protocol mismatch. The paper must either re-run all baselines with the identical protocol (same sample count, metric version, and diffusion-step schedule) or provide error bars and sample sizes to demonstrate that the margins exceed measurement noise. The current statement 'Under a fair comparison' in Sec. 4.2 is not supported without this evidence.
  2. [Table 1 and Sec. 4.2] Excluding PerVFI from ranking because of its larger training scale is a defensible fairness choice, but the unqualified sentence 'our method achieves the best performance in all metrics and datasets' is misleading because PerVFI is the strongest perceptual-quality competitor in the table and the text itself acknowledges that PerVFI's training data are different. The claim should be restricted to 'best among methods trained only on Vimeo-90K triplets', or PerVFI should be included with a separate rank so that readers can see the full comparison. As written, the headline result in the abstract and intro overstates the generality of the SOTA claim.
  3. [Sec. 3.3 and Supp. Sec. 9.3] Proposition 1 is not proven by the supplied induction. The proof in Supp. Sec. 9.3 restates that if xt - x0 is nonzero, then the expected sampled latent is not x0, which follows directly from the Brownian Bridge sampling equation and does not establish the proposition's requirement of a 'large shift between E(x0) and E(xT)'. The proposition is therefore essentially a tautology, and the subsequent t-test in Sec. 4.3 only confirms that a distribution shift exists for the proposed design; it does not validate that the shift is necessary or sufficient for the claimed effectiveness of the Brownian Bridge. The theoretical justification should be either replaced with a nontrivial quantitative statement or removed from the list of contributions.
  4. [Sec. 4.3] The t-test comparing MAPE values between the proposed method and Consec. BB reports t-statistics of greater than 21 and about 0.0001 on SNU-FILM-extreme, but the sample size, the specific feature layer used, and the normalization for computing MAPE are not specified. Without these details, the test cannot be assessed, and the conclusion that the Brownian Bridge 'learns to reduce this gap' is not supported by the reported statistics. Please provide the sample size, the feature space, and a confidence interval or effect size for the comparison.
minor comments (4)
  1. [Fig. 4 caption] The caption ends with 'PSNR/SSIM are not.' which is an incomplete sentence; it should read 'PSNR/SSIM are not consistent with visual quality.'
  2. [Supp. Sec. 9.2] The text defines both the high-pass and low-pass filters as [1/sqrt(2), -1/sqrt(2)]; the low-pass filter should be [1/sqrt(2), 1/sqrt(2)].
  3. [Sec. 8.2] The phrase 'do500% zoom-in' is missing a space; it should read 'do 500% zoom-in'.
  4. [Sec. 3.3] The notation 'height, and weight' should be 'height, and width'.

Circularity Check

2 steps flagged · score 4.0 of 10

Headline SOTA claim rests on baseline numbers adopted from the authors' own Consec. BB paper; the 'theoretical constraint' is an engineered identity-avoidance tautology.

  1. self definitional [Sec. 3.3 (Proposition 1) and Sec. 4.3 (Distribution Shift in Brownian Bridge)]
    "Therefore, we need a significantly large distribution shift between x0 and xT to prevent such a problem of identity mapping. Specifically, a big shift in the mean is required, which is described in the following proposition: Proposition 1. If the Brownian Bridge Diffusion is applied to translate between two distributions x0 and xT , there should be a large shift between E(x0) and E(xT ). A sufficient constraint is to reject the Null Hypothesis H0 : E(x0 − xT ) = 0 at significance level α."

    The proposition is an analytic restatement of the Brownian bridge equations: if E(x0) = E(xT), the diffusion and sampling processes are the identity, so rejecting H0 is trivially necessary for a non-identity map. The paper then engineers the shift by construction, replacing In with 0 to form Ṽ = [I0, 0, I1], and Sec. 4.3's t-test (>21) confirms that this engineered shift exists. Calling this a 'validating' theoretical constraint is self-definitional: the effectiveness criterion (reject H0) is manufactured by the design choice, and the empirical 'confirmation' checks the construction rather than independently demonstrating the Brownian bridge's temporal restoration ability.

  2. self citation load bearing [Tab. 1 footnote and Sec. 4.2 (Quantitative Evaluation)]
    "Results for baselines, except for PerVFI, are adopted from Consec.BB [27]. ... Under a fair comparison, our method achieves the best performance in all metrics and datasets, as shown in Tab. 1."

    The central SOTA claim is justified by Tab. 1, whose baseline rows are 'adopted from Consec.BB [27]' — a paper with the same first author as the present work. The comparison is therefore between the current method and the authors' own previously published numbers, computed under the present paper's protocol (10 sampling steps, with the source runtime reported separately as 2.60 s). No independent re-run or external evaluation protocol is supplied, so the headline 20% FID margins are inherited from a self-citation rather than established by an independent benchmark, making the load-bearing comparison self-referential.

full rationale

The paper's core architectural derivation — temporal-aware autoencoder, 3D-wavelet feature gating, optical-flow-guided latent Brownian bridge, and the ablation studies in Sec. 4.3 — is self-contained and not circular: the components are defined from the task and evaluated against the paper's own ablations. However, the headline claim has two self-referential components. First, Tab. 1 imports all non-PerVFI baseline numbers from Consec. BB [27], a same-first-author paper, and the 'best performance in all metrics and datasets' sentence is therefore a comparison of the present method against the authors' own previously published numbers rather than an independent re-evaluation; the protocol mismatch (unreported FID sample sizes, different native sampling setup) makes this burden concrete. Second, Proposition 1 is a tautology — a Brownian bridge with coincident endpoints is an identity map — and the paper then engineers the endpoint shift by zero-replacing In and uses a t-test to confirm that the engineered shift exists, which validates the design premise but not an independent capability. These issues do not make the method's architecture derivation circular, but they do make the central empirical claim partly self-referential and the stated 'theoretical constraint' self-definitional. No fitted parameter is renamed as a prediction, and the method's own ablations are independent, so the overall circularity score is moderate.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new physical entities or forces are postulated. The method introduces architectural components (temporal blocks, 3D-wavelet gating) that are not claimed to be new natural kinds. The main 'free parameter' is the diffusion time horizon T, inherited from prior work. The core assumptions are about latent-space learnability and the guidance provided by optical flow.

free parameters (1)
  • Diffusion time horizon T = 2
    Chosen by hand, following Consec. BB [27]; affects the diffusion variance but is not fitted to data.
assumptions (3)
  • domain assumption Brownian Bridge diffusion process (Eq. 5-6) correctly models the transition between latent codes of complete and zero-masked video clips.
    The method's success depends on the diffusion model learning to map E(tilde V) to E(V); the paper provides no proof this is learnable beyond empirical FID results.
  • domain assumption Replacing the middle frame with zeros creates a distribution shift large enough to avoid the identity-mapping failure of Consec. BB.
    Tested empirically via a t-test on SNU-FILM-extreme features (t>21), but the t-test only shows non-zero difference, not that the shift is in the direction needed for interpolation.
  • domain assumption Optical flow warping with mask/residual prediction provides sufficient pixel guidance; the flow estimator is adopted from Consec. BB and trained jointly.
    The method's efficiency relies on not generating pixels from scratch; this is standard in VFI but is an assumption about the availability and accuracy of flow estimates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TLB-VFI: Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame Interpolation." pith.science (2026). https://pith.science/paper/ZLVCJJ5I

@misc{pith2026250704984,
  author       = {Pith},
  title        = {Pith review of: TLB-VFI: Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame Interpolation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZLVCJJ5I}},
  note         = {Machine review of arXiv:2507.04984}
}
abstract

Video Frame Interpolation (VFI) aims to predict the intermediate frame $I_n$ (we use n to denote time in videos to avoid notation overload with the timestep $t$ in diffusion models) based on two consecutive neighboring frames $I_0$ and $I_1$. Recent approaches apply diffusion models (both image-based and video-based) in this task and achieve strong performance. However, image-based diffusion models are unable to extract temporal information and are relatively inefficient compared to non-diffusion methods. Video-based diffusion models can extract temporal information, but they are too large in terms of training scale, model size, and inference time. To mitigate the above issues, we propose Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame Interpolation (TLB-VFI), an efficient video-based diffusion model. By extracting rich temporal information from video inputs through our proposed 3D-wavelet gating and temporal-aware autoencoder, our method achieves 20% improvement in FID on the most challenging datasets over recent SOTA of image-based diffusion models. Meanwhile, due to the existence of rich temporal information, our method achieves strong performance while having 3times fewer parameters. Such a parameter reduction results in 2.3x speed up. By incorporating optical flow guidance, our method requires 9000x less training data and achieves over 20x fewer parameters than video-based diffusion models. Codes and results are available at our project page: https://zonglinl.github.io/tlbvfi_page.

Figures

Figures reproduced from arXiv: 2507.04984 by the authors.

Figure 1
Figure 1. Overview of the proposed method. (a) Training autoencoder. The autoencoder is trained with video clip V = [I0, In, I1] and aims to reconstruct In. It contains an image encoder (shared for all frames) and an image decoder, where multi-level encoder features from I0, I1 are passed to the decoder. Temporal blocks extract temporal information in the latent space and aggregate video features into a single image feature f… view at source ↗
Figure 2
Figure 2. (a) Model Pipeline. The Image Encoder is shared across all frames, and temporal blocks extract temporal information in the latent space. (b) Multi-level Feature Sharing. The Image Encoder and Decoder consist of several levels of resolution due to downsampling/upsampling latent features. At the i th level of the encoder and the decoder, features from I0 and I1 in the encoder are warped and concatenated with the origi… view at source ↗
Figure 3
Figure 3. Qualitative Comparison between our method and recent SOTAs. The leftmost image is the overlaid image of I0 and I1 (blended image). Areas with drastic motion changes are cropped with blue boxes to better visualize the results. Red circles and boxes indicate the area where we perform significantly better. Our method achieves better visual quality than recent SOTAs. Additional qualitative results are included in the Su… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Inconsistency between PSNR/SSIM and Visual Quality. Red circles and arrows indicate where the results from EMAVFI are distorted. PSNR/SSIM are not. Training Cost of Diffusion Based Models. As claimed, our method achieves 20× fewer model parameters and re￾quires 9000× l…
Figure 6
Figure 6. Figure 6: MAP E(E(In), E(I0)) in the setup of Consec. BB [27] and MAP E(E(V ), E(V˜ )) in our method. V˜ is the video clip with the inter￾mediate frame replaced with 0s. The MAPE in Consec. BB is less than 1%, resulting in a rough identity transformation. In our method, MAPE is …
Figure 7
Figure 7. Figure 7: (a) Given four neighboring frames I0, I1, I3, I4, we can predict the intermediate frame I2. (b) Given a sequence of frames I0, I2, I4, we can predict the intermediate frame between each adjacent pair I1, I3 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Visualization of optical flows. Algorithm 1 Diffusion Training Algorithm 1: Let E be the encoder part of our autoencoder. 2: for i = 1 to Ntraining steps do 3: Sample t ∼ ContinuousUniform(0, T). 4: Sample [I0, In, I1] from Dataset. 5: x0 = E([I0, In, I1]). 6: Compute …
Figure 9
Figure 9. Figure 9: Additional qualitative comparison between our method and recent SOTAs. The leftmost image is the overlaid image of I0 and I1 (blended image). Images inside blue boxes contain drastic motion changes and are cropped out to show details of interpolation results. Red circl…
Figure 10
Figure 10. Figure 10: Additional qualitative comparison between our method and recent SOTAs. The leftmost image is the overlaid image of I0 and I1 (blended image). Images inside blue boxes contain drastic motion changes and are cropped out to show details of interpolation results. Red circ…
Figure 11
Figure 11. Figure 11: Visual comparison of 8x× interpolation results. We include a visual comparison of 8× interpolation between our method and PerVFI. Red arrows indicate where our method is visually better. Additional comparisons (in video form) are provided in our Project Page. 6 [PITH…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 54 canonical work pages

  1. [27]

    Frame Interpolation with Consecutive Brownian Bridge Diffusion

    Zonglin Lyu, Ming Li, Jianbo Jiao, and Chen Chen. Frame interpolation with consecutive brownian bridge diffusion. arXiv preprint arXiv:2405.05953, 2024. 2, 3, 4, 5, 6, 8, 1

    work page Pith review arXiv 2024

  2. [1]

    Long-term video frame interpolation via feature propagation

    Dawit Mureja Argaw and In So Kweon. Long-term video frame interpolation via feature propagation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, 2022. 2, 3

    work page 2022

  3. [2]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G ¨ul Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. In IEEE International Conference on Computer Vision, 2021. 7

    work page 2021

  4. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  5. [4]

    D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In European Conf. on Computer Vision (ECCV), 2012. 6

    work page 2012

  6. [5]

    Video frame interpo- lation via deformable separable convolution

    Xianhang Cheng and Zhenzhong Chen. Video frame interpo- lation via deformable separable convolution. In Proceedings of the AAAI Conference on Artificial Intelligence, 2020. 2, 3

    work page 2020

  7. [6]

    High-quality frame interpolation via tridirectional inference

    Jinsoo Choi, Jaesik Park, and In So Kweon. High-quality frame interpolation via tridirectional inference. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021. 2, 3

    work page 2021

  8. [7]

    Channel attention is all you need for video frame interpolation

    Myungsub Choi, Heewon Kim, Bohyung Han, Ning Xu, and Kyoung Mu Lee. Channel attention is all you need for video frame interpolation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2020. 2, 5, 6, 8, 1

    work page 2020

Show all 60 references
  1. [8]

    Flolpips: A bespoke video quality metric for frame interpolation

    Duolikun Danier, Fan Zhang, and David Bull. Flolpips: A bespoke video quality metric for frame interpolation. In2022 Picture Coding Symposium (PCS). IEEE, 2022. 6

    2022

  2. [9]

    Ldmvfi: Video frame interpolation with latent diffusion models

    Duolikun Danier, Fan Zhang, and David Bull. Ldmvfi: Video frame interpolation with latent diffusion models. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 1472–1480, 2024. 2, 3, 4, 5, 6, 8, 1

    2024

  3. [10]

    Dosovitskiy, P

    A. Dosovitskiy, P. Fischer, E. Ilg, P. H¨ausser, C. Hazırbas ¸, V . Golkov, P. v.d. Smagt, D. Cremers, and T. Brox. Flownet: Learning optical flow with convolutional networks. In IEEE International Conference on Computer Vision (ICCV), 2015. 6

    2015

  4. [11]

    Non-linear motion estimation for video frame inter- polation using space-time convolutions

    Saikat Dutta, Arulkumar Subramaniam, and Anurag Mit- tal. Non-linear motion estimation for video frame inter- polation using space-time convolutions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022. 2, 3

    2022

  5. [12]

    Deepstereo: Learning to predict new views from the world’s imagery

    John Flynn, Ivan Neulander, James Philbin, and Noah Snavely. Deepstereo: Learning to predict new views from the world’s imagery. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. 1

    2016

  6. [13]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems ,

  7. [14]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2, 3

    2020

  8. [15]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 3

    2022

  9. [16]

    Real-time intermediate flow estimation for video frame interpolation

    Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. In European Conference on Com- puter Vision, 2022. 2, 3

    2022

  10. [17]

    Video interpolation with diffu- sion models

    Siddhant Jain, Daniel Watson, Eric Tabellion, Ben Poole, Janne Kontkanen, et al. Video interpolation with diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7341– 7351, 2024. 2, 3, 6, 7, 8

    2024

  11. [18]

    A unified pyramid recurrent net- work for video frame interpolation

    Xin Jin, Longhai Wu, Jie Chen, Youxin Chen, Jayoon Koo, and Cheul-hee Hahm. A unified pyramid recurrent net- work for video frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 3, 6, 2

    2023

  12. [19]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. 2

    2015

  13. [20]

    Ifrnet: Intermediate feature refine network for efficient frame interpolation

    Lingtong Kong, Boyuan Jiang, Donghao Luo, Wenqing Chu, Xiaoming Huang, Ying Tai, Chengjie Wang, and Jie Yang. Ifrnet: Intermediate feature refine network for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CV...

    2022

  14. [21]

    Adacof: Adaptive col- laboration of flows for video frame interpolation

    Hyeongmin Lee, Taeoh Kim, Tae-young Chung, Daehyun Pak, Yuseok Ban, and Sangyoun Lee. Adacof: Adaptive col- laboration of flows for video frame interpolation. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020. 2, 3

    2020

  15. [22]

    Bbdm: Image- to-image translation with brownian bridge diffusion models

    Bo Li, Kaitao Xue, Bin Liu, and Yu-Kun Lai. Bbdm: Image- to-image translation with brownian bridge diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2, 3, 5

    2023

  16. [23]

    Amt: All-pairs multi-field transforms for efficient frame interpolation

    Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun- Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. In IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) ,

  17. [24]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 2022. 3

    2022

  18. [25]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongx- uan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022. 3

    2022 arXiv

  19. [26]

    Video frame interpolation with transformer

    Liying Lu, Ruizheng Wu, Huaijia Lin, Jiangbo Lu, and Jiaya Jia. Video frame interpolation with transformer. In Proceed- 9 ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 2, 3, 6

    2022

  20. [28]

    S.G. Mallat. A theory for multiresolution signal decomposi- tion: the wavelet representation. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 1989. 5

    1989

  21. [29]

    Mayer, E

    N. Mayer, E. Ilg, P. H ¨ausser, P. Fischer, D. Cremers, A. Dosovitskiy, and T. Brox. A large dataset to train convo- lutional networks for disparity, optical flow, and scene flow estimation. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 6

    2016

  22. [30]

    Softmax splatting for video frame interpolation

    Simon Niklaus and Feng Liu. Softmax splatting for video frame interpolation. InIEEE Conference on Computer Vision and Pattern Recognition, 2020. 2, 5, 6, 1

    2020

  23. [31]

    Video frame in- terpolation via adaptive convolution

    Simon Niklaus, Long Mai, and Feng Liu. Video frame in- terpolation via adaptive convolution. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, 2017. 2, 3

    2017

  24. [32]

    Video frame in- terpolation via adaptive separable convolution

    Simon Niklaus, Long Mai, and Feng Liu. Video frame in- terpolation via adaptive separable convolution. In Proceed- ings of the IEEE international conference on computer vi- sion, 2017. 2, 3

    2017

  25. [33]

    Biformer: Learning bilateral motion estimation via bilateral trans- former for 4k video frame interpolation

    Junheum Park, Jintae Kim, and Chang-Su Kim. Biformer: Learning bilateral motion estimation via bilateral trans- former for 4k video frame interpolation. In Computer Vision and Pattern Recognition, 2023. 2, 3

    2023

  26. [34]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine- Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 5, 1

    2016

  27. [35]

    Frame interpolation transformer and uncertainty guidance

    Markus Plack, Karlis Martins Briedis, Abdelaziz Djelouah, Matthias B Hullin, Markus Gross, and Christopher Schroers. Frame interpolation transformer and uncertainty guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2, 3

    2023

  28. [36]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022. 3, 2

    2022

  29. [37]

    Stochastic processes

    Sheldon M Ross. Stochastic processes. 1995. 3

    1995

  30. [38]

    Dreammover: Leveraging the prior of diffusion models for image interpolation with large motion

    Liao Shen, Tianqi Liu, Huiqiang Sun, Xinyi Ye, Baopu Li, Jianming Zhang, and Zhiguo Cao. Dreammover: Leveraging the prior of diffusion models for image interpolation with large motion. arXiv preprint arXiv:2409.09605, 2024. 2, 3, 6, 7, 8

    2024 arXiv

  31. [39]

    Video frame interpolation via generalized deformable convolution

    Zhihao Shi, Xiaohong Liu, Kangdi Shi, Linhui Dai, and Jun Chen. Video frame interpolation via generalized deformable convolution. IEEE transactions on multimedia, 2021. 2, 3

    2021

  32. [40]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021. 3

    2021

  33. [41]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16, 2020. 6

    2020

  34. [42]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 3, 4, 2

    2017

  35. [43]

    Mcvd-masked conditional video diffusion for prediction, generation, and interpolation

    Vikram V oleti, Alexia Jolicoeur-Martineau, and Chris Pal. Mcvd-masked conditional video diffusion for prediction, generation, and interpolation. Advances in neural informa- tion processing systems, 2022. 2, 3, 6, 7, 8

    2022

  36. [44]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing,

  37. [45]

    Video compression through image interpolation

    Chao-Yuan Wu, Nayan Singhal, and Philipp Krahenbuhl. Video compression through image interpolation. InProceed- ings of the European conference on computer vision (ECCV),

  38. [46]

    Perception-oriented video frame interpolation via asymmetric blending

    Guangyang Wu, Xin Tao, Changlin Li, Wenyi Wang, Xiao- hong Liu, and Qingqing Zheng. Perception-oriented video frame interpolation via asymmetric blending. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2753–2762, 2024. 2, 3, 4, 5, 6, 1

    2024

  39. [47]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022. 6

    2022

  40. [48]

    Video enhancement with task-oriented flow

    Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with task-oriented flow. International Journal of Computer Vision (IJCV) ,

  41. [49]

    Vibidsam- pler: Enhancing video interpolation using bidirectional dif- fusion sampler

    Serin Yang, Taesung Kwon, and Jong Chul Ye. Vibidsam- pler: Enhancing video interpolation using bidirectional dif- fusion sampler. arXiv preprint arXiv:2410.05651, 2024. 2, 3, 6, 7, 8

    2024 arXiv

  42. [50]

    Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation

    Guozhen Zhang, Yuhan Zhu, Haonan Wang, Youxin Chen, Gangshan Wu, and Limin Wang. Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2, 3, 5, 6

    2023

  43. [51]

    Vfimamba: Video frame interpolation with state space models

    Guozhen Zhang, Chunxu Liu, Yutao Cui, Xiaotong Zhao, Kai Ma, and Limin Wang. Vfimamba: Video frame interpolation with state space models. arXiv preprint arXiv:2407.02315, 2024. 2

    2024 arXiv

  44. [52]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 6 10 TLB-VFI: Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame Interpolation Supplementary Material

    2018

  45. [53]

    7 includes optical basics: what are optical flows and warping

    Overview The supplementary material is structured as follows: • Sec. 7 includes optical basics: what are optical flows and warping. • Sec. 8.1 contains PSNR/SSIM evaluated on our selected datasets. • Sec. 8.2 contains additional qualitative results. • Sec. 9.1 contains impleme...

  46. [54]

    Optical Flow Basics Optical flow is the pixel-wise movement from frame to frame. If we have two images I0 and I1, and for a given pixel I0[i, j] the corresponding pixel appears in I1 I1[i′, j′] then f low(I0, I1)[i, j] is [i′ − i, j′ − j], indicating the pixel movement. Warpin...

  47. [55]

    Results in PSNR/SSIM We include the results in PSNR/SSIM on our selected datasets in Tab

    Additional Results 8.1. Results in PSNR/SSIM We include the results in PSNR/SSIM on our selected datasets in Tab. 4. We can see that PSNR/SSIM tends to be unstable and not correlated to visual qualities for meth- ods in 2024. For example, our method underperforms Con- sec. BB ...

    2024

  48. [56]

    same padding

    Additional Details 9.1. Implementation Details Flow Estimator. Optical flow estimation is not our research purpose, so we use the same architecture of flow estima- tor in Consec. BB [27] and trained together with our au- toencoder. The code for differentiable warping is availa...

  49. [57]

    (10) in the main paper, suppose that at a given time step t, xt − x0 ̸= 0 , then the expectation of sampled latent at any previous step s is E(xs|xt) = s t (xt − x0) + x0

    Based on Eq. (10) in the main paper, suppose that at a given time step t, xt − x0 ̸= 0 , then the expectation of sampled latent at any previous step s is E(xs|xt) = s t (xt − x0) + x0

  50. [58]

    By the inductive assumption, E(xs|xt) = x0 + δ, where δ ̸= 0 ⇐ ⇒E(xs|xt) ̸= x0

  51. [59]

    Note that this is especially important in DDIM [40] sam- pling because the variance term is removed, in which case we can directly conclude that xs|xt ̸= x0 without expectation

    Then, on expectation, we conclude that xs|xt ̸= x0. Note that this is especially important in DDIM [40] sam- pling because the variance term is removed, in which case we can directly conclude that xs|xt ̸= x0 without expectation

  52. [60]

    As a result, the sampling process is not an identity map

    Therefore, we can prove this proposition by the above in- duction because the sampling process is discretized into finite steps. As a result, the sampling process is not an identity map. On the other hand, the sampling process is trivial because it does not change the expectat...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.