Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise

T0 review · 2 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FreqPrior is a frequency-domain noise-refinement method that keeps the video-diffusion prior near a standard Gaussian and, the paper reports, achieves the highest VBench total scores on three video diffusion models.

desk verdict A genuinely new noise-refinement idea with a real speed-up and consistent empirical gains, but the central Gaussianity guarantee only holds under an assumption the deployed pipeline violates. read the letter →

arxiv 2502.03496 v2 pith:RKVFDSII submitted 2025-02-05 eess.IV cs.GR

Yunlong Yuan , Yuanfan Guo , Chunwei Wang , Wei Zhang , Hang Xu , Li Zhang This is my paper · ORCID
classification eess.IVcs.GR
keywords videodiffusionmodelsnoisepriorinitializationfrequency-domainfilteringFouriertransformvariancedecaytext-to-videogenerationDDIMsamplingGaussian
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FreqPrior is a new noise-initialization strategy for text-to-video diffusion models. The paper argues that the initial noise should stay close to a standard Gaussian distribution, and that FreeInit's frequency filtering does not: it decays the noise variance, producing overly smooth videos with weak motion and lost detail. FreqPrior refines the noise with a frequency filter whose low-pass and high-pass terms obey $m^2 + (1-m^2) = 1$ rather than $m + (1-m) = 1$, so variance is preserved while low frequencies are retained and high frequencies enriched. The paper proves this refined prior has covariance error at most $\frac{\cos^2\theta}{1+\cos^2\theta}$ times FreeInit's, at least a 50% reduction, and reports the highest VBench total scores on VideoCrafter, ModelScope, and AnimateDiff, with about 23% less inference time than FreeInit thanks to partial sampling.

What carries the argument

The load-bearing object is the noise-refinement operator built from three steps: two correlated mixtures $x_1,x_2$ of the diffused latent $z_{noise}$ with fresh Gaussians $\eta_1,\eta_2$; a Fourier-transform filtering stage that retains low frequencies through a low-pass mask $M$ and enriches high frequencies through $(1-M^2)^{1/2}$ instead of the classical $1-M$; and a post-processing step that combines real and imaginary parts of two filtered transforms symmetrically. The square-law filter choice preserves Gaussianity because $m u + (1-m^2)^{1/2}v$ is standard Gaussian when $u,v$ are independent standard Gaussians, and the real/imaginary combination cancels cross terms. The proof uses the DFT identities $AB=BA=0$ and $A^2+B^2=NI$ to obtain the closed-form covariances and the inequality in Eq. (10).

What would settle it

Measure the empirical distribution of $z_{noise}$ at the chosen intermediate timestep on a real model, for example by collecting hundreds of latents and running a normality test that includes third and fourth moments. If $z_{noise}$ is clearly non-Gaussian while the covariance error stays tiny, then the theoretical guarantee in Eq. (10) does not strictly apply to the deployed pipeline, and the observed quality gains would need another explanation.

Watch

Extended reading notes

Core claim

The paper claims that a good noise prior for pretrained video diffusion models is one that remains essentially standard Gaussian after refinement. It identifies variance decay in FreeInit's refined noise, whose covariance matrix $\Sigma_{FreeInit}=P^2+(I-P)^2$ is strictly smaller than $I$, and shows that this decay blurs frames and reduces motion. FreqPrior instead produces refined noise with covariance $I - \frac{2\cos^2\theta}{1+\cos^2\theta}Q^2$, and the paper proves $\|I-\Sigma_{FreqPrior}\|_F \le \frac{\cos^2\theta}{1+\cos^2\theta} \|I-\Sigma_{FreeInit}\|_F$, i.e. at least a 50% lower covariance error, with numerical covariance errors around $10^{-26}$. The same low-frequency retention that preserves semantic fidelity is kept, so the method improves both quality and semantic scores on VBench while partial sampling cuts inference time by roughly 23%.

Load-bearing premise

Assumption 1 in Section 3.3 states that $z_{noise}$ after the diffusion process is standard Gaussian, $N(0,I)$, and in the deployed algorithm it is formed from a partially denoised DDIM latent $z_t$ plus injected noise, so the Gaussian-derived covariance formulas and the 50% bound inherit that approximation.

Editorial extensions

If this is right

  • FreqPrior can be applied to existing pretrained text-to-video diffusion models without fine-tuning, and it improves VBench total scores over Gaussian noise and FreeInit on VideoCrafter, ModelScope, and AnimateDiff.
  • The partial-sampling prior search, which perturbs a latent at intermediate timestep $t=321$, reduces prior-refinement cost to about 23% less inference time than FreeInit while keeping generation quality.
  • The square-law high-pass filter provides a general recipe for manipulating noise in the frequency domain without making its distribution deviate from standard Gaussian, a recipe that should transfer to other diffusion-model noise-editing tasks.
  • Because the refined prior remains near-Gaussian, downstream denoising sees a prior compatible with the model's training distribution, unlike frame-correlated priors that crash pretrained models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The covariance-error guarantee is a second-order statement; if the actual $z_{noise}$ in the pipeline is non-Gaussian, because it is built from a partially denoised latent, the approximation could still hold in practice, but higher-order statistics would be needed to confirm the theory transfers.
  • The same filter design could be tested on image diffusion models, where low-frequency leakage has also been reported, as a cheap extension beyond video generation.
  • A testable prediction of the paper's diagnosis is that deliberately decaying the variance of the initial Gaussian should reproduce FreeInit-style blur and motion loss, while increasing variance should degrade quality in the opposite direction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper introduces FreqPrior, a noise-initialization strategy for text-to-video diffusion models. The method iteratively refines initial Gaussian noise by (i) partially denoising with DDIM to an intermediate timestep, (ii) mixing that latent with fresh Gaussian noise in a prescribed ratio, (iii) filtering in the 3D Fourier domain with a low-pass mask M and a complementary high-pass mask (1-M^2)^{0.5}, and (iv) post-processing real/imaginary parts to form the next prior. The authors derive the distribution of the refined noise under Assumption 1 (that the mixed latent z_noise is standard Gaussian), prove a bound on its covariance error relative to FreeInit, and report improved VBench scores on VideoCrafter, ModelScope, and AnimateDiff with about 23% inference-time savings.

Significance. If the theoretical guarantee held for the actual deployed pipeline, this would be a valuable contribution: a principled frequency-filtering noise prior that preserves low-frequency semantics while enriching high-frequency detail and stays close to a standard Gaussian. The algebraic derivations in Appendices B and C are self-contained, the covariance-error bound in Eq. (10) is a clean result, and the numerical table is consistent with the formulas. The practical partial-sampling strategy is well motivated. However, the central theoretical claim is conditional on an assumption that the algorithm itself violates, and the empirical hyperparameters were tuned on the same benchmark used for the headline scores. The method remains plausible, but the paper's main justification for 'closely approximates a standard Gaussian distribution' is not yet established for the actual algorithm.

major comments (2)
  1. [Sec. 3.3, Eq. (1), Algorithm 1] Assumption 1 states that z_noise follows N(0,I), but the actual pipeline computes z_noise = sqrt(alpha_bar_T/alpha_bar_t) * z_t + sqrt(1 - alpha_bar_T/alpha_bar_t) * epsilon, where z_t is the DDIM partial-sampling output. The latent z_t is a deterministic function of the previous refined noise (and hence of previous epsilons), not an independent standard Gaussian, and it is generally correlated with epsilon through the refinement loop. All distributional results, including Eq. (53), Table 1, and the bound Eq. (10), are derived under this Gaussianity assumption; for a non-Gaussian z_noise, covariance error alone does not characterize the distance to N(0,I). The authors should either (i) prove a quantitative bound on the non-Gaussianity of z_noise (e.g., in total variation distance) as a function of sqrt(alpha_bar_T/alpha_bar_t), or (ii) restrict the theoretical claim to the idealized full-diffusion setting and explicitly present the partial-sampling variant as an empirically validated heuristic.
  2. [Sec. 4.1, Table 3, Figure 6, Table 2] The hyperparameters cos theta (0.7 for VideoCrafter, 0.8 for ModelScope and AnimateDiff) and t (321) are selected via ablations on VBench, and the final scores in Table 2 are measured on the same VBench benchmark. This makes the empirical comparison partially circular: the reported gains may contain an optimistic-bias component from tuning on the evaluation set. The paper should either reserve a separate validation split of VBench prompts for hyperparameter selection, or report results for a prespecified configuration and present the ablations solely as sensitivity analyses.
minor comments (6)
  1. [Related Work] The name 'ModelScopoe' should be 'ModelScope'.
  2. [Figure 2 caption] The caption contains a duplicated 'and' in 'sampling process, diffusion process, and and noise refinement'.
  3. [Sec. 4.1] The word 'open-soruce' should be 'open-source'.
  4. [Figure 6] The word 'accross' should be 'across'.
  5. [Sec. 3.1] The statement that at timestep t around 400 the latent 'has roughly taken shape and resembles the clean latent z_0' is an empirical claim that should be supported by evidence or a reference.
  6. [Eq. (10) and Table 1] The worst-case bound in Eq. (10) guarantees a reduction of at least 50% in covariance error, while Table 1 reports FreqPrior errors on the order of 1e-26 versus FreeInit errors of order 1-10, i.e., a reduction of essentially 100%; the paper should clarify whether the near-zero values follow from the specific filter choices and whether a much tighter bound is available.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the covariance analysis is an explicit linear-Gaussian computation under a stated assumption, and the empirical gains are measured against the external VBench benchmark.

full rationale

The paper's central theoretical bound, Equation (10), is derived by direct matrix algebra from the defined filtering operation (Equations (34)-(37)), the DFT identities of Theorem A.3, and the explicit Assumption 1 that z_noise is standard Gaussian. The bound ||I - Sigma_FreqPrior||_F <= (cos^2 theta/(1+cos^2 theta)) ||I - Sigma_FreeInit||_F does not reuse the empirically reported VBench scores as an input; it follows from the PSD inequality in Equation (61). Table 1 is a numerical evaluation of the same closed-form covariance expressions, not a fitted prediction, and the near-zero values reflect the designed use of (1-M^2)^0.5 filtering and symmetric real/imag recombination, which is a construction choice, not a circular inference. The VBench comparisons use an external benchmark and are not generated from the paper's own constants. The main caveat, stated as Assumption 1, is a validity condition: in the deployed Algorithm 1 with partial sampling (t=321), z_noise = sqrt(alpha_bar_T/alpha_bar_t) z_t + sqrt(1 - alpha_bar_T/alpha_bar_t) epsilon is not literally standard Gaussian if z_t is non-Gaussian and correlated with epsilon. This is a correctness risk about whether the assumption holds, not a case where the conclusion equals the input by definition. No load-bearing self-citations or imported uniqueness claims appear in the derivation chain, and the paper's own Limitations section (Appendix G) admits behavioral limitations such as unnatural smoothness and layout changes, which are not circularity devices.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method's theoretical guarantee rests on an idealized Gaussianity assumption, standard DFT matrix algebra, and the specific filter construction; empirically, the design assumes frequency content maps to semantic/detail quality and that a more Gaussian prior is better. Tunable parameters cos theta, t, n, and the filter cutoff were selected on the evaluation benchmark. No new physical entities are introduced.

free parameters (4)
  • cos theta = 0.7 (VideoCrafter), 0.8 (ModelScope, AnimateDiff)
    Controls the proportion of z_noise retained as low-frequency content; selected by ablating VBench total score in Table 3.
  • middle timestep t = 321
    Timestep at which partial sampling is interrupted; chosen as a quality/speed trade-off using VBench total score in Figure 6.
  • number of refinement iterations n = 2
    Number of sampling-refinement loops; set to 2 for all experiments, consistent with the FreeInit baseline.
  • Butterworth filter cutoff frequency = 0.25 normalized
    Spatial-temporal cutoff for the low-pass filter; fixed for both FreeInit and FreqPrior, but no sensitivity analysis is reported and the filter order is not specified.
assumptions (5)
  • domain assumption Assumption 1: z_noise follows a standard Gaussian distribution N(0,I) after the diffusion process.
    Stated in Sec 3.3 and used throughout Appendix B. In practice z_noise is a re-noised partially denoised latent, so this is approximate.
  • standard math The 3D DFT matrix satisfies A^2+B^2=NI and AB=BA=0 (Theorem A.3).
    Proved in Appendix A.2 and used in the covariance derivations of Eqs. (33) and (53).
  • domain assumption The low-pass mask Lambda_x is diagonal with entries in [0,1] and the high-pass mask is (I-Lambda_x^2)^{1/2}.
    This is the definition of the filtering step in Eq. (4); the theoretical bound in Eq. (10) relies on these properties.
  • domain assumption A noise prior closer to a standard Gaussian distribution produces better generated videos.
    Motivates the design and the interpretation of covariance error as a quality predictor; supported only by qualitative evidence in Figure 1.
  • domain assumption Low-frequency noise signals improve semantic fidelity and high-frequency signals improve imaging details.
    Used to justify retaining low frequencies while enriching high frequencies; not formally established in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise." pith.science (2026). https://pith.science/paper/RKVFDSII

@misc{pith2026250203496,
  author       = {Pith},
  title        = {Pith review of: FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RKVFDSII}},
  note         = {Machine review of arXiv:2502.03496}
}
read the original abstract

Text-driven video generation has advanced significantly due to developments in diffusion models. Beyond the training and sampling phases, recent studies have investigated noise priors of diffusion models, as improved noise priors yield better generation results. One recent approach employs the Fourier transform to manipulate noise, marking the initial exploration of frequency operations in this context. However, it often generates videos that lack motion dynamics and imaging details. In this work, we provide a comprehensive theoretical analysis of the variance decay issue present in existing methods, contributing to the loss of details and motion dynamics. Recognizing the critical impact of noise distribution on generation quality, we introduce FreqPrior, a novel noise initialization strategy that refines noise in the frequency domain. Our method features a novel filtering technique designed to address different frequency signals while maintaining the noise prior distribution that closely approximates a standard Gaussian distribution. Additionally, we propose a partial sampling process by perturbing the latent at an intermediate timestep during finding the noise prior, significantly reducing inference time without compromising quality. Extensive experiments on VBench demonstrate that our method achieves the highest scores in both quality and semantic assessments, resulting in the best overall total score. These results highlight the superiority of our proposed noise prior.

Figures

Figures reproduced from arXiv: 2502.03496 by the authors.

Figure 1
Figure 1. (Left) Generated video frames corresponding to Gaussian noise with different vari￾ance. As the variance, denoted as σ 2 , decreases from 1.002 to 0.962 , the imaging quality deteriorates and background details gradually lost. (Right) Comparisons of our method against the FreeInit and standard Gaussian noise. The frames generated using FreeInit appear overly smooth and blurred in the area of the highlighted red box, … view at source ↗
Figure 2
Figure 2. The framework of FreqPrior. It consists of three stages: sampling process, diffusion process, and and noise refinement. In the noise refinement stage, the noise is refined in three steps including noise preparation, noise processing, and post-processing. z i 0 , indicating the latent already has recovered large low-frequency information. Consequently, this modification yields nearly identical outcomes compared to di… view at source ↗
Figure 3
Figure 3. Generation results using PYoCo prior. Both mixed noise prior and progressive noise prior lead to crashes on pretrained video diffusion models. mantic consistency of the generated videos. In contrast, the mixed noise prior and progressive noise prior lead to crashes and failure in generating normal videos, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative results and comparisons. The cases in the top row are generated using AnimateDiff, while the middle row displays cases from ModelScope, and the bottom row shows cases generated by VideoCrafter. For each case, we present the generation results from different…
Figure 5
Figure 5. Figure 5: Generation results on different values of cos θ. Though there are some changes in the generated video frames as cos θ varies, they are quite similar. effects of cos θ, we set timestep t to 0 to eliminate the influence of t. In Equation (2), both x i 1 and x i 2 contrib…
Figure 6
Figure 6. Figure 6: Ablation study on the impact of timestep t. Total Score is assessed accross different diffusion timesteps t for three distinct text-to-video diffusion models. Overall, the timestep t has little effect on the evaluated metric. Influence of timestep t [PITH_FULL_IMAGE:f…
Figure 7
Figure 7. Figure 7: Visualization results of the influence of timestep. We present five cases where other settings are fixed to isolate the effects of varying timestep t. Overall, timestep t has minimal impact on the generation outcomes. However, it does exert some influences on the imagi…
Figure 8
Figure 8. Figure 8: More qualitative results. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. StrideDiffusion: Accelerating Diffusion Models for Time-series Generation

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A training-free sampler that adapts diffusion denoising strides to spectral band activity, cutting inference steps from 500-1000 to 14-66 with mostly comparable quality.

Reference graph

Works this paper leans on

61 extracted references · 43 canonical work pages · cited by 1 Pith paper

  1. [1]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G \" u l Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In ICCV, 2021

    work page 2021

  2. [2]

    Disentangling multiple features in video sequences using gaussian processes in variational autoencoders

    Sarthak Bhagat, Shagun Uppal, Zhuyun Yin, and Nengli Lim. Disentangling multiple features in video sequences using gaussian processes in variational autoencoders. In ECCV, 2020

    work page 2020

  3. [3]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In CVPR, 2023

    2023

  4. [4]

    Efros, and Tero Karras

    Tim Brooks, Janne Hellsten, Miika Aittala, Ting - Chun Wang, Timo Aila, Jaakko Lehtinen, Ming - Yu Liu, Alexei A. Efros, and Tero Karras. Generating long videos of dynamic scenes. In NeurIPS, 2022

    work page 2022

  5. [5]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \' e J \' e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021

    2021

  6. [6]

    Pascal Chang, Jingwei Tang, Markus Gross, and Vinicius C. Azevedo. How i warped your noise: a temporally-correlated noise prior for diffusion models. In ICLR, 2024

    work page 2024

  7. [7]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint, 2023

    work page 2023

  8. [8]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans on image synthesis. In NeurIPS, 2021

    work page 2021

Show all 61 references
  1. [9]

    Long video generation with time-agnostic VQGAN and time-sensitive transformer

    Songwei Ge, Thomas Hayes, Harry Yang, Xi Yin, Guan Pang, David Jacobs, Jia - Bin Huang, and Devi Parikh. Long video generation with time-agnostic VQGAN and time-sensitive transformer. In ECCV, 2022

    2022

  2. [10]

    Preserve your own correlation: A noise prior for video diffusion models

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia - Bin Huang, Ming - Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In ICCV, 2023

    2023

  3. [11]

    Goodfellow, Jean Pouget - Abadie, Mehdi Mirza, Bing Xu, David Warde - Farley, Sherjil Ozair, Aaron C

    Ian J. Goodfellow, Jean Pouget - Abadie, Mehdi Mirza, Bing Xu, David Warde - Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS, 2014

    2014

  4. [12]

    Reuse and diffuse: Iterative denoising for text-to-video generation

    Jiaxi Gu, Shicong Wang, Haoyu Zhao, Tianyi Lu, Xing Zhang, Zuxuan Wu, Songcen Xu, Wei Zhang, Yu - Gang Jiang, and Hang Xu. Reuse and diffuse: Iterative denoising for text-to-video generation. arXiv preprint, 2023

    2023

  5. [13]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In ICLR, 2024

    2024

  6. [14]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint, 2022

    2022

  7. [15]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020

    2020

  8. [16]

    Gritsenko, Diederik P

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey A. Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv preprint, 2022 a

    2022

  9. [17]

    Gritsenko, William Chan, Mohammad Norouzi, and David J

    Jonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models. In NeurIPS, 2022 b

    2022

  10. [18]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. In ICLR, 2023

    2023

  11. [19]

    Learning to decompose and disentangle representations for video prediction

    Jun - Ting Hsieh, Bingbin Liu, De - An Huang, Li Fei - Fei, and Juan Carlos Niebles. Learning to decompose and disentangle representations for video prediction. In NeurIPS, 2018

    2018

  12. [20]

    T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. In NeurIPS, 2023

    2023

  13. [21]

    Tag2text: Guiding vision-language model via image tagging

    Xinyu Huang, Youcai Zhang, Jinyu Ma, Weiwei Tian, Rui Feng, Yuejie Zhang, Yaqian Li, Yandong Guo, and Lei Zhang. Tag2text: Guiding vision-language model via image tagging. In ICLR, 2024 a

    2024

  14. [22]

    VBench : Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench : Comprehensive benchmark suite for video generative models. ...

    2024

  15. [23]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In NeurIPS, 2022

    2022

  16. [24]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In ICCV, 2021

    2021

  17. [25]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014

    2014

  18. [26]

    aesthetic-predictor, 2022

    LAION-AI. aesthetic-predictor, 2022. URL https://github.com/LAION-AI/aesthetic-predictor

    2022

  19. [27]

    Unmasked teacher: Towards training-efficient video foundation models

    Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang, Yinan He, Limin Wang, and Yu Qiao. Unmasked teacher: Towards training-efficient video foundation models. In ICCV, 2023 a

    2023

  20. [28]

    Amt: All-pairs multi-field transforms for efficient frame interpolation

    Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun-Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. In CVPR, 2023 b

    2023

  21. [29]

    Common diffusion noise schedules and sample steps are flawed

    Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In WACV, 2024

    2024

  22. [30]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan LI, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In NeurIPS, 2022

    2022

  23. [31]

    Videofusion: Decomposed diffusion models for high-quality video generation

    Zhengxiong Luo, Dayou Chen, Yingya Zhang, Yan Huang, Liang Wang, Yujun Shen, Deli Zhao, Jingren Zhou, and Tieniu Tan. Videofusion: Decomposed diffusion models for high-quality video generation. arXiv preprint, 2023

    2023

  24. [32]

    The lottery ticket hypothesis in denoising: Towards semantic-driven initialization

    Jiafeng Mao, Xueting Wang, and Kiyoharu Aizawa. The lottery ticket hypothesis in denoising: Towards semantic-driven initialization. In ECCV, 2024

    2024

  25. [33]

    GLIDE: towards photorealistic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: towards photorealistic image generation and editing with text-guided diffusion models. In ICML, 2022

    2022

  26. [34]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling. In ICLR, 2024

    2024

  27. [35]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021

    2021

  28. [36]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \" o rn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022

    2022

  29. [37]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015

    2015

  30. [38]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, 2023

    2023

  31. [39]

    Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with de...

    2022

  32. [40]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR, 2022

    2022

  33. [41]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion...

    2022

  34. [42]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. In ICLR, 2023

    2023

  35. [43]

    Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2

    Ivan Skorokhodov, Sergey Tulyakov, and Mohamed Elhoseiny. Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2. In CVPR, 2022

    2022

  36. [44]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015

    2015

  37. [45]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021 a

    2021

  38. [46]

    Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl - Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021 b

    2021

  39. [47]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023

    2023

  40. [48]

    RAFT: recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. RAFT: recurrent all-pairs field transforms for optical flow. In ECCV, 2020

    2020

  41. [49]

    Metaxas, and Sergey Tulyakov

    Yu Tian, Jian Ren, Menglei Chai, Kyle Olszewski, Xi Peng, Dimitris N. Metaxas, and Sergey Tulyakov. A good image generator is what you need for high-resolution video synthesis. In ICLR, 2021

    2021

  42. [50]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 2008

    2008

  43. [51]

    MCVD - masked conditional video diffusion for prediction, generation, and interpolation

    Vikram Voleti, Alexia Jolicoeur - Martineau, and Chris Pal. MCVD - masked conditional video diffusion for prediction, generation, and interpolation. In NeurIPS, 2022

    2022

  44. [52]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint, 2023

    2023

  45. [53]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, Ping Luo, Ziwei Liu, Yali Wang, Limin Wang, and Yu Qiao. Internvid: A large-scale video-text dataset for multimodal understanding and generation. In ICLR, 2024

    2024

  46. [54]

    GODIVA: generating open-domain videos from natural descriptions

    Chenfei Wu, Lun Huang, Qianxi Zhang, Binyang Li, Lei Ji, Fan Yang, Guillermo Sapiro, and Nan Duan. GODIVA: generating open-domain videos from natural descriptions. arXiv preprint, 2021

    2021

  47. [55]

    N \" u wa: Visual synthesis pre-training for neural visual world creation

    Chenfei Wu, Jian Liang, Lei Ji, Fan Yang, Yuejian Fang, Daxin Jiang, and Nan Duan. N \" u wa: Visual synthesis pre-training for neural visual world creation. In ECCV, 2022 a

    2022

  48. [56]

    Grit: A generative region-to-text transformer for object understanding

    Jialian Wu, Jianfeng Wang, Zhengyuan Yang, Zhe Gan, Zicheng Liu, Junsong Yuan, and Lijuan Wang. Grit: A generative region-to-text transformer for object understanding. arXiv preprint, 2022 b

    2022

  49. [57]

    Freeinit: Bridging initialization gap in video diffusion models

    Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu. Freeinit: Bridging initialization gap in video diffusion models. In ECCV, 2024

    2024

  50. [58]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  51. [59]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  52. [60]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  53. [61]

    a cat sleeping in a bowl

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.