Pith. sign in

REVIEW 4 major objections 6 minor 60 references

E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training a diffusion model by backpropagating a reconstruction loss through the full few-step sampling chain—instead of predicting noise one step at a time—directly optimizes the final generated sample, and the paper reports that this…

desk verdict The empirical fine-tuning recipe gives modest few-step gains over PixArt-δ, but the paper's central claim—that the reconstruction objective learns a direct noise-to-data map—is mathematically wrong because z_T is independent of z_0, making the Bayes-optimal solution the per-prompt conditional mean. read the letter →

arxiv 2412.21044 v2 pith:CKTMG2OX submitted 2024-12-30 cs.CV

Zhiyu Tan , WenXu Qian , Hesen Chen , Mengping Yang , Lei Chen , Hao Li This is my paper · ORCID
classification cs.CV
keywords diffusionmodelsend-to-endtrainingfew-stepsamplinglatentconsistencytext-to-imagegenerationperceptuallossadversarialtraining-samplinggap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that diffusion models should be trained by directly mapping pure Gaussian noise to the target data in one differentiable pass, rather than by predicting noise at randomly sampled intermediate steps. The authors introduce an end-to-end objective that measures the reconstruction error between the real latent code and the final output of the full sampling chain, so the loss is applied exactly where generation quality matters. They report that this approach removes the training-sampling discrepancy, avoids reliance on the terminal noise being perfectly Gaussian, and permits perceptual and adversarial losses to be folded into the objective. On COCO30K and HW30K benchmarks, their fine-tuned model improves FID and CLIP scores relative to stepwise-trained few-step baselines at 3–4 sampling steps, which is why a reader would care: it suggests a simple loss change can make diffusion models faster and more controllable without new architectures.

What carries the argument

The differentiable unrolled sampling chain is the load-bearing object. It is a fixed few-step denoising trajectory, run with a consistency-model scheduler from pure Gaussian noise to a reconstructed latent, in which every step uses the same shared network parameters; the reconstruction loss $d(z_0, \hat{z}_0)$ is then backpropagated through the whole chain. This turns the sampling procedure itself into the training objective, so the model is optimized for the exact computation performed at inference, and it is what makes perceptual and adversarial losses attachable to the final image.

What would settle it

Train the identical end-to-end objective from random initialization rather than from a pretrained model; if the loss fails to decrease or the FID does not beat stepwise training, the end-to-end objective alone is not sufficient and the pretrained start is the load-bearing ingredient. A second check is to lengthen the unrolled chain to 10 or more steps and measure whether gradient norms and output quality remain stable.

Watch

Extended reading notes

Core claim

The central claim is that the end-to-end reconstruction loss $L_{\mathrm{recon}}(\theta) = \mathbb{E}[d(z_0, \hat{z}_0)]$, evaluated on the final predicted latent $\hat{z}_0$ of a fixed, fully unrolled 3–4 step denoising chain, is a sufficient and better-aligned training signal for diffusion models than the usual stepwise noise-prediction loss. The authors argue that because the loss is placed on the final output, the model learns to manage cumulative errors across steps, the training computation graph matches the sampling computation graph, and the terminal latent need not be exactly Gaussian because training is formulated as a direct noise-to-data transformation. They report that fine-tuning a pretrained text-to-image latent consistency model with this objective, using L1, L2, and LPIPS losses (and optionally an adversarial term), improves FID and CLIP scores on COCO30K and HW30K at 3–4 sampling steps, and that human evaluators prefer its outputs over the pretrained baseline.

Load-bearing premise

The method rests on the assumption that backpropagating the final reconstruction loss through the fixed 3–4 step sampling chain, starting from a pretrained model, produces stable and useful gradients that improve the generation distribution rather than destabilizing training.

Editorial extensions

If this is right

  • A diffusion model can be fine-tuned with a single final-output reconstruction loss instead of per-step noise prediction, and this alone improves FID and CLIP at 3–4 sampling steps.
  • Perceptual and adversarial losses become straightforward additions to the core objective, because the entire sampling chain is differentiable end to end.
  • The training graph and the inference graph become the same computation, so error accumulation across steps is optimized directly rather than left to chance.
  • The same end-to-end objective is compatible with different reconstruction metrics (L1, L2, LPIPS, or combinations), making the choice of visual similarity criterion a direct design knob.
  • Few-step generation can be improved by fine-tuning an already-trained few-step consistency model, without adding distillation-specific losses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported gains may depend heavily on the pretrained initialization: the paper includes no training-from-scratch experiment, so the end-to-end objective alone has not been shown to train a diffusion model stably.
  • The ablation shows the adversarial term actually degrades FID and CLIP slightly while adding high-frequency detail, so the advertised benefit is a qualitative realism trade-off, not a uniform improvement.
  • A natural testable extension is to monitor gradient norms through the unrolled chain; if gradients vanish or explode beyond the first few steps, the fixed short chain is what keeps the method stable, not the objective itself.
  • The information-leakage claim is conceptual: because training starts from freshly sampled Gaussian noise and only the final output is scored, the model never needs the forward noising process to reach a perfectly Gaussian terminal state.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes E2ED^2, a fine-tuning procedure for the pretrained PixArt-delta model in which the full few-step LCM sampling chain is unrolled and a reconstruction loss d(z0, z0_hat) is applied directly to the final predicted latent, optionally combined with LPIPS and adversarial losses. The authors claim that this end-to-end objective removes the training-sampling discrepancy and the information leakage of the forward noising process, and that it enables perceptual and GAN losses. Experiments on COCO30K and HW30K report improved FID and CLIP scores relative to PixArt-delta at 3 and 4 sampling steps, with ablations over L1/L2/LPIPS/GAN combinations and a human preference study.

Significance. If the central claim were valid, unrolled fine-tuning with a simple reconstruction loss would be a practically valuable and easy-to-reproduce recipe for few-step text-to-image generation. The paper has several strengths: it evaluates on external benchmarks (COCO30K, HW30K), reports ablations over loss terms, includes a careful human evaluation protocol, and explicitly acknowledges in Appendix A.1 that training from random initialization has not been explored. However, the main theoretical claim is undermined by a fundamental degeneracy in Eq. (7), and the paper's own Tables 1 and 2 contradict several headline statements. The empirical observation that unrolled fine-tuning of PixArt-delta improves few-step metrics may be of interest, but the paper does not provide a sound account of why it works, and the proposed objective as written does not learn a noise-to-data mapping.

major comments (4)
  1. [§3.2, Eq. (7), Algorithm 1] The reconstruction objective in Eq. (7), L_recon(θ) = E_{q(z_T,z_0)}[d(z_0, z0_hat)], is degenerate as a generative objective because z_T is sampled independently of z_0 (Algorithm 1, line 6). For L2 distance, the Bayes-optimal predictor satisfies G_θ(z_T,c) = E[z_0 | c] for every z_T; for L1 it is the conditional median. Thus the objective rewards mapping every noise input to the same per-prompt average latent, not mapping noise to the data distribution. The claim that the method "directly transforms pure Gaussian noise to the target data distribution" (Abstract, §1, §3.2) is therefore not supported by Eq. (7). The model avoids visible collapse only because it is initialized from pretrained PixArt-delta and fine-tuned with a small learning rate for 24k steps, and Appendix A.1 explicitly states that training from random initialization has not been explored. The only term in Eq. (8) that encourages distribution matching is the GAN loss, and Table 2 shows that adding it worsens both FID and CLIP, so no component in the proposed objective counteracts the degeneracy.
  2. [§4.2.1, Table 1] The text claims the method "surpasses larger models like SDXL in both image quality and alignment." Table 1 contradicts this: E2ED2 achieves COCO FID 25.27 versus SDXL's 14.28, and HW FID 9.76 versus SDXL's 7.96, so SDXL is substantially better on both FID measures. The CLIP score is higher (32.76 versus 31.68), but the image-quality claim as stated is false. The comparison should be reworded to report the actual trade-off between FID and CLIP, and the "state-of-the-art" claim in the abstract and §4.2.1 should be calibrated to the numbers in Table 1.
  3. [§4.2.3, Table 2] The ablation in Table 2 shows that the best configuration is L2+LPIPS (COCO FID 25.27, COCO CLIP 32.76) and that adding GAN loss degrades both metrics (COCO FID 25.74, COCO CLIP 31.75). This directly contradicts the paper's repeated claims that the framework "seamlessly integrates" GAN losses and that GAN loss improves image quality and text-image alignment. The discussion admits a "trade-off," but the quantitative evidence is that the GAN term hurts both headline metrics, and the qualitative claim about high-frequency detail is not supported by any perceptual metric. This undermines the second stated contribution and the attribution of the main result to the GAN component.
  4. [§3.2 and Appendix C.1] Algorithm 1 is underspecified at a load-bearing point: it loops t=T to 1 without stating how many unrolled steps are used during training. If T=1000, the procedure is computationally implausible and inconsistent with the 3-4 step evaluation; if T is small, the paper should state the exact unrolled sampling schedule and analyze the behavior of gradients through the unrolled chain. No gradient-stability or training-curve analysis is provided. In addition, the information-leakage claim based on D_KL(q(x_T)||N(0,I)) > 0 is not measured or addressed: starting training from z_T ~ N(0,I) changes the training input distribution but does not alter the mutual information I(x_T;x_0) of the diffusion forward process, and the paper provides no experiment showing that leakage is actually reduced.
minor comments (6)
  1. [§3.2] The reference to "Algorithm ??" should be "Algorithm 1."
  2. [Figure 1] The caption and diagram contain garbled placeholder symbols such as "??," "?," and an unlabeled formula; these need to be cleaned up before submission.
  3. [§4.1.1, References] Reference [3] is cited for the COYO dataset, but the listed paper by Carlini et al. concerns poisoning web-scale datasets and does not appear to be the COYO dataset source; this citation should be corrected.
  4. [Table 1] The abbreviation "NFE" is not defined in the table caption; it should be spelled out as the number of function evaluations or sampling steps.
  5. [Abstract] The phrase "even with fewer sampling steps (less than 4)" is imprecise because the experiments use exactly 3 and 4 steps; the wording should be adjusted accordingly.
  6. [Appendix E] The human evaluation is reported only in the appendix; the main text should at least reference it when discussing qualitative and perceptual improvements.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported results are external benchmark evaluations after fine-tuning a pretrained diffusion model, and no equation or prediction reduces to its own inputs by construction.

full rationale

The paper does not claim a formal derivation from first principles; its central quantitative claims are FID and CLIP scores on COCO30K and HW30K for a model fine-tuned from PixArt-delta on a separate 120k internal dataset. The reconstruction objective in Eq. (7), L_recon(theta)=E[d(z0,z0_hat)], is a training loss, not a prediction derived from fitted parameters; the benchmarks are not fitted by the method. The single self-citation, [41], appears only in a list of text-to-image applications and is not load-bearing. The narrative claim that end-to-end training 'eliminates information leakage' and 'directly transforms pure Gaussian noise to the target data distribution' is rhetorical framing, and the paper itself acknowledges in Appendix A.1 that training from random initialization was not explored. The ablation in Table 2 also shows that adding the GAN loss degrades COCO FID and CLIP, which undercuts the advertised benefit of incorporating adversarial losses, but this is an empirical inconsistency rather than a circularity. No equation in the paper is equivalent to its output by definition, no fitted parameter is renamed as a prediction, and no load-bearing premise depends on an unverified self-citation. Accordingly, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central method rests on the pretrained PixArt-delta prior and on the assumption that gradient-based fine-tuning of the full sampling chain is stable. No new physical or mathematical entities are introduced. The paper's information-leakage framing is an ad hoc narrative claim that is never quantitatively verified.

free parameters (4)
  • Adversarial loss weight lambda_GAN = 0.01
    Set in Appendix C.2; ablation shows adding it worsens FID and CLIP, so the chosen value does not improve the central metrics.
  • Hybrid loss weights lambda_L1, lambda_L2, lambda_LPIPS = not reported
    Defined in Section 3.2 but their values are never given, so the exact objective is not reproducible from the paper.
  • EMA decay = 0.95
    Hyperparameter in Section 4.1.4; chosen by hand to stabilize training.
  • Learning rate = 1e-6
    Fixed LR in Section 4.1.4; a hyperparameter of the fine-tuning procedure.
assumptions (5)
  • standard math The DDPM forward and reverse processes (Eqs. 1-2) and the noise-prediction objective (Eq. 3) accurately describe the pretrained model being fine-tuned.
    Background formalism adopted from DDPM [10] and LDM [32]; treated as valid and used to motivate the method.
  • domain assumption Pretrained PixArt-delta provides a strong prior, and fine-tuning it with the end-to-end loss preserves and improves few-step generation without catastrophic forgetting.
    The method initializes from PixArt-delta (Section 4.1.4) and only fine-tunes; the authors acknowledge in Appendix A.1 they have not explored training from random initialization.
  • domain assumption The fixed LCM scheduler (linear schedule, beta_0=0.0001, beta_T=0.02, T=1000, CFG=0) is an appropriate sampling framework for the fine-tuned model.
    Appendix C.1 fixes these parameters; the paper does not analyze sensitivity to the scheduler.
  • domain assumption Backpropagation through 3-4 unrolled sampling steps yields stable and informative gradients.
    Algorithm 1 differentiates through the whole loop; no gradient analysis or convergence theory is provided, and EMA (decay 0.95) is used as a stability insurance.
  • ad hoc to paper The information-leakage measure DKL(q(x_T)||N(0,I)) > 0 is a meaningful problem that is removed by starting training from pure noise.
    Introduced in Section 1 as motivation; no measurement before/after is provided, making it a narrative assumption rather than a tested quantity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models." pith.science (2026). https://pith.science/paper/CKTMG2OX

@misc{pith2026241221044,
  author       = {Pith},
  title        = {Pith review of: E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CKTMG2OX}},
  note         = {Machine review of arXiv:2412.21044}
}
read the original abstract

Diffusion models have established themselves as the de facto primary paradigm in visual generative modeling, revolutionizing the field through remarkable success across various diverse applications ranging from high-quality image synthesis to temporal aware video generation. Despite these advancements, three fundamental limitations persist, including 1) discrepancy between training and inference processes, 2) progressive information leakage throughout the noise corruption procedures, and 3) inherent constraints preventing effective integration of modern optimization criteria like perceptual and adversarial loss. To mitigate these critical challenges, we in this paper present a novel end-to-end learning paradigm that establishes direct optimization from the final generated samples to initial noises. Our proposed End-to-End Differentiable Diffusion, dubbed E2ED^2, introduces several key improvements: it eliminates the sequential training-sampling mismatch and intermediate information leakage via conceptualizing training as a direct transformation from isotropic Gaussian noise to the target data distribution. Additionally, such training framework enables seamless incorporation of adversarial and perceptual losses into the core optimization objective. Comprehensive evaluation across standard benchmarks including COCO30K and HW30K reveals that our method achieves substantial performance gains in terms of Fr\'echet Inception Distance (FID) and CLIP score, even with fewer sampling steps (less than 4). Our findings highlight that the end-to-end mechanism might pave the way for more robust and efficient solutions, \emph{i.e.,} combining diffusion stability with GAN-like discriminative optimization in an end-to-end manner.

Figures

Figures reproduced from arXiv: 2412.21044 by the authors.

Figure 1
Figure 1. Comparison of two training methods for diffusion models. (a) illustrates the single-step training method, where the model is trained to predict noise in a single denoising step for randomly sampled time steps. This approach introduces a training-sampling gap, as the training focuses on single-step denoising, whereas testing requires iterative multi-step denoising. Furthermore, the forward noising process can result … view at source ↗
Figure 2
Figure 2. Qualitative comparison of synthesized images across different methods on various subjects (human portraits, objects) and styles. Our method demonstrates superior performance in terms of image quality, aesthetic appeal, and text-image alignment. The generated images show finer details, richer textures, and better adherence to the input prompts compared to SOTA methods. This highlights the effectiveness of our end-to-… view at source ↗
Figure 3
Figure 3. Qualitative comparison of generated results across different loss configurations. Each column represents a specific loss setting: L1, L2, LPIPS, L2+LPIPS, and L2+LPIPS+GAN. The inclusion of LPIPS loss improves perceptual quality, while combining L2+LPIPS with GAN loss adds high-frequency de￾tails, such as fine hair strands in portraits and intricate petal tex￾tures in flowers. Although this introduces a slight trade… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of training loss curves between parameter [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: User Interface Demonstration: Our custom-designed user interface sequentially presents evaluators with pairs of images alongside the corresponding generation prompt. Additionally, we formulated three specific evaluation questions to comprehensively measure user prefere…
Figure 6
Figure 6. Figure 6: Human Evaluation Results. Our model consistently receives higher preference ratings from human evaluators, demon￾strating its superior capability in generating images with enhanced visual quality. quality, text-aligned images. 14 [PITH_FULL_IMAGE:figures/full_fig_p014…
Figure 7
Figure 7. Figure 7: Additional qualitative results demonstrating the effectiveness of our text-to-image generation method. The images showcase [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 23 canonical work pages

  1. [1]

    Analytic- dpm: an analytic estimate of the optimal reverse vari- ance in diffusion probabilistic models

    Fan Bao, Chongxuan Li, Jun Zhu, and Bo Zhang. Analytic- dpm: an analytic estimate of the optimal reverse vari- ance in diffusion probabilistic models. arXiv preprint arXiv:2201.06503, 2022. 3

    arXiv 2022

  2. [2]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 1

    work page 2023

  3. [3]

    Poisoning web-scale training datasets is practical

    Nicholas Carlini, Matthew Jagielski, Christopher A Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum An- derson, Andreas Terzis, Kurt Thomas, and Florian Tram `er. Poisoning web-scale training datasets is practical. In 2024 IEEE Symposium on Security and Privacy (SP) , pages 407–

    work page 2024

  4. [4]

    Pixart- {\delta}: Fast and controllable image generation with latent consis- tency models

    Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart- {\delta}: Fast and controllable image generation with latent consis- tency models. arXiv preprint arXiv:2401.05252, 2024. 2, 5, 6, 12, 13

    arXiv 2024

  5. [5]

    Scaling rectified flow transformers for high-resolution image synthesis, 2024

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis, 2024. URL https://arxiv. org/abs/2403.03206, 2. 1

    arXiv 2024

  6. [6]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 1, 2

    2020

  7. [7]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and Jos ´e Lezama. Photorealistic video generation with diffusion models. In European Conference on Computer Vision, pages 393–411. Springer, 2024. 1

    work page 2024

  8. [8]

    Cameractrl: Enabling camera control for text-to-video generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101, 2024. 1

    arXiv 2024

Show all 60 references
  1. [9]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 2

    2017

  2. [10]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 2, 3

    2020

  3. [11]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 1

    2022

  4. [12]

    Diffusion models for video prediction and infilling

    Tobias H ¨oppe, Arash Mehrjou, Stefan Bauer, Didrik Nielsen, and Andrea Dittadi. Diffusion models for video prediction and infilling. arXiv preprint arXiv:2206.07696, 2022. 1

    2022 arXiv

  5. [13]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 1

    2024

  6. [14]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 3

    2022

  7. [15]

    Analyzing and improving the training dynamics of diffusion models

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24174–24184, 2024. 1, 2

    2024

  8. [16]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 1, 2

    2013 arXiv

  9. [17]

    Photo- realistic single image super-resolution using a generative ad- versarial network

    Christian Ledig, Lucas Theis, Ferenc Husz´ar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo- realistic single image super-resolution using a generative ad- versarial network. In Proceedings of the IE...

  10. [18]

    Sdxl- lightning: Progressive adversarial diffusion distillation

    Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl- lightning: Progressive adversarial diffusion distillation. arXiv preprint arXiv:2402.13929, 2024. 3, 5, 6

    2024 arXiv

  11. [19]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  12. [20]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 5

    2024

  13. [21]

    Generic perceptual loss for modeling structured output de- pendencies

    Yifan Liu, Hao Chen, Yu Chen, Wei Yin, and Chunhua Shen. Generic perceptual loss for modeling structured output de- pendencies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5424– 5432, 2021. 2

    2021

  14. [22]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775–5787,

  15. [23]

    Latent consistency models: Synthesizing high- resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high- resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 3, 5, 6, 12

    2023 arXiv

  16. [24]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 1

    2024 arXiv

  17. [25]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 14297–14306, 2023. 3

    2023

  18. [26]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR,

  19. [27]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- 9 national conference on computer vision , pages 4195–4205,

  20. [28]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 5, 6

    2023 arXiv

  21. [29]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 1

    2022 arXiv

  22. [30]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  23. [31]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022. 1

    2022 arXiv

  24. [32]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3

    2022

  25. [33]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  26. [34]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512, 2022. 3

    2022 arXiv

  27. [35]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  28. [36]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021. 2

    2021

  29. [37]

    Improved tech- niques for training consistency models

    Yang Song and Prafulla Dhariwal. Improved tech- niques for training consistency models. arXiv preprint arXiv:2310.14189, 2023. 1, 3

    2023 arXiv

  30. [38]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023. 1, 3

    2023 arXiv

  31. [39]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 2

    2011 arXiv

  32. [40]

    Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent

    Xingwu Sun, Yanfeng Chen, Yiqing Huang, Ruobing Xie, Jiaqi Zhu, Kai Zhang, Shuaipeng Li, Zhen Yang, Jonny Han, Xiaobo Shu, et al. Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent. arXiv preprint arXiv:2411.02265, 2024. 1

    2024 arXiv

  33. [41]

    An em- pirical study and analysis of text-to-image generation us- ing large language model-powered textual representation

    Zhiyu Tan, Mengping Yang, Luozheng Qin, Hao Yang, Ye Qian, Qiang Zhou, Cheng Zhang, and Hao Li. An em- pirical study and analysis of text-to-image generation us- ing large language model-powered textual representation. In European Conference on Computer Vision, pages 472–489. ...

    2024

  34. [42]

    Esrgan: En- hanced super-resolution generative adversarial networks

    Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: En- hanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops, pages 0–0, 2018. 2

    2018

  35. [43]

    Tack- ling the generative learning trilemma with denoising diffu- sion gans

    Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tack- ling the generative learning trilemma with denoising diffu- sion gans. arXiv preprint arXiv:2112.07804, 2021. 3

    2021 arXiv

  36. [44]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 1

    2024 arXiv

  37. [45]

    Cami2v: Camera-controlled image-to-video dif- fusion model

    Guangcong Zheng, Teng Li, Rui Jiang, Yehao Lu, Tao Wu, and Xi Li. Cami2v: Camera-controlled image-to-video dif- fusion model. arXiv preprint arXiv:2410.15957, 2024. 1

    2024 arXiv

  38. [46]

    Fast ode-based sampling for diffusion models in around 5 steps

    Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 7777–7786,

  39. [48]

    Throughout the evaluation process, you will systematically compare 300 pairs of images using the provided user interface

  40. [49]

    • A pair of images generated by different models

    For each comparison, the interface will display: • A sequential number at the top-left corner to indicate your current progress. • A pair of images generated by different models. • A text prompt used for generating the images. • Six buttons with various functions to facilitate...

  41. [50]

    To indicate your preferred image, click the like button located directly below the image

  42. [51]

    If you have no clear preference between the two images, click the central red “X” button below the images

  43. [52]

    After selecting your preference or indicating no preference, the interface will automatically move to the next image pair

  44. [53]

    Navigation arrows located at the bottom-right corner of the interface allow you to revisit previously viewed pairs and revise your choices if nec- essary

  45. [54]

    Part II: Evaluation Criteria and Guidelines

    If you encounter any uncertainties or difficulties during the evaluation, please click theraise hand button at the bottom-right corner, and assistance will be promptly provided. Part II: Evaluation Criteria and Guidelines

  46. [55]

    Generally, your evaluation should reflect your personal preference

  47. [56]

    Faithfulness: Assess if the generated image appears plausible and realis- tically conforms to the laws of the real world

    If you find it challenging to decide based solely on personal preference, we suggest using the following criteria: Text-Image Alignment: Determine whether the generated image accu- rately aligns with the provided reference prompt. Faithfulness: Assess if the generated image ap...

  48. [57]

    Afterwards, proceed to assess all remaining image pairs sequentially

    Initially, carefully evaluate 30 image pairs to establish a consistent eval- uation standard. Afterwards, proceed to assess all remaining image pairs sequentially

  49. [58]

    Should you feel uncertain or confused at any stage, use the raise hand button at the bottom-right corner for assistance

  50. [59]

    Upon completing the evaluation of all 300 image pairs, submit your results by clicking the right arrow button

  51. [60]

    We sincerely appreciate your contribution to this research project

    After submitting your evaluation, your task is complete. We sincerely appreciate your contribution to this research project. compelling evidence underscores the effectiveness of our approach in generating images that align closely with hu- man preferences and exhibit superior ...

  52. [2024]

    Limitations and Future Works A.1

    2 10 A. Limitations and Future Works A.1. Limitations. Despite demonstrating substantial improvements in genera- tive performance, our proposed method has several limita- tions. Firstly, our current end-to-end training strategy pri- marily involves fine-tuning from pretrained ...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.