Pith. sign in

REVIEW 4 major objections 5 minor 73 references

Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that the uncertainty of a text-to-image model about a prompt can be measured by captioning the generated image with a vision-language model and comparing the caption to the prompt, and that this text-space score…

desk verdict First real attempt at prompt-space uncertainty for T2I, but the LVLM's access to the original prompt confounds the central result. read the letter →

arxiv 2412.03178 v1 pith:VZJD3I7C submitted 2024-12-04 cs.AI cs.CVcs.LG

Gianni Franchi , Dat Nguyen Trong , Nacim Belkhir , Guoxuan Xia , Andrea Pilzer This is my paper · ORCID
classification cs.AIcs.CVcs.LG
keywords uncertaintyquantificationtext-to-imagegenerationvision-languagemodelsaleatoricepistemicout-of-distributiondetectionprompt-spacesemanticsimilarity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes a new task: quantifying how uncertain a text-to-image model is about a prompt, and argues that the right place to measure that uncertainty is in semantic space rather than pixel space. It introduces PUNC, which generates an image from a prompt, uses a large vision-language model (LVLM) to caption that image, and scores similarity between the caption and the prompt. The paper claims that PUNC outperforms most adapted image-space methods on out-of-distribution, vague, corrupted, and adversarially altered prompts. It also claims that, because text similarity metrics carry precision and recall components, PUNC can separate aleatoric uncertainty (imprecise or corrupted prompts) from epistemic uncertainty (concepts the model does not know). If the claim holds, uncertainty scores for text-to-image systems can be obtained from a single generation and used for out-of-distribution detection, bias detection, deepfake monitoring, and copyright-relevant content detection.

What carries the argument

The central machinery is the prompt-caption similarity score. Given a prompt $c^*$, the text-to-image model draws one image $x \sim p_\theta(x|c^*)$; a large vision-language model turns the image into a caption $\hat{c} = f^{\mathrm{txt}}_\omega(c^*, f^{\mathrm{img}}_\omega(x))$; and the uncertainty score is the similarity $S(c^*,\hat{c})$ computed by ROUGE or BERTScore. Those text similarity metrics carry precision and recall at the token or embedding level, which is how PUNC claims to separate aleatoric uncertainty (extra concepts in the image lower precision) from epistemic uncertainty (missing concepts lower recall). This replaces the image-space similarity used by adapted baselines such as noising/reconstruction or two-sample ensembling.

What would settle it

Take a set of ambiguous prompts that human raters agree are ambiguous, plus crisp prompts matched for vocabulary; if PUNC's scores do not separate the two while human ratings do, the similarity proxy fails. A sharper check: hand-write captions for the same generated images without seeing the prompt, and see whether PUNC's score changes when the caption wording changes; if it does, the captioner's phrasing, not the generator's uncertainty, is driving the score.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that prompt-space semantic uncertainty can be captured by comparing the original prompt with an LVLM-generated caption of the output image, and that this text-space comparison outperforms image-space methods that noise, mask, or ensemble generations. Uncertainty is defined semantically: aleatoric uncertainty arises when a prompt allows multiple concepts (vagueness, typos, corruption), and epistemic uncertainty arises when the model lacks knowledge of a concept in the prompt (out-of-distribution concepts). PUNC computes $S(c^*, \hat{c}) = \mathrm{sim}(c^*, \hat{c})$ between the prompt and the caption, and uses the precision/recall asymmetry of ROUGE or BERTScore to attribute low recall to epistemic uncertainty and low precision to aleatoric uncertainty. The paper's experiments span four text-to-image models and report that PUNC outperforms most existing uncertainty techniques on average across the out-of-distribution, vague, adversarial, and corrupted prompt datasets.

Load-bearing premise

PUNC assumes that the similarity between the original prompt and a caption of the generated image, written by an AI model that describes images, truly reflects how unsure the image generator is; if the describer makes mistakes or uses different wording, those errors get counted as generator uncertainty, and the benchmark also assumes that prompts about rare subjects, vague wording, and corrupted text really do produce the two kinds of uncertainty they are meant to test.

Editorial extensions

If this is right

  • Single-generation scoring: PUNC needs only one image per prompt plus a caption, so it can score any pretrained text-to-image model without retraining or ensembling.
  • Precision/recall separation: using ROUGE recall to detect missing concepts and precision to detect extra concepts lets a practitioner tell whether a failure comes from an under-specified or corrupted prompt (aleatoric) or from an unfamiliar concept (epistemic).
  • Benchmark availability: the released prompt datasets covering normal, out-of-distribution, vague, adversarial, and corrupted prompts let other text-to-image models be scored on the same uncertainty benchmark.
  • Application signal: the uncertainty score can flag generated images containing named politicians or copyrighted characters, giving a concrete signal for deepfake-style and copyright-relevant content monitoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to replace the caption-similarity step with a direct LVLM answer to the question "does this image match the prompt?" and compare that score with PUNC, which would show whether the caption bottleneck is the limiting factor.
  • Because PUNC's score depends on the LVLM's wording, prompts whose meaning is faithfully preserved but phrased differently may be scored as uncertain; calibrating the similarity metric on paraphrases of the same prompt would test this sensitivity.
  • The same prompt-caption protocol could be applied to video or audio generation by captioning the output modality and comparing the caption with the conditioning text.
  • The paper's own limitations section concedes that the LVLM may inject its own biases or inaccuracies inherited from training data and that the analysis is English-only; a robust deployment would need captioner calibration and multilingual coverage.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces PUNC, a method for quantifying uncertainty of text-to-image (T2I) models with respect to the input prompt. PUNC generates an image from the prompt, uses a large vision-language model (LVLM) to caption that image, and then computes a text-space similarity (ROUCE or BERTScore) between the original prompt and the LVLM caption, with precision and recall variants intended to separate aleatoric and epistemic uncertainty. The authors construct prompt datasets (Normal, three OOD domains, Vague, Adversarial, and two Corrupted levels), adapt image-space baselines (DDPM-OOD, LMD, 2XDM) to the prompt-conditioned setting, and evaluate AUROC/AUPR/FPR95 across four T2I models. They also present applications to politician recognition, copyrighted characters, and bias detection. The central claim is that PUNC outperforms existing uncertainty estimation techniques.

Significance. If the central claim were established, PUNC would be a valuable contribution: it is a simple, computationally light method that leverages LVLMs to move uncertainty quantification from image space to semantic text space, and it is the first systematic study of prompt-space uncertainty for T2I models. The paper also provides a useful benchmark structure and explores interesting downstream applications. However, the current experimental evidence is weakened by a systematic confound in the PUNC protocol (the LVLM receives the original prompt) and by benchmark construction that may separate prompt styles rather than model uncertainty. These issues are fixable with additional controlled experiments, but they are load-bearing for the paper's main claim.

major comments (4)
  1. [Sec. 4.2] The PUNC protocol feeds the original prompt c* to the LVLM together with the generated image when producing the caption c-hat. Because modern LVLMs are instruction followers, c-hat can be anchored to c* even when the T2I output is poor, so the score S(c*, c-hat) may measure LVLM prompt-following rather than T2I uncertainty. This affects all benchmark families: corrupted prompts can be auto-corrected from the text, OOD prompts (which are LLaVA captions) can be echoed in domain vocabulary, and vague prompts can be elaborated from the class name. The large spread across LVLMs in Appendix C (Molmo vs. Llava vs. Llama) is consistent with this shortcut, and the perfect 100% AUROC on Vague in Table 3 is suspicious. The Limitations paragraph mentions LVLM biases in general but does not address this specific anchoring mechanism. Please run a controlled ablation in which the LVLM captions the image without receiving the prompt (e.g., a neutral instruction), and include a text-only baseline that computes the score from c* alone without the image. This is necessary to attribute the reported AUROC values to genuine T2I uncertainty.
  2. [Sec. 5] The benchmark labels are defined by prompt source and construction, not by independently verified T2I uncertainty. The Normal prompts are GPT-4 descriptions of ImageNet images, while the OOD prompts are LLaVA-Next captions of remote-sensing, texture, and microscopic images; these sets differ in vocabulary, sentence length, and captioner identity. Vague prompts are short templates, and Corrupted prompts are truncated or grammatically damaged Normal prompts. As a result, the AUROC values may reflect these stylistic differences rather than the model's uncertainty about the prompt. The paper does not provide an independent check that the constructed OOD prompts actually induce high epistemic uncertainty in the T2I models (e.g., human or LVLM ratings of image-prompt alignment). Please include a control experiment in which prompt style is matched across ID and OOD (same captioner, similar length and vocabulary), or otherwise demonstrate that the separation is driven by model uncertainty.
  3. [Table 2/3] The claim that 'PUNC outperforms state-of-the-art uncertainty estimation techniques' is not supported across all reported settings. On the Texture OOD dataset, PUNC's AUROC is about 54.95-57.01% with Molmo, which is substantially lower than DDPM-OOD mse (72.64%) and LMD mse (69.68%). Conversely, on the Vague dataset, PUNC achieves near-perfect AUROC (99.99-100%) and 0% FPR95, which is implausibly high for an uncertainty measure and likely reflects a trivial separation of short template prompts from long descriptive prompts. The paper should report global averages over all OOD datasets with per-dataset breakdowns, and temper the abstract and Section 5.1 claims to match the actual per-dataset results.
  4. [Tables 2-3] The evaluation reports point estimates of AUROC, AUPR, and FPR95 with no confidence intervals, no number of repetitions, and no significance tests. The paper also does not release code, the prompt dataset, or the exact LVLM instruction used in Step 2 at the time of writing, which hampers reproducibility and makes it difficult to assess whether the observed differences (often only a few AUROC points) are robust. Please provide the exact prompt template for the LVLM, release the dataset/code, and add error bars or significance tests for the main tables.
minor comments (5)
  1. [Tables 2-3] The notation 'PUNC BERT' is ambiguous: please state explicitly whether BERTScore precision or recall is used in each table, since the choice is not evident without context.
  2. [Appendix B] The LVLM prompt template used in Step 2 of PUNC is not specified; please provide the exact instruction given to the LVLM, as this is essential for reproducibility and for interpreting the anchoring effect discussed above.
  3. [Section 1] The claim of being 'the first' to quantify uncertainty in T2I generation appears several times; it would be more measured to say 'to our knowledge' and to discuss any concurrent or related work on uncertainty in generative text-to-image models more thoroughly.
  4. [Table 3] There are minor typos, including 'Corrup Lvl2' in Table 3 and 'choosen' in Appendix D; please proofread the manuscript.
  5. [Figure 1] The applications listed in Figure 1 (deepfake prevention, bias detection, copyright protection) are only covered in the appendix; consider adding a short pointer in the main text so readers know where these results are discussed.

Circularity Check

1 steps flagged · score 6.0 of 10

PUNC's uncertainty score is computed from a caption that is itself conditioned on the target prompt, making the reported superiority partially self-referential.

  1. self definitional [Section 4.2, Steps 2-3 (PUNC), page 5; also Appendix A Eqs. (6)-(7)]
    "Step 2: Image Interpretation via LVLM With the generated image x and the initial prompt c∗, the LVLM produces a new descriptive caption ˆc: ˆc = f txt ω (c∗, fimg ω (x)) Step 3: Uncertainty Score Calculation The core of PUNC lies in measuring the alignment between c and ˆc ... S(c∗, ˆc) = sim(c∗, ˆc)"

    The uncertainty score is a similarity between the original prompt c* and a caption that the LVLM generates while conditioning on that same c*. The prompt therefore appears on both sides of the comparison: it is the reference text and an input to the caption generator. A caption that parrots or lightly rephrases c* yields a high similarity regardless of what the T2I model actually drew, so the score measures LVLM prompt-following as much as image fidelity. The reported superiority over image-space baselines is thus not an independent test of T2I uncertainty; the central evaluation partly reduces to a tautological text-matching loop. The Limitations paragraph mentions generic LVLM biases but does not address this specific prompt-anchoring mechanism.

full rationale

The only load-bearing circular step is in PUNC's construction: the caption is generated from the image together with the very prompt whose uncertainty it is supposed to measure, and the score is then the similarity between that prompt and that caption. This makes the method's central prediction partially self-referential, since a prompt-echoing LVLM would score high similarity without needing to inspect the image. The paper's empirical comparisons, dataset construction, and applications contain independent content, and there is no fitted-parameter or self-citation circularity; however, the headline claim that PUNC outperforms state-of-the-art techniques is weakened by the unaddressed confound that c* is fed into the LVLM. The Limitations paragraph admits LVLM 'biases or inaccuracies' but does not acknowledge that the initial prompt is part of the LVLM input, which is the specific mechanism that creates the circularity. Score 6 reflects a central construction that partially reduces to its own input, while the method still retains some independent empirical content.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. Its load-bearing assumptions are about the reliability of LVLM captions and the construct validity of the benchmark datasets. The main post-hoc element is the selection of the text similarity metric (and precision/recall direction) separately for each uncertainty type based on the same test sets.

free parameters (1)
  • Similarity metric and direction per uncertainty type = ROUGE recall for epistemic, BERTScore precision for aleatoric
    The paper reports that 'the optimal metric is less clear for aleatoric uncertainty' (Section 5.1) and selects ROUGE recall for OOD detection and BERTScore precision for corrupted prompts based on the benchmark results, which is a post-hoc choice on the test sets rather than a prespecified rule.
assumptions (3)
  • domain assumption LVLMs accurately extract semantics from generated T2I images.
    Section 4.2 states 'Our hypothesis is that as LVLMs become increasingly adept at understanding complex relationships between text and images, they will allow us to better analyze and quantify uncertainty.' The entire method relies on this.
  • domain assumption Precision and recall of caption concepts approximate aleatoric and epistemic uncertainty respectively.
    Section 4.2 introduces an abstract example and states 'a lack of knowledge about concepts in the prompt will result in fewer concepts being preserved, reducing recall... a lack of specificity will result in additional concepts being generated, reducing precision.' This mapping is assumed without direct validation.
  • domain assumption The constructed OOD datasets (remote sensing, texture, microscopic) represent epistemic uncertainty for the tested T2I models.
    Section 5 and Appendix D argue these domains are underrepresented in LAION-5b, but the claim that the models are uncertain about them is not measured independently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation." pith.science (2026). https://pith.science/paper/VZJD3I7C

@misc{pith2026241203178,
  author       = {Pith},
  title        = {Pith review of: Towards Understanding and Quantifying Uncertainty for Text-to-Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VZJD3I7C}},
  note         = {Machine review of arXiv:2412.03178}
}
read the original abstract

Uncertainty quantification in text-to-image (T2I) generative models is crucial for understanding model behavior and improving output reliability. In this paper, we are the first to quantify and evaluate the uncertainty of T2I models with respect to the prompt. Alongside adapting existing approaches designed to measure uncertainty in the image space, we also introduce Prompt-based UNCertainty Estimation for T2I models (PUNC), a novel method leveraging Large Vision-Language Models (LVLMs) to better address uncertainties arising from the semantics of the prompt and generated images. PUNC utilizes a LVLM to caption a generated image, and then compares the caption with the original prompt in the more semantically meaningful text space. PUNC also enables the disentanglement of both aleatoric and epistemic uncertainties via precision and recall, which image-space approaches are unable to do. Extensive experiments demonstrate that PUNC outperforms state-of-the-art uncertainty estimation techniques across various settings. Uncertainty quantification in text-to-image generation models can be used on various applications including bias detection, copyright protection, and OOD detection. We also introduce a comprehensive dataset of text prompts and generation pairs to foster further research in uncertainty quantification for generative models. Our findings illustrate that PUNC not only achieves competitive performance but also enables novel applications in evaluating and improving the trustworthiness of text-to-image models.

Figures

Figures reproduced from arXiv: 2412.03178 by the authors.

Figure 1
Figure 1. Examples of Applications for Uncertainty Quantification in Text-to-Image Generation. Text-to-image generation models may exhibit uncertainty, and that need to be quantified since it can provide insights into the model’s training dataset, aiding in deepfake prevention, detecting model biases, and protecting copyrighted content from unauthorized generation. Abstract Uncertainty quantification in text-to-image (T2I) ge… view at source ↗
Figure 2
Figure 2. Diagram illustrating generation/image-space uncertainty [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Three generations from PixArt-Σ [7] illustrating uncer￾tainty with regards to prompt semantics. For the Normal image, we used an ImageNet-inspired prompt; for the corrupted image, additional corruption was applied to the prompt (e.g. fish was perturbed to fis) increasing the aleatoric uncertainty; and for the out-of-distribution (OOD) case, the model was prompted to gen￾erate an image of the Prime Minister of Japan … view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: Illustration showing the different baselines and PUNC. PUNC leverages a LVLM to describe generated images and assess similarity with the original prompt, providing a refined uncertainty score. In contrast, baseline methods employ traditional techniques such as noise in…
Figure 5
Figure 5. Figure 5: Illustration of gender bias in diffusion models with respect to job representation. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Illustration of racial bias in diffusion models with respect to job representation. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Example of a generated microscopic image with PixArt- [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Example of a generated microscopic image with SDXL and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Example of a generated microscopic image with SDXS and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Example of a generated microscopic image with SDv1.5 and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Example of a generated Remote Sensing image with PixArt- [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Example of a generated Remote Sensing image with SDXL and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Example of a generated Remote Sensing image with SDXS and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Example of a generated Remote Sensing image with SD1.5 and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: Example of a generated Texture image with PixArt- [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Example of a generated Texture image with SDXL and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p022_16.png]
Figure 17
Figure 17. Figure 17: Example of a generated Texture image with SDXS and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]
Figure 18
Figure 18. Figure 18: Example of a generated Texture image with SD1.5 and the caption generated [PITH_FULL_IMAGE:figures/full_fig_p023_18.png]
Figure 19
Figure 19. Figure 19: Normal images with different models. The same prompts are used to generate the images with the four models 13 [PITH_FULL_IMAGE:figures/full_fig_p024_19.png]
Figure 20
Figure 20. Figure 20: Microscopic images with different models. The same prompts are used to generate the images with the four models 14 [PITH_FULL_IMAGE:figures/full_fig_p025_20.png]
Figure 21
Figure 21. Figure 21: Texture images with different models. The same prompts are used to generate the images with the four models 15 [PITH_FULL_IMAGE:figures/full_fig_p026_21.png]
Figure 22
Figure 22. Figure 22: Remote Sensing images with different models. The same prompts are used to generate the images with the four models 16 [PITH_FULL_IMAGE:figures/full_fig_p027_22.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 61 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Cheb- otar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 2

    arXiv 2022

  3. [3]

    Ai- generated faces free from racial and gender stereotypes

    Nouar AlDahoul, Talal Rahwan, and Yasir Zaki. Ai- generated faces free from racial and gender stereotypes. arXiv preprint arXiv:2402.01002, 2024. 4

    arXiv 2024

  4. [4]

    Detecting Out- Of-Distribution Earth Observation Images with Diffusion Models

    Georges Le Bellier and Nicolas Audebert. Detecting Out- Of-Distribution Earth Observation Images with Diffusion Models . In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 481–491, Los Alamitos, CA, USA, 2024. IEEE Computer Society. 2, 3

    work page 2024

  5. [5]

    Shedding light on large generative networks: Estimating epistemic uncertainty in diffusion models

    Lucas Berry, Axel Brando, and David Meger. Shedding light on large generative networks: Estimating epistemic uncertainty in diffusion models. In The 40th Conference on Uncertainty in Artificial Intelligence, 2024. 2, 3, 5

    work page 2024

  6. [6]

    Hyper-diffusion: Estimating epistemic and aleatoric uncertainty with a single model

    Matthew A Chan, Maria J Molina, and Christopher A Metzler. Hyper-diffusion: Estimating epistemic and aleatoric uncertainty with a single model. arXiv preprint arXiv:2402.03478, 2024. 3

    arXiv 2024

  7. [7]

    Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. arXiv preprint arXiv:2403.04692, 2024. 3, 6, 1

    arXiv 2024

  8. [8]

    Pixart-$ \alpha$: Fast train- ing of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng YU, Chongjian GE, Lewei Yao, Enze Xie, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-$ \alpha$: Fast train- ing of diffusion transformer for photorealistic text-to-image synthesis. In The Twelfth International Conference on Learning Representations, 2024. 2, 4

    work page 2024

Show all 73 references
  1. [9]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, , and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014. 2

    2014

  2. [10]

    Addressing failure prediction by learning model confidence

    Charles Corbi `ere, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick P ´erez. Addressing failure prediction by learning model confidence. Advances in Neural Information Processing Systems, 32, 2019. 2

    2019

  3. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 6, 7, 2

    2009

  4. [13]

    Gpt4image: Can large pre-trained models help vision models on perception tasks? arXiv e-prints , pages arXiv–2306, 2023

    Ning Ding, Yehui Tang, Zhongqian Fu, Chao Xu, Kai Han, and Yunhe Wang. Gpt4image: Can large pre-trained models help vision models on perception tasks? arXiv e-prints , pages arXiv–2306, 2023. 6, 7, 2

    2023

  5. [14]

    Diffusion-based probabilis- tic uncertainty estimation for active domain adaptation

    Zhekai Du and Jingjing Li. Diffusion-based probabilis- tic uncertainty estimation for active domain adaptation. Advances in Neural Information Processing Systems , 36: 17129–17155, 2023. 3

    2023

  6. [15]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  7. [16]

    Masksembles for uncertainty estimation

    Nikita Durasov, Timur Bagautdinov, Pierre Baque, and Pascal Fua. Masksembles for uncertainty estimation. In CVPR, 2021. 2

    2021

  8. [17]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 12873–12883,

  9. [18]

    Scaling rec- tified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rec- tified flow transformers for high-resolution image syn...

    2024

  10. [19]

    TRADI: Tracking deep neural network weight distributions

    Gianni Franchi, Andrei Bursuc, Emanuel Aldea, S ´everine Dubuisson, and Isabelle Bloch. TRADI: Tracking deep neural network weight distributions. In ECCV, 2020. 2

    2020

  11. [20]

    Latent discriminant deterministic uncertainty

    Gianni Franchi, Xuanlong Yu, Andrei Bursuc, Emanuel Aldea, Severine Dubuisson, and David Filliat. Latent discriminant deterministic uncertainty. In Computer Vision – ECCV 2022, pages 243–260, Cham, 2022. Springer Nature Switzerland. 2, 3

    2022

  12. [21]

    Uncertainty in Deep Learning

    Yarin Gal. Uncertainty in Deep Learning . PhD thesis, University of Cambridge, 2016. 2, 3

    2016

  13. [22]

    Selective classification for deep neural networks

    Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. In Proceedings of the 31st International Conference on Neural Information Processing Systems, page 4885–4894, Red Hook, NY , USA, 2017. Curran Associates Inc. 3, 4

    2017

  14. [23]

    Denoising diffusion models for out-of-distribution detection

    Mark S Graham, Walter HL Pinaya, Petru-Daniel Tudosiu, Parashkev Nachev, Sebastien Ourselin, and Jorge Cardoso. Denoising diffusion models for out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2948–2957, ...

    2023

  15. [24]

    A baseline for de- tecting misclassified and out-of-distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for de- tecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations, 2017. 3, 4

    2017

  16. [25]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 4 9

    2020

  17. [26]

    Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods

    Eyke H ¨ullermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning, 110(3), 2021. 2, 3

    2021

  18. [27]

    What are bayesian neural network posteriors really like? In ICML, 2021

    Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Gordon Wilson. What are bayesian neural network posteriors really like? In ICML, 2021. 2

    2021

  19. [28]

    Highly accurate protein structure prediction with alphafold

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvu- nakool, Russ Bates, Augustin ˇZ´ıdek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583–589, 2021. 2

    2021

  20. [29]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Sys- tems, pages 26565–26577. Curran Associates, Inc., 2022. 4

    2022

  21. [30]

    What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017

    Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017. 2

    2017

  22. [31]

    Variational diffusion models

    Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems, 2021. 4

    2021

  23. [32]

    Semantic uncertainty: Linguistic invariances for uncertainty estima- tion in natural language generation

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estima- tion in natural language generation. In The Eleventh Interna- tional Conference on Learning Representations, 2023. 3, 5

    2023

  24. [33]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In NeurIPS, 2017. 2, 3

    2017

  25. [34]

    Learning skillful medium-range global weather forecasting

    Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416–1421, 2023. 2

    2023

  26. [35]

    Packed-ensembles for efficient uncertainty estimation

    Olivier Laurent, Adrien Lafage, Enzo Tartaglione, Ge- offrey Daniel, Jean-Marc Martinez, Andrei Bursuc, and Gianni Franchi. Packed-ensembles for efficient uncertainty estimation. In ICLR, 2023. 2, 3

    2023

  27. [36]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024. 6, 1, 2

    2024 arXiv

  28. [37]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004. 6, 1

    2004

  29. [38]

    Detecting the unexpected via image resynthesis

    Krzysztof Lis, Krishna Nakka, Pascal Fua, and Mathieu Salzmann. Detecting the unexpected via image resynthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2152–2161, 2019. 3

    2019

  30. [39]

    Unsupervised out-of-distribution detection with diffusion inpainting

    Zhenzhen Liu, Jin Peng Zhou, Yufan Wang, and Kilian Q Weinberger. Unsupervised out-of-distribution detection with diffusion inpainting. In International Conference on Ma- chine Learning, pages 22528–22538. PMLR, 2023. 2, 3, 4, 6

    2023

  31. [40]

    Dpm-solver: A fast ode solver for diffu- sion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan LI, and Jun Zhu. Dpm-solver: A fast ode solver for diffu- sion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems , pages 5775–5787. Curran Associates, Inc., 2022. 4

    2022

  32. [41]

    Predictive uncertainty es- timation via prior networks

    Andrey Malinin and Mark Gales. Predictive uncertainty es- timation via prior networks. In Advances in Neural Informa- tion Processing Systems. Curran Associates, Inc., 2018. 3

    2018

  33. [42]

    Mcmc using hamiltonian dynamics

    Radford M Neal et al. Mcmc using hamiltonian dynamics. Handbook of markov chain monte carlo, 2011. 2

    2011

  34. [43]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318,

  35. [44]

    Nuclei segmentation in micro- scope cell images

    Gangadhar Payyavula. Nuclei segmentation in micro- scope cell images. https : / / www . kaggle . com / datasets/gangadhar/nuclei- segmentation- in - microscope - cell - images, 2018. Accessed: 2024-09-01. 7, 2

    2018

  36. [45]

    Understand- ing softmax confidence and uncertainty, 2021

    Tim Pearce, Alexandra Brintrup, and Jun Zhu. Understand- ing softmax confidence and uncertainty, 2021. 3

    2021

  37. [46]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  38. [47]

    Bigearthnet

    Jeremy Pinto. Bigearthnet. https://github.com/ jerpint/bigearthnet, 2022. Accessed: 2024-09-01. 7, 2

    2022

  39. [48]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations ,

  40. [49]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In Proceedings of the 38th International Conference on Machine Learning , pages 8821–8831. PMLR, 2021

    2021

  41. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 4, 6, 1

    2022

  42. [51]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Infor...

    2022

  43. [52]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021. 4

    2021

  44. [53]

    Score- based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score- based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. 4

    2021

  45. [54]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2409.17146, 2024. 4 10

    2024 arXiv

  46. [55]

    Road anomaly detection by partial image reconstruction with segmentation coupling

    Tomas V ojir, Tom ´aˇs ˇSipka, Rahaf Aljundi, Nikolay Chumerin, Daniel Olmeda Reino, and Jiri Matas. Road anomaly detection by partial image reconstruction with segmentation coupling. In Proceedings of the IEEE/CVF international conference on computer vision , pages 15651– 156...

    2021

  47. [56]

    Bayesian learning via stochastic gradient langevin dynamics

    Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In ICML, 2011. 2

    2011

  48. [57]

    BatchEnsemble: an alternative approach to efficient ensemble and lifelong learning

    Yeming Wen, Dustin Tran, and Jimmy Ba. BatchEnsemble: an alternative approach to efficient ensemble and lifelong learning. In ICLR, 2019. 2

    2019

  49. [58]

    Augmenting softmax information for selective classification with out-of- distribution data

    Guoxuan Xia and Christos-Savvas Bouganis. Augmenting softmax information for selective classification with out-of- distribution data. In Proceedings of the Asian Conference on Computer Vision (ACCV), pages 1995–2012, 2022. 3

    1995

  50. [59]

    On the usefulness of deep ensemble diversity for out-of-distribution detection, 2022

    Guoxuan Xia and Christos-Savvas Bouganis. On the usefulness of deep ensemble diversity for out-of-distribution detection, 2022. 3

    2022

  51. [60]

    Window- based early-exit cascades for uncertainty estimation: When deep ensembles are more efficient than single models

    Guoxuan Xia and Christos-Savvas Bouganis. Window- based early-exit cascades for uncertainty estimation: When deep ensembles are more efficient than single models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 17368–17380, 2023. 2, 3

    2023

  52. [61]

    Score Normalization for a Faster Diffusion Exponential Integrator Sampler

    Guoxuan Xia, Duolikun Danier, Ayan Das, Stathi Fotiadis, Farhang Nabiei, Ushnish Sengupta, and Alberto Bernacchia. Score Normalization for a Faster Diffusion Exponential Integrator Sampler. In NeurIPS 2023 Workshop on Diffusion Models, 2023. 4

    2023

  53. [62]

    Towards understanding why label smooth- ing degrades selective classification and how to fix it, 2024

    Guoxuan Xia, Olivier Laurent, Gianni Franchi, and Christos- Savvas Bouganis. Towards understanding why label smooth- ing degrades selective classification and how to fix it, 2024. 3

    2024

  54. [63]

    Synthesize then compare: Detecting failures and anomalies for semantic segmentation

    Yingda Xia, Yi Zhang, Fengze Liu, Wei Shen, and Alan L Yuille. Synthesize then compare: Detecting failures and anomalies for semantic segmentation. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 145–161. ...

    2020

  55. [64]

    Jaakkola

    Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi S. Jaakkola. Restart sampling for improving generative processes. In Thirty-seventh Confer- ence on Neural Information Processing Systems, 2023. 4

    2023

  56. [65]

    Generalized out-of-distribution detection: A survey

    Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. International Journal of Computer Vision, 2024. 3

    2024

  57. [66]

    Sdxs: Real-time one-step latent diffusion models with image conditions

    Xuanwu Yin Yuda Song, Zehao Sun. Sdxs: Real-time one-step latent diffusion models with image conditions. arxiv, 2024. 6, 1

    2024

  58. [67]

    Fast sampling of dif- fusion models with exponential integrator

    Qinsheng Zhang and Yongxin Chen. Fast sampling of dif- fusion models with exponential integrator. In The Eleventh International Conference on Learning Representations ,

  59. [68]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 586–595, 2018. 5

    2018

  60. [69]

    Bertscore: Evaluating text genera- tion with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi. Bertscore: Evaluating text genera- tion with bert. arXiv preprint arXiv:1904.09675, 2019. 6, 1

    1904 arXiv

  61. [70]

    To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images

    Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yihua Zhang, Jiancheng Liu, Ke Ding, and Sijia Liu. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images ... for now, 2024. 7, 2 11 Towards Understanding and Quantifying Uncerta...

    2024

  62. [71]

    These prompts create an ambiguous context, simulating scenarios where the input information is too sparse for the model to fully comprehend

    Vague: This dataset contains 2,000 prompts with deliberately vague descriptions, structured to provide minimal context, such as ”An image of ***” or ”A pic- ture of ***”, where ”***” is replaced by the ImageNet class name. These prompts create an ambiguous context, simulating ...

  63. [72]

    Adversarial: This dataset contains 1,000 prompts altered from the Normal dataset using UnlearnDiffAtk [70], a gradient-based adversarial attack method optimizing adversarial prompts within the diffusion process

  64. [73]

    yes” or “no

    Corrupted: To simulate real-world scenarios with input noise, we created prompts with grammatical errors and word omissions. Using LLAMA-3-2, we generated captions with varying levels of corruption: (1) Level 1: We introduced grammatical mistakes and spelling errors to the pro...

  65. [74]

    Notably, while PixArt performed less effectively in the earlier experiments with politicians, it emerges as one of the top performers in this context, successfully generating recognizable representations of copyrighted characters. This contrast suggests differences in the trai...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.