Pith. sign in

REVIEW 3 major objections 5 minor 34 references

Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Grounding a vision-language model's answers back to the input image, and combining that grounding confidence with self-consistency, calibrates its uncertainty far more effectively than temperature scaling alone.

desk verdict Grounding-based calibration is a plausible and useful idea, but the headline claim is not yet established because the comparison baseline has one fewer fitted parameter. read the letter →

arxiv 2505.03788 v1 pith:PBNW7KGW submitted 2025-04-30 cs.CL cs.AIcs.CV

Trilok Padhi , Ramneet Kaur , Adam D. Cobb , Manoj Acharya , Anirban Roy , Colin Samplawski , Brian Matejek , Alexander M. Berenbeim
show 2 more authors
Nathaniel D. Bastian Susmit Jha
This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords uncertaintyquantificationmulti-modalLLMcalibrationgroundingself-consistencyexpectederrorvision-languagemodeltemperaturescaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that self-consistency-based uncertainty scores for multi-modal LLMs, such as semantic entropy, are poorly calibrated because a model can be consistently wrong yet report high confidence. It proposes fixing this by grounding each generated response to the input image with a separate grounding model, then combining the grounding confidence with the self-consistency confidence using a two-parameter formula. The observed calibration error (ECE) drops substantially, for example from 0.108 to 0.029 on VQA with a large vision-language grounding model, and from 0.376 to 0.008 on a medical QA dataset. A sympathetic reader would care because the method is training-free and operates at inference time, potentially making LLM confidence trustworthy in safety-critical settings like medical question answering.

What carries the argument

The load-bearing object is the multiplicative confidence formula $\mathrm{Conf} = \mathrm{Conf}_{\mathrm{baseline}} \times \mathrm{Conf}_{\mathrm{GM}}^{1/T} + C$ (Eq. 2). The grounding model's confidence $\mathrm{Conf}_{\mathrm{GM}}$ is the average over the sampled responses of how well each response is grounded to the image, measured by a bounding-box segmentation model, a text-image similarity model, or a foundation-model verdict of 'Yes'/'No'/'Not sure'. The exponent $1/T$ temperature-scales the grounding confidence to account for the grounding model's own uncertainty, and the constant $C$ offsets the shrinkage caused by multiplying two values in $[0,1]$; both $T$ and $C$ are fit on a validation set. This formula carries the argument because it converts cross-modal evidence into a calibrated confidence score that is then evaluated by expected calibration error.

What would settle it

Fit a two-parameter recalibration of the baseline confidence alone, such as a linear mapping $a\cdot\mathrm{Conf}_{\mathrm{baseline}} + b$, to the same validation sets with the same number of free parameters as Eq. (2); if its ECE on VQA and Slake matches or beats the grounding-based scores, the claim that cross-modal grounding improves calibration would be unsupported.

Watch

Extended reading notes

Core claim

The central discovery is that cross-modal consistency provides a corrective signal that self-consistency alone lacks: if a textual answer cannot be grounded in the image, the model is likely wrong even when multiple sampled answers agree. The paper operationalizes this by multiplying the baseline self-consistency confidence $\mathrm{Conf}_{\mathrm{baseline}}$ with a temperature-scaled grounding-model confidence $\mathrm{Conf}_{\mathrm{GM}}^{1/T}$, plus a small constant $C$, with both $T$ and $C$ fit on a validation set. Across four self-consistency baselines (lexical similarity, predictive entropy, semantic entropy, and number of semantic sets) and several grounding models (segmentation-based, CLIP-style similarity, and visual foundation models), the proposed confidence score yields lower expected calibration error than the same baselines recalibrated by temperature scaling alone. The paper concludes that calibrating confidence scores with external grounding is substantially more effective than calibrating them via temperature scaling.

Load-bearing premise

The paper assumes the observed ECE reduction is caused by the grounding signal rather than by the extra free parameter $C$ in Eq. (2).

Editorial extensions

If this is right

  • If grounding-based calibration works as claimed, any vision-language system using self-consistency UQ can be made more trustworthy at inference time without retraining or fine-tuning.
  • The choice of grounding model matters: domain-matched grounding models (e.g., BiomedCLIP on medical data) give the best calibration, so practitioners should match the grounding model to the deployment domain.
  • The approach extends beyond image-text pairs to other modalities: the authors note future work on audio and video, suggesting the same product formula may apply whenever a response can be grounded in a second modality.
  • Because the method needs only a small validation set to set two parameters, it is practical for safety-critical applications such as medical QA where answers must be vetted.
  • The paper's observation that consistency does not imply accuracy implies that self-consistency scores alone should not be relied on as confidence signals in high-stakes settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The comparison baseline uses only a power exponent $T$, so a concrete check the paper leaves implicit is whether a two-parameter recalibration of the baseline alone, such as a linear mapping $a\cdot\mathrm{Conf}_{\mathrm{baseline}} + b$, achieves similar ECE; if it does, the grounding signal may be serving mainly as an extra recalibration dimension rather than genuinely new evidence.
  • The framework could be applied to pure-text LLMs by grounding answers against a retrieved corpus or knowledge base instead of an image, turning 'grounding' into a general external-consistency check that does not require vision.
  • A testable extension is to measure per-question-type ECE to see whether grounding helps most on questions that require visual evidence versus commonsense knowledge, since the paper's best grounding models differ by domain (LLaMA3.2V for general VQA, BiomedCLIP for medical).
  • The reported gains rely on mapping grounding-model outputs to binary or similarity confidence; using richer probability outputs from the grounding model, such as the full softmax distribution in a closed-form setting, could yield a more informative calibration signal than the binary Yes/No mapping used here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes a post-hoc calibration method for self-consistency-based uncertainty quantification in multimodal LLMs. Given a baseline confidence Conf_baseline and a grounding model's confidence Conf_GM, Eq. (2) defines calibrated confidence as Conf_baseline × Conf_GM^(1/T) + C, with T and C fitted on a validation split. The method is evaluated on VQAv2 with LLaVA and on Slake with LLaVA-Med, using four self-consistency baselines (LexSim, NumSets, PredEnt, SemEnt) and several grounding models (GroundedSAM, CLIP, LLaMA3.2V, QwenVL, BiomedCLIP, Biomed-QwenVL). The authors report large ECE reductions relative to the raw baselines and claim that grounding is much more effective than temperature scaling alone.

Significance. If the grounding advantage survives a same-complexity baseline, the paper offers a simple, training-free, model-agnostic way to improve calibration for multimodal LLMs, and it directly targets the important failure mode in which self-consistency is high but accuracy is low. The paper does not ship code, proofs, or released artifacts, and its central empirical comparison currently does not isolate the effect of cross-modal grounding from the effect of an extra fitted offset. The direction is timely and the proposed pipeline is plausible, but the evidence as presented is not yet sufficient to support the headline claim.

major comments (3)
  1. [§5.4, Eq. (2), Tables 1–2] The central claim in §5.5 that grounding is 'much more effective than calibrating these confidence scores via temperature scaling' is confounded by model complexity. Eq. (2) has two fitted parameters, T and C, while the 'Without Grounding' baseline Conf_baseline^(1/T) has only T. Appendix Tables 3 and 4 show that many of the strongest grounding results use C = 0.2 or 0.5, so part of the ECE reduction may come from the additive offset rather than from cross-modal grounding. Please report a two-parameter grounding-free recalibration of the baseline, e.g., a·Conf_baseline + b or a·Conf_baseline^(1/T) + b, fitted and evaluated on the same validation/test splits, before attributing ECE drops such as Slake SemEnt 0.376→0.008 with BiomedCLIP to the grounding signal.
  2. [§4, Eq. (2)] The proposed confidence can exceed 1: with C = 0.5 and a product term above 0.5, Conf is greater than 1, and the paper does not state whether or how the result is clipped or renormalized to [0,1]. ECE and reliability diagrams are only well-defined for confidence in [0,1], so the exact post-processing must be specified. If clipping is used, the effective transform is piecewise, which should be reflected in the description and in the baseline comparison.
  3. [§5.5, Appendix A.4] The empirical claim of 'significantly improved calibration' is supported only by mean ECE and variance across five random splits. The paper gives no confidence intervals, bootstrap intervals, or significance tests, and it does not state the number of test examples per condition; from the Introduction, the Slake sample appears very small (75 manually verified cases). Without this information, reductions such as 0.376 to 0.008 cannot be assessed for statistical reliability, and the comparison becomes even harder to interpret once the baseline is given the same number of fitted parameters. Add per-condition confidence intervals or paired tests and report the test-set sizes.
minor comments (5)
  1. [Table 2 caption] The caption says 'Comparison of ECE over accuracy of LLaVA for VQA', but Table 2 reports Slake results with LLaVA-Med, BiomedCLIP, and Biomed-QwenVL; the caption should be corrected.
  2. [Appendix A.4] The table headers call C a 'confidence threshold (C)', but C in Eq. (2) is an additive offset, not a threshold; rename it to avoid confusion.
  3. [§5.1] The sentence 'For our experiments, we use again use 80/20 percentage split' contains a grammatical repetition ('use again use'); please rephrase.
  4. [§5.4] The baseline transformation Conf_baseline^(1/T) is described as temperature scaling, but standard temperature scaling operates on logits; since the transformation here is a power transform of confidence values, clarify the relationship or rename the baseline to avoid overloading the term.
  5. [§5.5] The explanation for Biomed-QwenVL's poor performance invokes synthetic post-training data; this is presented as a hypothesis rather than a demonstrated cause, and the sentence should be softened or supported.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the grounding-based calibration method tunes T and C on a validation split and evaluates ECE on held-out test splits, so no prediction reduces to its fitting inputs by construction.

full rationale

The paper's central claim is empirical rather than derivational: Eq. (2) defines Conf = Conf_baseline x Conf_GM^(1/T) + C, with T and C selected on a validation set, and ECE is then reported on separate test splits in Tables 1 and 2. This is standard model selection followed by held-out evaluation, not a fitted parameter renamed as a prediction. The temperature-scaled baselines are also fitted on the same validation split and evaluated on the same test splits, so the comparison is not circular. The most serious concern is that Eq. (2) introduces an extra offset C relative to the one-parameter baseline Conf_baseline^(1/T), so some of the reported ECE improvement could stem from the added flexibility of a second fitted parameter rather than from grounding information. That is a fair-baseline confound and a correctness risk, but it is not a circularity: the grounding signal is genuinely external to the fitted confidence values, and no equation forces the grounded result to equal the baseline or the fit. The self-citation to Kaur et al. (2024) for NumSets and semantic clustering is not load-bearing, since the same baselines are also supported by Kuhn et al. (2023) and other cited work, and the central grounding comparison does not depend on that citation. Overall, the derivation chain is self-contained as an experimental evaluation, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces two fitted parameters (T and C) on top of the baseline confidence, plus hand-chosen sampling settings. It relies on the domain assumption that grounding confidence signals correctness. No new physical or conceptual entities are posited. The key free parameters are the exponent and offset, and the key unverified assumption is the informativeness of the grounding signal.

free parameters (4)
  • T (GM power exponent) = Varies per GM and dataset; e.g., VQA SemEnt LLaMA3.2V T=0.3, VQA LexSim GroundedSAM T=0.7, Slake LexSim BiomedCLIP T=0.1
    Controls the sharpening of the grounding model's confidence. Fitted to the validation split to minimize ECE.
  • C (additive offset) = Varies per GM and dataset; e.g., VQA SemEnt LLaMA3.2V C=0.4, VQA LexSim GroundedSAM C=0.5, Slake LexSim BiomedCLIP C=0.2
    Offset added to the product of confidences. Fitted to the validation split. Not present in the one-parameter baseline, creating an unfair comparison.
  • T_baseline (baseline power exponent) = Varies per baseline and dataset; e.g., VQA LexSim T=1.7, VQA NumSets T=5.1
    Temperature scaling exponent applied to the self-consistency baseline confidence. Fitted to validation; the proposed method has this plus C.
  • Sampling temperature 0.5 and top_p 1.0 = 0.5, 1.0
    Hand-chosen sampling settings for generating the 20 responses used by all baselines; these affect the distribution of responses and therefore all confidence estimates.
assumptions (4)
  • domain assumption Grounding model confidence correlates with correctness of the LLM response.
    The entire method relies on this correlation to down-weight ungroundable answers. Figure 1 gives one qualitative example, but no systematic evaluation of the grounding models' predictive power on the test sets is provided.
  • standard math Expected Calibration Error (ECE) is a sufficient and reliably computed measure of calibration.
    The paper uses ECE exclusively to compare methods; ECE depends on binning, and the bin scheme is not fully specified, so minor differences could be bin artifacts.
  • domain assumption The validation split is representative enough to tune T and C.
    T and C are chosen per dataset and GM on a random 20% partition of the test set. With small test sets, the validation estimate of ECE may be noisy, and the paper reports no sample sizes.
  • domain assumption GPT-4-Turbo provides correct accuracy labels for the motivation figure.
    Figure 2 relies on GPT-4-Turbo to judge whether Llama-2 answers are correct; if GPT-4-Turbo is itself imperfect, the baseline ECE values in that figure would shift.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding." pith.science (2026). https://pith.science/paper/PBNW7KGW

@misc{pith2026250503788,
  author       = {Pith},
  title        = {Pith review of: Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PBNW7KGW}},
  note         = {Machine review of arXiv:2505.03788}
}
read the original abstract

We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for multi-modal large language models (LLMs). Existing state-of-the-art UQ methods rely on consistency among multiple responses generated by the LLM on an input query under diverse settings. However, these approaches often report higher confidence in scenarios where the LLM is consistently incorrect. This leads to a poorly calibrated confidence with respect to accuracy. To address this, we leverage cross-modal consistency in addition to self-consistency to improve the calibration of the multi-modal models. Specifically, we ground the textual responses to the visual inputs. The confidence from the grounding model is used to calibrate the overall confidence. Given that using a grounding model adds its own uncertainty in the pipeline, we apply temperature scaling - a widely accepted parametric calibration technique - to calibrate the grounding model's confidence in the accuracy of generated responses. We evaluate the proposed approach across multiple multi-modal tasks, such as medical question answering (Slake) and visual question answering (VQAv2), considering multi-modal models such as LLaVA-Med and LLaVA. The experiments demonstrate that the proposed framework achieves significantly improved calibration on both tasks.

Figures

Figures reproduced from arXiv: 2505.03788 by the authors.

Figure 1
Figure 1. Consistently incorrect responses generated by LLaVA-Med-v1.5-Mistral-7B ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Reliability diagrams with the expected accuracy of Llama-2-13B on COQA and TriviaQA [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Confbaseline: confidence from a self-consistency UQ baseline such as LexSim (Fomicheva et al., 2020), PredEnt (Malinin & Gales), NumSets (Kuhn et al., 2023; Lin et al., 2023), SemEnt (Kuhn et al., 2023), etc. on a multi-modal LLM such as LLaVA-Med. Conf 1/T GM : temperature-scaled calibrated confidence of grounding model on the accuracy of the generated responses. A grounding model can be as simple as the CLIP-based… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: (a) Histogram on the frequency of LLaVA’s accuracy on VQA. Reliability Diagrams for [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: (a) Histogram on the frequency of LLaVA-Med’s accuracy on Slake. Reliability Diagrams [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Example questions and answers from the VQA dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Reliability Diagrams for UQ by the four self-consistency baselines (PredEnt, SemEnt, [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Reliability Diagrams for UQ by the two self-consistency baselines (PredEnt, NumSets) of [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 7 canonical work pages

  1. [1]

    VQD: Visual Query Detection in Natural Scenes

    Manoj Acharya, Karan Jariwala, and Christopher Kanan. Vqd: Visual query detection in natural scenes. arXiv preprint arXiv:1904.02794,

    work page Pith review arXiv 1904

  2. [7]

    A confederacy of models: A comprehensive evaluation of llms on creative writing

    Carlos G´omez-Rodr´ıguez and Paul Williams. A confederacy of models: A comprehensive evaluation of llms on creative writing. arXiv preprint arXiv:2310.08433,

    arXiv

  3. [9]

    Overconfidence is key: Verbalized uncertainty evaluation in large language and vision-language models

    Tobias Groot and Matias Valdenegro-Toro. Overconfidence is key: Verbalized uncertainty evaluation in large language and vision-language models. arXiv preprint arXiv:2405.02917,

    arXiv

  4. [10]

    Deberta: Decoding-enhanced bert with disentangled attention

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654,

    arXiv 2006

  5. [11]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,

    arXiv

  6. [14]

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551,

    arXiv

  7. [15]

    Language models (mostly) know what they know

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221,

    arXiv

  8. [17]

    Uncertainty-aware evaluation for vision-language models

    Vasily Kostumov, Bulat Nutfullin, Oleg Pilipenko, and Eugene Ilyushin. Uncertainty-aware evaluation for vision-language models. arXiv preprint arXiv:2402.14418,

    arXiv

Show all 34 references
  1. [18]

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664,

    arXiv

  2. [19]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36: 28541–2...

    arXiv

  3. [21]

    Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering

    Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1650–1654. IEEE,

    2021

  4. [23]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159,

    arXiv

  5. [24]

    Pelican: Correcting hallucination in vision-llms via claim decomposition and program of thought verification

    Pritish Sahu, Karan Sikka, and Ajay Divakaran. Pelican: Correcting hallucination in vision-llms via claim decomposition and program of thought verification. arXiv preprint arXiv:2407.02352,

    arXiv

  6. [25]

    A survey on uncertainty quantification of large language models: Taxonomy, open research challenges, and future directions

    Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z Ren, and Anirudha Majumdar. A survey on uncertainty quantification of large language models: Taxonomy, open research challenges, and future directions. arXiv preprint arXiv:2412.05563,

    arXiv

  7. [26]

    Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback

    Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In The 2023 Conf...

    2023

  8. [27]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288,

    arXiv

  9. [28]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191,

    arXiv

  10. [29]

    Copu: Conformal prediction for uncertainty quantification in natural language generation

    Sean Wang, Yicheng Jiang, Yuxin Tang, Lu Cheng, and Hanjie Chen. Copu: Conformal prediction for uncertainty quantification in natural language generation. arXiv preprint arXiv:2502.12601,

    arXiv

  11. [31]

    Benchmarking llms via uncertainty quantification

    Fanghua Ye, Mingming Yang, Jianhui Pang, Longyue Wang, Derek F Wong, Emine Yilmaz, Shum- ing Shi, and Zhaopeng Tu. Benchmarking llms via uncertainty quantification. arXiv preprint arXiv:2401.12794,

    arXiv

  12. [33]

    Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.009...

    arXiv

  13. [34]

    Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)

    Q1: What does the sign say? A: ’no cursing’ Q2: What color is the engine? A: Red Q3: Are most of the people wearing hats? A: No Q4: Who does the man on the right resemble? A: ’surfer’ Figure 6: Example questions and answers from the VQA dataset. Distribution Statement “A” (App...

    1904

  14. [1957]

    Distribution Statement “A” (Approved for Public Release, Distribution Unlimited). Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistr...

    arXiv

  15. [1983]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805,

    arXiv

  16. [2004]

    Generating with confidence: Uncertainty quantifica- tion for black-box large language models

    Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantifica- tion for black-box large language models. arXiv preprint arXiv:2305.19187,

    arXiv

  17. [2014]

    On domain-specific post-training for multimodal large language models

    Daixuan Cheng, Shaohan Huang, Ziyu Zhu, Xintong Zhang, Wayne Xin Zhao, Zhongzhi Luan, Bo Dai, and Zhenliang Zhang. On domain-specific post-training for multimodal large language models. arXiv preprint arXiv:2411.19930,

    arXiv

  18. [2015]

    Conformal language modeling

    Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S Jaakkola, and Regina Barzilay. Conformal language modeling. arXiv preprint arXiv:2306.10193,

    arXiv

  19. [2017]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    arXiv

  20. [2018]

    Vl-uncertainty: Detecting hallucination in large vision-language model via uncertainty estimation

    Ruiyang Zhang, Hu Zhang, and Zhedong Zheng. Vl-uncertainty: Detecting hallucination in large vision-language model via uncertainty estimation. arXiv preprint arXiv:2411.11919,

    arXiv

  21. [2019]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  22. [2020]

    Explaining multi-modal large language models by analyzing their vision perception

    Loris Giulivi and Giacomo Boracchi. Explaining multi-modal large language models by analyzing their vision perception. arXiv preprint arXiv:2405.14612,

    arXiv

  23. [2022]

    Ad- dressing uncertainty in llms to enhance reliability in generative ai

    Ramneet Kaur, Colin Samplawski, Adam D Cobb, Anirban Roy, Brian Matejek, Manoj Acharya, Daniel Elenius, Alexander Michael Berenbeim, John A Pavlik, Nathaniel D Bastian, et al. Ad- dressing uncertainty in llms to enhance reliability in generative ai. In Neurips Safe Generative ...

    2024

  24. [2023]

    Cursorcore: Assist programming through aligning anything

    Hao Jiang, Qi Liu, Rui Li, Shengyu Ye, and Shijin Wang. Cursorcore: Assist programming through aligning anything. arXiv preprint arXiv: 2410.07002,

    arXiv

  25. [2024]

    An empirical study of in-context learning in llms for machine translation

    Pranjal Chitale, Jay Gala, and Raj Dabre. An empirical study of in-context learning in llms for machine translation. In Findings of the Association for Computational Linguistics ACL 2024, pp. 7384–7406,

    2024

  26. [2025]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171,

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.