Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that training vision-language models on images whose key object regions are deliberately degraded makes the models' stated confidence track their actual correctness on object-level queries.

desk verdict A clever but under-supported training recipe for verbalized confidence; the main evaluation protocol is missing and the labels are a noise-derived proxy. read the letter →

arxiv 2504.14848 v1 pith:ZKJMBYAS submitted 2025-04-21 cs.CV cs.AI

Yunpu Zhao , Rui Zhang , Junbin Xiao , Ruibo Hou , Jiaming Guo , Zihao Zhang , Yifan Hao , Yunji Chen This is my paper · ORCID
classification cs.CVcs.AI
keywords verbalizedconfidencecalibrationvision-languagemodelsobjecthallucinationsemanticperturbationpreferenceoptimizationuncertaintyestimationmultimodaltrustworthiness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a vision-language model's verbalized confidence, such as saying it is 80% sure a cat is present, can be made to match whether that object claim is actually correct. The proposed approach, Confidence Calibration through Semantic Perturbation (CSP), works by adding controlled Gaussian noise to the specific object region a question asks about, labeling each degraded image with a confidence value, and fine-tuning the model first with supervised learning and then with preference optimization. On object-hallucination benchmarks, the method reduces the gap between stated confidence and correctness while holding or improving task accuracy. The reason to care is that users and downstream systems could then treat a model's stated confidence as a trustworthy signal for accepting or rejecting object claims, rather than being misled by confident errors.

What carries the argument

The load-bearing mechanism is semantic perturbation: a linear schedule maps a target confidence label c to a diffusion step T_c = floor(T_max * (1 - c/100)), Gaussian noise is iteratively added to the masked object region, and the noised region is composited onto the unperturbed background. Each perturbed image becomes a training pair whose target verbalized confidence is exactly c. The training then uses supervised cross-entropy on those pairs, followed by the SimPO preference objective with c as the chosen response and 100% - c as the rejected response. This chain is what teaches the model to tie stated confidence to local visual reliability.

What would settle it

Take a held-out set of real images with genuine occlusion, blur, or low lighting, obtain oracle correctness labels for each object query, and compare expected calibration error before and after CSP training; if the improvement disappears, or the model merely becomes pessimistic about noisy-looking objects without better predicting correctness, the assumed noise-to-correctness mapping fails to generalize.

Watch

Extended reading notes

Core claim

The central claim is that object-level verbalized confidence in vision-language models can be calibrated by supervised training on images whose key object regions have been degraded to known noise levels, followed by preference optimization that rewards confidence statements matching the noise-derived label. The paper reports that this two-stage procedure lowers expected calibration error and Brier score, raises accuracy, F1, and AUC on POPE and AMBER across four different models, and preserves or improves performance on the MME benchmark. It also reports that internal token-level probabilities and verbalized confidence become more strongly correlated, even though the training never directly supervises internal probabilities.

Load-bearing premise

The synthetic mapping from pixel noise to confidence label is treated as a valid proxy for the probability that the model's answer is actually correct, so that training on noisy objects transfers to real unperturbed images.

Editorial extensions

If this is right

  • After CSP training, the answer a vision-language model selects by its highest stated confidence is correct more often, making verbalized confidence usable for answer selection and abstention.
  • Calibration gains appear in models with very different starting points, including both severely overconfident and already well-calibrated models.
  • Task accuracy on object-probing and hallucination benchmarks, and on the broader MME evaluation, stays flat or improves, so the calibration is not bought by degrading overall competence.
  • Internal token-level probabilities and verbalized confidence become more consistent even though the training objective never directly supervises internal probabilities.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test, not run in the paper, would replace the synthetic noise-derived confidence labels with human-annotated confidence on real occluded or low-light images; if CSP still improves calibration, the noise schedule is only a convenient surrogate rather than the essential ingredient.
  • Because the perturbation targets semantic object regions, the same construction could be applied to other uncertainty dimensions the paper leaves open, such as attribute, relation, or temporal ambiguity in video.
  • The reported gains suggest a cheap route to selective prediction: a deployment threshold on verbalized confidence could flag likely hallucinations in object-centric systems, although the paper does not evaluate such a rejection rule.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes Confidence Calibration through Semantic Perturbation (CSP), a two-stage training framework for object-level verbalized confidence in vision-language models. In the dataset construction stage, the authors localize query-referenced objects with GroundingDINO and SAM, inject Gaussian noise into those regions using a diffusion schedule, and assign a confidence label c determined by the noise step. They then fine-tune VLMs with supervised learning to output that label in a third-person confidence prompt, followed by SimPO preference optimization with the rejected confidence 100%-c. Evaluation is carried out on POPE and AMBER with metrics Acc, F1, ECE, AUC, and Brier score, and on MME for task performance; the paper reports consistent calibration gains across Qwen-VL, Qwen2-VL, InternVL2, and Phi-3.5, together with ablations and an analysis of internal/verbalized confidence correlation. The central claim is that CSP improves the alignment between verbalized confidence and response correctness while maintaining or enhancing task performance.

Significance. The paper addresses an important and timely problem: verbalized confidence calibration in VLMs, especially in the context of object hallucination. The idea of using semantic mask perturbation to create training data with controlled visual uncertainty is simple and potentially useful, and the authors test it on four model families and several benchmarks. The manuscript is clearly organized, and the limitation section honestly acknowledges the scope of the study. That said, the current evidence does not yet establish the central claim, because the training labels are a noise-level proxy rather than correctness-based labels, and because the evaluation protocol for the headline metrics is not described. With additional correctness-conditional analysis, a fully specified evaluation harness, and a fuller ablation table, the work could become a solid contribution.

major comments (3)
  1. [Section 3.2] The ground-truth confidence label is defined by the diffusion-step schedule Tc = floor(Tmax*(1-c/100)), so c is a monotone function of injected pixel noise and not a measure of P(correct | perturbed image). For a fixed query-response pair, perturbing the object region does not change the truth value of the answer: a correct answer remains correct even at c=0%, and a hallucinated answer remains wrong even at c=100%. The SFT objective therefore trains the model to verbalize low confidence for degraded-looking images and high confidence for clean images, and the SimPO rejected response 100%-c inherits the same artifact. Because the POPE and AMBER test images are clean, the reported ECE/AUC/Brier improvements could reflect a shift in score distributions rather than improved alignment with correctness. I request (i) a validation of the assumed noise-to-correctness mapping, for example by reporting the empirical correctness rate of the underlying answers at each noise level, and (ii) a correctness-conditional analysis on the held-out clean benchmarks, such as ECE and score distributions computed separately for correct and incorrect responses, to show that the learned scores track correctness rather than image degradation.
  2. [Sections 3.1, 4.2, Table 1] The evaluation protocol is under-specified. The paper repeatedly refers to candidate answers {a_1,...,a_k} and to 'the prediction chosen by the highest confidence c(ŷ_i)', but it never states how the candidate set is constructed for POPE/AMBER, how many candidates are used, whether the correct answer is always among them, or how the verbalized response (e.g., '70.37%') is parsed into a numeric probability p_i. The reported baseline values, Acc=0.25 for Qwen-VL on POPE random and Acc=0.11 for Qwen2-VL on POPE random, are far below both chance for binary questions and standard POPE accuracy for these models; the table caption explicitly says these metrics do not measure dataset performance, but no alternative protocol is defined. Without a precise description of candidate construction, confidence parsing, and tie handling, the Acc/F1/AUC/ECE numbers are not reproducible, and the claimed improvements cannot be attributed to the method. Please provide a complete evaluation protocol in the appendix and ideally release the evaluation code.
  3. [Section 4.3.2, Figure 4] The ablation study is reported only for one model/setting (Qwen2 on POPE adversarial) and without numerical values or uncertainty estimates. The text states that 'SimPO Only ... shows nearly no gains' and 'Original RLAIF ... does not have improvement', but no table presents these numbers across the four models and multiple datasets. This matters because these ablations are the main evidence that the semantic mask perturbation, rather than the extra SFT data or preference optimization, is responsible for the reported gains. Please add a full ablation table across models and datasets, with the same metrics as Table 1, and report repeated-run variability if feasible.
minor comments (6)
  1. [Introduction] The first sentence of the Introduction contains a duplicated citation block: '(Achiam et al., 2023; Bubeck et al., 2023)' appears twice in a row.
  2. [Sections 3.2 and 4.1, Appendix B.2] There are several typos: 'we begin with extract' should be 'we begin by extracting', 'designed to evaluation' should be 'designed to evaluate', 'donwloaded' should be 'downloaded' (twice in Appendix B.2), and Section 3.2 ends with a double period after 'recognition tasks..'.
  3. [Section 3.2, Appendix B.1] The hyperparameters Tmax and gamma are never reported, and the distribution from which confidence labels are sampled is not specified; please add these values to the implementation details in Appendix B.1.
  4. [Section 4.2] The definition of ECE uses K bins of equal confidence range, but K and the binning strategy are not reported; please state the number of bins and whether they are equal-width or equal-frequency.
  5. [Table 2] The Spearman and Kendall correlations are reported without confidence intervals or significance tests, and several values are near zero (for example, Qwen-VL on AMBER relation has rho = -0.02); the claim of consistent improvement should be qualified accordingly.
  6. [Table 5, Appendix C.3] The MME results report a macro-averaged F1 score without defining how it is computed across the 14 subtasks; standard MME reporting uses perception and cognition accuracy scores, so please clarify the F1 aggregation.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the training confidence targets are self-defined by the noise schedule, but the reported calibration gains are measured on held-out external benchmarks against ground-truth correctness, so the central claim is not forced by construction.

full rationale

The paper's training labels are defined by the injected noise schedule (rc=c, with c mapped to diffusion step Tc=floor(Tmax*(1-c/100))), so the supervised and preference objectives teach the model to reproduce the paper's own perturbation-to-confidence mapping. This is a genuine construct-validity concern: the correctness of the fixed response is not used to set c, the same response is paired with all noise levels, and the empirical probability P(correct|c) is not controlled. However, this does not meet the bar for circularity as defined here. The central claim—improved alignment between verbalized confidence and response correctness—is evaluated on held-out POPE, AMBER, and MME samples with ground-truth correctness labels, not on the synthetic training distribution, and no reported evaluation metric is computed from the fitted c values. The paper does not invoke a load-bearing self-citation or an imported uniqueness theorem; the only salient self-citation (Zhao et al., 2024a) is motivational for the sycophancy framing. The limitations section acknowledges scope restrictions but does not assert or reveal a circular step. Therefore, while the label-proxy assumption is a substantive weakness that could undermine interpretation of the empirical gains, the derivation chain is not circular in the sense of predicting the evaluation target from the fitted input by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a synthetic confidence label that is defined by the perturbation schedule rather than by answer correctness. The paper adds no new physical or mathematical entities. Most assumptions are domain assumptions about what noise level means for confidence, not standard mathematical axioms.

free parameters (3)
  • Tmax (max diffusion steps in confidence labeling) = not specified in paper
    Controls the linear schedule Tc = Tmax*(1-c/100). The mapping from noise level to confidence label depends directly on this chosen bound; without a value, the training confidence labels are not fully specified.
  • gamma (per-step noise intensity) = not specified in paper
    Controls how much Gaussian noise is added per diffusion step in the forward process v_t ~ N(sqrt(1-gamma)v_{t-1}, gamma I). Together with Tmax it defines the visual uncertainty corresponding to each confidence label.
  • confidence label sampling distribution = uniform over 0%-100% (assumed)
    The paper says 'The confidence label is sampled from 0% to 100%' (Sec 3.2) but does not specify the distribution; this affects the training set balance.
assumptions (5)
  • domain assumption Gaussian noise on key object regions monotonically maps to visual uncertainty and therefore to the probability that a candidate answer is correct
    Core premise of dataset construction in Sec 3.2. Used to assign ground-truth confidence labels c = 100% - (Tc/Tmax)*100 without consulting actual correctness.
  • domain assumption Verbalized confidence scores can be treated as probability estimates in ECE, Brier, and AUC computations
    Sec 3.1 and Sec 4.2 use the verbalized number directly as p_i in calibration metrics.
  • domain assumption Third-person confidence prompts yield more objective confidence than first-person prompts
    Sec 3.2 states first-person prompts cause inflated self-assessments and TPP discourages sycophancy; this choice shapes all training and evaluation data.
  • standard math The diffusion forward process equations from Ho et al. (2020) apply as a noise schedule
    Sec 3.2 uses v_t ~ N(sqrt(1-gamma)v_{t-1}, gamma I) without derivation; this is standard.
  • domain assumption Rejected confidence response 100% - c is a valid negative example for preference optimization
    Sec 3.3 defines y_l = 100% - c, assuming that the complement of the noise-derived label is the incorrect confidence level.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation." pith.science (2026). https://pith.science/paper/ZKJMBYAS

@misc{pith2026250414848,
  author       = {Pith},
  title        = {Pith review of: Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZKJMBYAS}},
  note         = {Machine review of arXiv:2504.14848}
}
read the original abstract

Vision-language models (VLMs) excel in various multimodal tasks but frequently suffer from poor calibration, resulting in misalignment between their verbalized confidence and response correctness. This miscalibration undermines user trust, especially when models confidently provide incorrect or fabricated information. In this work, we propose a novel Confidence Calibration through Semantic Perturbation (CSP) framework to improve the calibration of verbalized confidence for VLMs in response to object-centric queries. We first introduce a perturbed dataset where Gaussian noise is applied to the key object regions to simulate visual uncertainty at different confidence levels, establishing an explicit mapping between visual ambiguity and confidence levels. We further enhance calibration through a two-stage training process combining supervised fine-tuning on the perturbed dataset with subsequent preference optimization. Extensive experiments on popular benchmarks demonstrate that our method significantly improves the alignment between verbalized confidence and response correctness while maintaining or enhancing overall task performance. These results highlight the potential of semantic perturbation as a practical tool for improving the reliability and interpretability of VLMs.

Figures

Figures reproduced from arXiv: 2504.14848 by the authors.

Figure 1
Figure 1. (Top): Most current VLMs tend to generate [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The image illustrates the dataset construction and training pipeline for improving confidence calibra [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. ROC curves (top row) and probability calibration plots (bottom row) on the AMBER attribute dataset, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Ablation results for different variants of our method under POPE adversarial of model Qwen2 SimPO does not have improvement. Indeed, the added supervision and varied noise conditions in our semantic perturbation dataset appear crucial for learning robust, multimodal co…
Figure 5
Figure 5. Figure 5: Comparison of Accuracy, Precision, Recall, [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of token-level probability dis [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: ROC curves (top row) and probability calibration plots (bottom row) on the POPE adversarial dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: ROC curves (top row) and probability calibration plots (bottom row) on the POPE popular dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: ROC curves (top row) and probability calibration plots (bottom row) on the POPE random dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: ROC curves (top row) and probability calibration plots (bottom row) on the AMBER relation dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Vision-language models vary widely in how trustworthy their confidence scores are on document extraction, with stronger models and OCR-plus-image input helping most, as measured on the new ConfBench benchmark.

  2. Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Current vision-language models are largely miscalibrated when they verbalize confidence, visual reasoning models such as o3 and o4-mini are better calibrated, and Visual Confidence-Aware Prompting reduces ECE on IsoBench.

Reference graph

Works this paper leans on

31 extracted references · 5 canonical work pages · cited by 2 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

    arXiv 2024

  4. [4]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  5. [5]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966

    arXiv 2023

  6. [6]

    S \'e bastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712

    arXiv 2023

  7. [7]

    Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2023. Discovering latent knowledge in language models without supervision. In The Eleventh International Conference on Learning Representations

    2023

  8. [8]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. 2024. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185--24198

    2024

Show all 31 references
  1. [9]

    Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. 2024. https://doi.org/10.18653/v1/2024.acl-long.276 Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models ...

    2024 doi

  2. [10]

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. 2023. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394

    2023 arXiv

  3. [11]

    Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. 2024. A survey of confidence estimation and calibration in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Ling...

    2024

  4. [12]

    Tobias Groot and Matias Valdenegro-Toro. 2024. Overconfidence is key: Verbalized uncertainty evaluation in large language and vision-language models. In Proceedings of TrustNLP Workshop@ NAACL 2024

    2024

  5. [13]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840--6851

    2020

  6. [14]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Info...

    2023

  7. [15]

    Zaid Khan and Yun Fu. 2024. Consistency and uncertainty: Identifying unreliable responses from black-box vision-language models for selective visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10854--10863

    2024

  8. [16]

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015--4026

    2023

  9. [17]

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. In The Eleventh International Conference on Learning Representations

    2023

  10. [18]

    Abhishek Kumar, Robert Morabito, Sanzhar Umbet, Jad Kabbara, and Ali Emami. 2024. https://doi.org/10.18653/v1/2024.acl-long.20 Confidence under the hood: An investigation into the confidence-probability alignment in large language models . In Proceedings of the 62nd Annual Mee...

    2024 doi

  11. [19]

    Bhawesh Kumar, Charlie Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, and Andrew Beam. 2023. Conformal prediction with large language models for multi-choice question answering. arXiv preprint arXiv:2305.18404

    2023 arXiv

  12. [20]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292--305

    2023

  13. [21]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. Transactions on Machine Learning Research

    2022

  14. [22]

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. 2025. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pages 38--55. Springer

    2025

  15. [23]

    Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. In Advances in Neural Information Processing Systems (NeurIPS)

    2024

  16. [24]

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. 2023. An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv preprint arXiv:2311.07397

    2023 arXiv

  17. [25]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  18. [26]

    Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. 2023. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. In The Twelfth International Conference on Learning Representations

    2023

  19. [27]

    Tianyang Xu, Shujin Wu, Shizhe Diao, Xiaoze Liu, Xingyao Wang, Yangyi Chen, and Jing Gao. 2024. Sayself: Teaching llms to express confidence with self-reflective rationales. arXiv preprint arXiv:2405.20974

    2024 arXiv

  20. [28]

    Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuan-Jing Huang. 2023. Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023, pages 8653--8665

    2023

  21. [29]

    Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, and Maosong Sun. 2024. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220

    2024

  22. [30]

    Yunpu Zhao, Rui Zhang, Junbin Xiao, Changxin Ke, Ruibo Hou, Yifan Hao, Qi Guo, and Yunji Chen. 2024 a . Towards analyzing and mitigating sycophancy in large vision-language models. arXiv preprint arXiv:2408.11261

    2024

  23. [31]

    Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, et al. 2024 b . Swift: a scalable lightweight infrastructure for fine-tuning. arXiv preprint arXiv:2408.05517

    2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.