{"id":"ef96c985-eddc-4649-8ce2-e9d3e75ccbb7","arxiv_id":"2412.14660","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Multimodal LLMs are persistently miscalibrated, their calibration barely changes after fine-tuning or multimodal training, and they prefer giving answers to admitting ignorance, though prompting and temperature scaling can reduce overconfidence.","lead":"This paper studies how confident multimodal AI models (LLaVA, Qwen-VL) are when answering image-and-text questions, and finds they are often miscalibrated. It introduces a new dataset of questions the models do not know and tests prompts and calibration methods to make model confidence more reliable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The model-specific IDK labels are derived from the same model's 10 stochastic samples with a perfect-accuracy threshold, so the reported self-assessment gains may reflect sampling noise rather than genuine knowledge or uncertainty.","rationale":"The reader's weakest assumption is the same one I identify as most load-bearing: the model-specific IDK ground truth is constructed from the same model's stochastic answers, and every self-assessment metric on those rows inherits that labeling. Unlike the missing significance tests and the in-sample calibration evaluation, this flaw directly corrupts the 'MLLMs tend to give answers rather than admit uncertainty, but prompting improves self-assessment' contribution, which is one of the two claims highlighted in the abstract. The threshold-1, 10-sample rule is quantitatively fragile: even a model that is correct 90% of the time on a question is usually labeled as not knowing it, so the comparison of no-prompt versus prompted behavior can be mechanically inflated by converting mislabeled known questions into apparent correct 'I don't know' responses. The OOD dataset provides partial independent support, but its own validity depends on an unverified assumption that the accompanying image does not contain the answer, so it does not fully rescue the claim. The proposed robustness check—recompute the split with a more reliable oracle and see whether the qualitative pattern survives—would settle whether this concern actually lands. Other concerns (no significance tests, same-data calibration fitting) are real but secondary; the reader's reject verdict is therefore unchanged.","tokens_in":17299,"tokens_out":10801,"duration_ms":89308,"concrete_test":"Recompute the model-specific portion of Table 3 (at least MMBench and SEED-Bench for LLaVA-7B and LLaVA-13B) with 'known' defined by accuracy >= 0.8 over 100 samples at a lower temperature, or with a held-out human sanity set of clearly known and clearly unknown questions, and compare TRUTHFUL before and after prompting. If the prompting gain drops by more than 5 percentage points or reverses, the 10-sample/perfect-accuracy labeling is load-bearing; if the gain survives, the concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central self-assessment claim rests on the model-specific IDK split in Section 5. A question is labeled 'known' only if 10 samples at temperature 1 and top-p 0.95 are all correct (threshold 1); otherwise it is labeled 'unknown.' This is a fragile oracle. For a question the model actually knows with per-sample accuracy p, the probability of being labeled 'known' is p^10, so a question answered correctly 90% of the time has only a 35% chance of being labeled 'known,' and one answered correctly 95% of the time has only a 60% chance. Consequently, a large fraction of 'IDK' items are likely known-but-noisy items. The TRUTHFUL metric in Table 3 then counts the model answering those mislabeled items as 'failing to admit uncertainty,' and counts a prompt-induced 'I don't know' on them as 'improved self-assessment.' The OOD rows avoid this particular circularity but instead assume the image does not leak the answer, which is not verified for image-grounded questions. Because the abstract's claim that 'self-assessment improves with proper prompt adjustments' is evaluated mainly against this 10-sample/perfect-accuracy oracle, the claim is not currently supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents an empirical study of uncertainty calibration in multimodal large language models. It compares ECE, MCE, and ENCE of LLaVA and Qwen-VL before and after visual fine-tuning (Tables 1 and 5-10) and relative to their base LLMs on linguistic tasks (Tables 2 and 11-14), concluding that MLLMs are miscalibrated but that calibration does not differ significantly across these scenarios. It then analyzes confidence from image versus text modalities and their integration (Section 4), constructs a model-specific IDK dataset and a July 2024 OOD news VQA dataset (Section 5), reports that MLLMs rarely say 'I don't know' and that prompting improves truthfulness metrics, and proposes temperature scaling and iterative prompt optimization (Section 6).","tokens_in":17497,"tokens_out":6720,"duration_ms":104646,"significance":"If the claims were supported, the paper would provide a useful negative result about fine-tuning and multimodal training not systematically degrading calibration, a new benchmark-style resource in the IDK and July24-NewsVQA datasets, and practical calibration recipes. The authors also release code and dataset links, which is a positive feature. However, the central empirical claims are currently under-supported: the 'no significant differences' conclusion lacks inferential statistics, the IDK oracle is circular and fragile, and the calibration gains are evaluated in-sample. The study's value is therefore conditional on a substantial revision that either strengthens the evidence or weakens the claims.","major_comments":[{"comment":"The abstract and Section 3 state that there are \"no significant differences in calibration across these scenarios,\" but the paper reports only point estimates of accuracy, confidence, ECE, MCE, and ENCE on randomly sampled subsets whose sizes are never stated, with no significance tests, confidence intervals, or error bars. For instance, Table 1 shows LLaVA-13B Stage 1 ECE of 0.235 versus Stage 2 ECE of 0.101 on MMBench; without an uncertainty estimate, the claim that such differences are \"not significant\" cannot be assessed. Please report sample sizes, bootstrap confidence intervals or Bayesian credible intervals, and an explicit test or power analysis for the negative claim.","section":"Section 3 and Abstract"},{"comment":"The model-specific IDK split labels a question as \"known\" only if all 10 stochastic samples at temperature 1 and top-p 0.95 are correct (threshold 1). For a question the model actually knows with per-sample accuracy p, the probability of being labeled \"known\" is p^10, so a known question answered correctly 90% of the time has only a 35% chance of being labeled \"known.\" The TRUTHFUL metric in Tables 3 and 4 therefore treats known-but-noisy items as \"unknown\" and counts prompted refusals on those items as improved self-assessment. Because the abstract's claim that \"self-assessment improves with proper prompt adjustments\" is evaluated mainly against this oracle, that claim is not currently supported. Please validate the oracle against an independent criterion (for example, human labels or a calibrated threshold) or restrict the self-assessment claim to the OOD data.","section":"Section 5, Construction of dataset"},{"comment":"Temperature scaling and iterative prompt optimization are fit and evaluated on the same data. Equation (2) minimizes the negative log-likelihood over M samples, and Fig. 4 reports post-calibration ECE on MMBench without stating whether T was optimized on a separate validation split. Algorithm 1 selects suffixes by accuracy and ECE using the same evaluation function f, so the reported improvements are partly in-sample selection artifacts. Please use a held-out test set or cross-validation, and report the selection bias; in particular, state how the hyperparameters k, m, and n were chosen.","section":"Section 6, Eq. (2), Algorithm 1, Fig. 4"},{"comment":"The modality uncertainty comparison relies on GPT-4V text descriptions as faithful proxies for image content, but this assumption is not verified against the actual image (Fig. 1), so text-versus-image confidence differences may reflect description quality rather than modality properties. Additionally, Fig. 2 reports point estimates from unspecified subsets without error bars or repeated sampling. Please verify the fidelity of the descriptions, report sample sizes, and provide uncertainty estimates before concluding that MLLMs have lower uncertainty on text than on images.","section":"Section 4, Figs. 1-3"}],"minor_comments":[{"comment":"There are typos: \"LMMs\" in Section 2.1 should be \"MLLMs,\" and \"adversial\" in Section 7 should be \"adversarial.\"","section":"Section 2.1 and Section 7"},{"comment":"Line 3 is malformed: \"S ← p∈G generate(p, m)\" should be written as a set comprehension, e.g., S ← {generate(p, m) : p ∈ G}.","section":"Algorithm 1"},{"comment":"The symbol |y| is used without being defined, and M in Eq. (2) overloads the M that denotes the number of bins in Eq. (1); please use different symbols for the number of samples and the number of classes.","section":"Eq. (2)"},{"comment":"The caption uses \"N (0, N oise)\" and \"NoisyImage=Image+N (0, N oise)\" without stating whether \"Noise\" is the variance or the standard deviation of the Gaussian perturbation.","section":"Figure 3 caption"},{"comment":"The four categories IK-IDK, IDK-IDK, IK-IK, and IDK-IK are introduced only in the table; please define each label combination explicitly in the text before using the acronyms.","section":"Section 5 and Table 3"},{"comment":"There is an incomplete sentence in Appendix D: \"The dataset con-\" should be completed or removed.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"I am sympathetic to the effort and the released resources, but the current manuscript should not be accepted as is. The issues are methodological rather than suggestive of misconduct: the negative result needs inferential statistics, the IDK oracle needs validation or a weaker claim, and the calibration experiments need held-out evaluation. If these cannot be supplied, the claims should be softened substantially. The code and dataset release is a positive feature, and the paper fits the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read of the MLLM calibration paper. It's a decent empirical survey—they measure ECE across fine-tuning stages and against base LLMs, and they run a genuinely nice protocol that progressively adds text descriptions to noisy images to watch uncertainty drop. That last bit is the freshest thing in the paper, and they ship code and data.\n\nThe problems are where the headline claims live. First, 'no significant differences' appears throughout, but there are no significance tests, confidence intervals, or error bars anywhere. The tables are point estimates from random subsets of unspecified size, so the central null finding is not actually demonstrated. Second, the calibration techniques in Section 6—temperature scaling and iterative prompt optimization—are fit and evaluated on the same dataset. The temperature objective in Eq. (2) and Algorithm 1 both use the same evaluation function on the same data, so the ECE improvements in Fig. 4 are partly in-sample selection. A separate validation split would fix this. Third, the IDK oracle is fragile. Labeling a question 'known' only if 10 stochastic samples are all correct means a question the model answers correctly 90% of the time has a 35% chance of being labeled known. That mislabels a lot of known-but-noisy items as unknown, and the TRUTHFUL metric then counts a refusal on those items as good self-assessment. The OOD news dataset avoids this particular circularity, but the in-distribution IDK claims rest on the same shaky oracle.\n\nThe abstract's claim that self-assessment improves with prompting is partially rescued by the OOD results, but it's still evaluated against a flawed in-distribution split. None of this is fatal to the paper's empirical map; the fine-tuning results are plausible and the uncertainty-integration experiment is a useful contribution. But the load-bearing claims need held-out evaluation, proper statistical tests, and an independent or better-justified IDK labeling before they can be accepted.\n\nMy take: this deserves a serious referee. The topic is timely, the experiments are broad, and the flaws are fixable. It's not ready for publication as is, but a careful reviewer could help the authors turn it into something solid. I'd send it out.","headline":"Useful empirical map of MLLM calibration, but the headline claims rest on unsupported statistics and a self-referential IDK oracle; deserves referee time but needs major revisions.","tokens_in":18076,"tokens_out":3017,"would_cite":false,"duration_ms":33197,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multimodal large language models remain overconfident across fine-tuning, but prompt adjustments and temperature scaling improve calibration.","keywords":["multimodal large language models","calibration","uncertainty quantification","expected calibration error","IDK dataset","visual question answering","overconfidence","prompt optimization"],"falsifier":"Re-label the IDK questions with more samples (say 30 or 100) or with a lower accuracy threshold and recompute TRUTHFUL; if many questions labeled 'doesn't know' are answered correctly in 9 of 10 or 20 of 30 draws, the paper's overconfidence figures would be inflated.","tokens_in":17030,"feed_emoji":"🎯","tokens_out":8935,"duration_ms":79391,"temperature":0.7,"pith_summary":"This paper asks whether multimodal large language models (MLLMs) can be trusted to know when they do not know, and whether their confidence tracks their accuracy. The authors find that MLLMs are persistently miscalibrated—typically overconfident—and that this overconfidence is stable: visual fine-tuning and multimodal training on top of a base language model do not significantly change calibration. Using a new dataset called IDK, built by having each model answer every question ten times, they show that models almost always give an answer rather than admit ignorance, although a simple prompt like 'if you don't know, say so' visibly improves self-assessment. They also report that MLLMs carry more uncertainty about image content than about text, and that adding textual information reduces overall uncertainty. The practical upshot is that calibration techniques such as temperature scaling and prompt optimization can make MLLMs more reliable in high-stakes applications.","feed_headline":"Multimodal LLMs stay miscalibrated after fine-tuning; prompts help","feed_subtitle":"Calibration barely changes across training; the IDK dataset shows models answer when unsure, and prompting helps.","key_machinery":"The IDK dataset construction is the load-bearing device: a model answers each question 10 times at temperature 1 and top-p 0.95, and a question is labeled 'doesn't know' only if none of the 10 answers is correct. This binary split, together with the TRUTHFUL metric (the fraction of items where the model's stated answer matches the know/don't-know label), carries the overconfidence claim. Calibration is measured by Expected Calibration Error (ECE) and its variants MCE and ENCE, with logits-based likelihood as the primary uncertainty score and semantic entropy for open-ended responses. The proposed fixes are temperature scaling on the predictive distribution and an iterative prompt-optimization loop that refines suffixes such as 'Answer:' into calibration-friendly versions.","core_discovery":"The central claim is that miscalibration is the default state of MLLMs, and it is stable across the training scenarios the paper studies: calibration does not significantly differ before versus after visual fine-tuning, nor between a base LLM and its multimodal counterpart on language tasks. At the same time, uncertainty is modality-dependent—image information produces higher uncertainty than text—and the two modalities combine to reduce uncertainty as more text is supplied. On the newly constructed IDK dataset, MLLMs overwhelmingly answer questions even when they do not know, but prompting them to say 'I don't know' substantially increases the truthfulness of their self-assessment. The paper also claims that temperature scaling and iterative prompt optimization reduce expected calibration error, with the combination sometimes working best.","pith_inferences":["The 10-sample all-or-nothing rule likely underestimates what a model knows; a model that knows an answer but occasionally samples a wrong token is counted as ignorant, so the reported overconfidence rates are probably upper bounds.","The out-of-distribution dataset assumes July 2024 news is unknowable to models released earlier, but pretraining corpora can contain near-duplicate or analogous facts, so TRUTHFUL there may partly reflect generic refusal behavior rather than a clean test of knowledge boundaries.","The larger closed models improve more from prompting, which suggests instruction-following ability rather than calibration itself drives the gain; separating the two would require comparing prompted truthfulness with logit-based confidence on the same items.","The modality asymmetry suggests an untested intervention: explicitly suppressing confidence when only image evidence is present could improve calibration beyond what the paper's prompt tuning achieves."],"forward_implications":["Visual fine-tuning does not systematically hurt calibration, so practitioners can add vision to a language model without expecting a calibration penalty, though they should still plan for overconfidence.","Multimodal training leaves linguistic-task calibration roughly unchanged, meaning the base LLM's uncertainty behavior largely carries over to the MLLM.","Supplying reliable text alongside an image reduces model uncertainty even for heavily noised images, so textual context is a practical lever for trustworthy confidence.","Explicit 'if you don't know, say so' prompts raise TRUTHFUL scores substantially on both in-distribution unknowns and out-of-distribution news questions.","Temperature scaling and prompt tuning both reduce ECE, and combining them sometimes yields further improvement."],"supporting_citations":[{"why":"Supplies the LLaVA models and visual instruction tuning that serve as the main open testbed.","marker":"Liu et al., 2023a"},{"why":"Supplies the Qwen-VL models, the second testbed for calibration comparisons.","marker":"Bai et al., 2023"},{"why":"Provides the repeated-answer construction for 'I don't know' labeling that the IDK dataset adapts to visual QA.","marker":"Cheng et al., 2024"},{"why":"Establishes the LLM calibration baseline that the paper's MLLM overconfidence results are contrasted with.","marker":"Kadavath et al., 2022"},{"why":"Supplies the logits-based likelihood uncertainty measure and the finding that aligned LLMs become overconfident, extended here to MLLMs.","marker":"He et al., 2023"},{"why":"Supplies semantic entropy, used to quantify uncertainty for open-ended answers.","marker":"Kuhn et al., 2023"},{"why":"Defines expected calibration error and temperature scaling, the paper's core metric and first calibration fix.","marker":"Guo et al., 2017"},{"why":"Provides the APE prompt-optimization framework used for iterative suffix tuning.","marker":"Zhou et al., 2022"}],"fun_headline_variants":["MLLM calibration stable across training; prompting boosts honesty","Multimodal LLMs say 'I don't know' only when prompted","Training doesn't fix MLLM miscalibration; prompts do","Prompting makes multimodal LLMs admit uncertainty","Calibration unchanged by fine-tuning; prompts improve it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 10 correct answers out of 10 sampled attempts cleanly separates what a model knows from what it does not, so a model that knows an answer but occasionally samples a wrong response is counted as not knowing.","fun_headline_variants_meta":{"raw":{"variants":["MLLM calibration stable across training; prompting boosts honesty","Multimodal LLMs say 'I don't know' only when prompted","Training doesn't fix MLLM miscalibration; prompts do","Prompting makes multimodal LLMs admit uncertainty","Calibration unchanged by fine-tuning; prompts improve it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1500,"prompt_tokens":933,"completion_tokens":567,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":494}},"tokens_in":549,"tokens_out":567,"duration_ms":3242,"temperature":1.0,"reasoning_tokens":494,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:01:30.715476+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-label the IDK questions with more samples (say 30 or 100) or with a lower accuracy threshold and recompute TRUTHFUL; if many questions labeled 'doesn't know' are answered correctly in 9 of 10 or 20 of 30 draws, the paper's overconfidence figures would be inflated.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Qwen-VL models, the second testbed for calibration comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the APE prompt-optimization framework used for iterative suffix tuning."}],"review_version":1}