{"id":"538a298b-8482-45e3-96cd-0f957efd6a25","arxiv_id":"2505.23224","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MMBoundary trains multimodal LLMs to express per-step natural-language confidence and uses reinforcement learning with three rewards to calibrate those statements, cutting calibration error by 7.5% and boosting task accuracy by up to 8.3%.","lead":"A new training method teaches multimodal AI models to state how confident they are after each step of their reasoning, instead of only at the end. This reduces confidently wrong answers and improves accuracy on visual question answering benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Internal confidence estimator in Eq. 7 combines CLIPScore with the wrong sign: high image-text alignment is treated as high uncertainty, so warm-up labels and RCS may reward confidently-wrong statements.","rationale":"The reader's weakest assumption is the validity of the internal confidence estimator, and I agree that this is the load-bearing point: the same estimator creates every warm-up label and defines the RCS reward, so if it is miscalibrated the model is trained to express confident-sounding statements that track the wrong signal. I found a concrete reason to take the concern seriously: under the paper's own convention that UFinal=0 means certainty, UCLIPScore enters Eq. 7 with the wrong sign. Raw CLIPScore is a similarity score, so a high value means strong image-text alignment and should lower uncertainty, not raise it. The only validation in Sec 5.2 compares ICE to self-consistency on 50 samples, which is neither a ground-truth correctness check nor large enough to expose such a systematic sign error. This does not prove the method fails: the REC reward can still push expressed confidence toward correctness, and the ablations show each signal contributes. However, because the sign issue is an internal inconsistency, the paper should either correct Eq. 6/7 or demonstrate that the sign convention does not affect results. The paper has genuine support in released code, systematic ablations, human evaluation, and a second-backbone experiment, so the conditional status is appropriate; a targeted sign-flip or code inspection would settle whether the proposed self-calibration signal is actually valid.","tokens_in":20088,"tokens_out":11969,"duration_ms":135227,"concrete_test":"Inspect the released code for the implementation of Eq. 7; if raw CLIPScore is used, re-run the warm-up and RL pipeline with UCLIPScore replaced by (1 - normalized CLIPScore), keeping all other hyperparameters fixed, and recompute MECE/ECE on A-OKVQA and ScienceVQA. If the sign-flipped variant improves MECE by more than 1 point, the internal confidence signal and RCS reward are miscalibrated in a way that affects the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1.1 defines UFinal = w0 ULNLP + w1 UMTE + w2 UTokenSAR + w3 UCLIPScore and states that the closer UFinal is to 0, the greater the certainty. The first three terms are uncertainty measures where low values mean high certainty, but UCLIPScore = max(cos(v,s),0) is a similarity score: high values mean the generated sentence is strongly aligned with the image, i.e., low uncertainty. Under the paper's own convention, Eq. 7 should contain (1 - normalized CLIPScore), not the raw score. As written, a well-grounded sentence receives a high UFinal and is mapped to a low confidence level, while a poorly grounded sentence can receive high confidence. This reversed signal is used to create the 55K warm-up labels and to compute RCS (Eq. 12), which trains the model to align expressed confidence with this miscalibrated internal score. The only validation in Sec 5.2 compares ICE to self-consistency on 50 samples; self-consistency is not ground-truth correctness and 50 samples cannot expose a systematic sign error. If the sign is wrong, the model may be trained to express least confidence exactly where its statement is most supported by the image.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MMBoundary, a two-stage framework that trains multimodal large language models (MLLMs) to emit per-sentence natural-language confidence statements. In the warm-up stage, an internal confidence estimator combines length-normalized log probability, mean token entropy, TokenSAR, and CLIPScore into a scalar UFinal (Eq. 7), which is mapped to one of five confidence levels; the corresponding confidence statements are inserted into the model's responses and the model is fine-tuned on 55K such sentences. In the reinforcement learning stage, PPO is applied with three rewards: knowledge accuracy (RKA, Eq. 10), expected calibration (REC, Eq. 11), and confidence self-calibration (RCS, Eq. 12). Experiments on A-OKVQA, ScienceVQA, and CulturalVQA report reduced ECE/MECE, improved accuracy and Reasoning Chain F1, and higher human evaluation scores, with ablations and a Qwen2VL transfer experiment.","tokens_in":20305,"tokens_out":8048,"duration_ms":84129,"significance":"If the technical details hold, the paper addresses a genuine gap: prior confidence-calibration methods operate at the response level and cannot localize which reasoning step is unreliable. The claimed average 7.5% MECE reduction and up to 8.3% task improvement on an out-of-distribution dataset would be practically important for trustworthy multimodal reasoning. Strengths include public code, extensive ablations, human evaluation with inter-annotator agreement, and validation on a second backbone. However, the core internal-confidence estimator contains apparent sign errors that are load-bearing for both the warm-up labels and the RCS reward, and the only validation of the estimator (Section 5.2) is too weak to detect those errors. The empirical results therefore cannot be fully trusted as written.","major_comments":[{"comment":"Two of the four terms in Eq. (7) enter with the wrong sign under the paper's own convention. Eq. (3) defines ULNLP = exp(-(1/L) log P(y|x,theta)) = P(y|x,theta)^(1/L), which is a probability-like score; larger values mean higher probability and thus higher confidence, not lower uncertainty. Eq. (6) defines UCLIPScore = max(cos(v,s),0), which is a similarity score; larger values mean stronger image-text alignment and thus higher confidence. Yet Eq. (7) adds both with positive weights to UFinal, with the explicit statement that the closer UFinal is to 0, the greater the certainty. Consequently, a well-grounded, high-probability sentence receives a high UFinal and is mapped to a low confidence level, while a poorly grounded sentence can receive a high confidence level. Because UFinal determines the 55K warm-up labels and the RCS reward in Eq. (12), the training signal is reversed for two of the four components. The authors should correct Eqs. (3) and (6)–(7) or state the alternative convention explicitly, and re-run the experiments; if the code uses different definitions, the paper must say so.","section":"Section 3.1.1, Eqs. (3), (6), (7)"},{"comment":"The validation of the internal confidence estimator (ICE) is insufficient for the claim being made. It uses only 50 randomly sampled examples, reports no error bars, confidence intervals, or significance tests, and compares ICE against self-consistency-based confidence (SCE) rather than against ground-truth correctness. A systematic sign error in Eq. (7) would not be exposed by a small average absolute difference between two noisy confidence estimates. The paper should validate ICE directly against per-sentence correctness (e.g., AUROC for distinguishing correct from incorrect reasoning steps) with confidence intervals, and should also clarify the statement that the method has 'about 6 instances of deviation': with 50 samples and an average |CICE - CSCE| of 0.0578, the total deviation is about 2.89, not 6, unless 'instances' means something else.","section":"Section 5.2, Table 3"},{"comment":"The out-of-distribution claim rests on a test set of 800 CulturalVQA samples that were 'manually selected and annotated' by the authors. The selection criteria are not described, and no evidence is provided that the selected subset is representative of the full dataset rather than cherry-picked. Because the OOD generalization result is one of the main contributions (Table 1), the authors should document the selection procedure, provide the annotation instructions, and ideally use a predefined public split or multiple random subsets with variance estimates.","section":"Section 4.2 / Appendix B (CulturalVQA)"}],"minor_comments":[{"comment":"The confidence-level interval specification is ambiguous: 'Confidence levels Cv from 5 to 1 correspond to the intervals of UFinal as [0, mu - sigma, mu + sigma, mu + 2sigma, mu + 3sigma, 1]' lists six endpoints for five intervals. Please state the five intervals separately and specify explicitly which interval maps to which confidence level.","section":"Section 3.1.1"},{"comment":"The human evaluation claims 'statistically significant improvements' but no p-values, standard deviations, or details of the statistical test are reported for the three dimensions (Faithful, Concise, Granular). Please provide this information.","section":"Section 4.3 and Table 2"},{"comment":"For the Qwen2VL experiments, the paper should specify which CLIP model is used to compute CLIPScore, since Qwen2VL's vision encoder is not a CLIP ViT; otherwise it is unclear how Eq. (6) is applied to that backbone.","section":"Appendix E"},{"comment":"There are several typographical and referencing issues: 'Se cion' in Section 3.2; the 'Step Matched' example is referenced as Figure 6 without defining the matching procedure in the main text; and the sentence about 'about 6 instances of deviation' in Section 5.2 is numerically unclear. Please clean these up.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The apparent sign errors in Section 3.1.1 are the critical issue. If the equations match the code, the training objective is partially reversed for two of the four confidence signals; if the equations do not match the code, the paper needs a correction and a re-run. Either way, the current text is not publishable as is. I recommend major revision rather than rejection because the issue is localized and correctable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: MMBoundary is a serious, well-engineered piece on sentence-level confidence calibration for multimodal LLMs, and it should go to review. But as written, Eq. 7 has a likely sign error that could invert the CLIPScore confidence signal, and the paper's only direct validation of its internal confidence estimator is too weak to catch it.\n\nThe genuinely new thing is combining text-based internal signals (log prob, entropy, tokenSAR) with a cross-modal CLIPScore, and using a mutual score-statement mapping to generate per-step natural-language confidence, then applying a three-reward RL stage. The experiments are extensive: three VQA datasets, two base models, ablations of each component, human evaluation, and public code. The authors are upfront about the core assumption, and the ablations show each signal contributes.\n\nThe soft spots, in order of seriousness. First, the sign issue. They define UCLIPScore = max(cos(v,s),0), a similarity score where higher means the sentence matches the image. But Eq. 7 sums it with the three uncertainty terms, and the paper says the closer UFinal is to 0, the greater the certainty. Under that convention, a well-grounded sentence gets a high UFinal and is mapped to a low confidence level; a poorly grounded sentence can get high confidence. The fix would be using (1 - normalized CLIPScore) in the sum. If the implementation matches the paper, both the 55K warm-up labels and the RCS reward (Eq. 12) train the model to express low confidence exactly where the image supports it. The validation in Sec 5.2 compares ICE to self-consistency on 50 samples—self-consistency isn't correctness, and 50 points can't surface a systematic sign flip. Second, the OOD test set is manually selected 800 samples from CulturalVQA with no stated sampling criteria; that weakens the generalization claim somewhat, though not fatally. Third, the circularity concern: RCS aligns expressed confidence to an internal signal derived from the model itself. That's real, but the external RKA and REC rewards based on reference chains provide an independent anchor, so I don't think it's a fundamental flaw.\n\nThe paper deserves a serious referee. I'd send it out, but the referee instructions should explicitly ask the authors to clarify the sign in Eq. 7, ideally with a corrected equation and, if the implementation differs, a statement of what the code actually does. If the sign issue is confirmed, the main numbers would need to be re-examined, but the framework itself is still worth engaging with.","headline":"Well-engineered step-level confidence calibration for MLLMs, with a likely sign error in the CLIPScore term that needs checking before its results can be trusted.","tokens_in":20870,"tokens_out":3050,"would_cite":false,"duration_ms":31729,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that teaching multimodal large language models to attach a natural-language confidence statement to every sentence of their reasoning, rather than one overall score, cuts calibration error and improves answer accuracy.","keywords":["multimodal large language models","confidence calibration","reasoning step confidence","knowledge boundary awareness","reinforcement learning","uncertainty estimation","self-correction","hallucination mitigation"],"falsifier":"Take a held-out set where each reasoning step’s correctness is known, and check whether the four internal signals rank correct steps above incorrect ones; if they do not, MMBoundary’s warm-up labels and self-calibration reward are teaching the model to express a signal that is not its true uncertainty, and the apparent calibration gains would have to be re-attributed to the reinforcement-learning structure rather than to the confidence measurement.","tokens_in":19873,"feed_emoji":"🎯","tokens_out":7682,"duration_ms":69320,"temperature":0.7,"pith_summary":"The paper tries to fix a failure mode of multimodal large language models: they can be wrong at one early visual-perception step and still announce high confidence in the final answer. The proposed method, MMBoundary, trains the model to append a short natural-language confidence statement to every sentence it generates, so the model says “I’m not sure” exactly where it is actually unsure. Confidence labels for training come from the model’s internal signals — token log probabilities, token entropy, relevance weighting, and image–text agreement — and are refined by reinforcement learning with three rewards: knowledge accuracy, expected calibration, and consistency between expressed and internal confidence. The paper reports an average 7.5% reduction in multimodal confidence calibration error and up to 8.3% improvement in task performance on three visual question-answering datasets, including one outside the training distribution.","feed_headline":"Confidence per reasoning step cuts multimodal AI calibration error by 7.5%","feed_subtitle":"Teaching vision-language models to say \"I'm not sure\" after each sentence helps them self-correct and answer better.","key_machinery":"The load-bearing component is the confidence estimation module, which measures per-sentence uncertainty from the model’s internal state: length-normalized log probability, mean token entropy, tokenSAR (relevance-weighted token likelihood), and CLIPScore (image–sentence agreement). These signals are min-max normalized, weighted equally, and mapped to five confidence levels using the dataset-wide mean and standard deviation. The second piece is the mutual score–statement mapping: each level has a pool of 40 preset natural-language statements, so the model can be trained to say them and, at reinforcement-learning time, have its expressed statements converted back into scores via encoder cosine similarity. The third piece is the reinforcement learning stage, which optimizes a weighted sum of three rewards — knowledge accuracy against an automatically annotated reference chain, expected calibration against step correctness, and self-calibration against the internal signal — using PPO.","core_discovery":"The central discovery is that per-step confidence calibration works: an MLLM can learn to state, in natural language, how confident it is in each sentence of its reasoning chain, and training it to do so narrows the gap between expressed confidence and actual step correctness. The framework estimates a confidence score for each generated sentence from a weighted blend of four signals — length-normalized log probability, mean token entropy, tokenSAR, and CLIPScore — then bins scores into five confidence levels and maps each level to a pool of preset statements such as “with total certainty” or “but I’m not entirely sure.” The model is first fine-tuned to emit these statements, then trained with PPO so the expressed confidence is pulled toward both the internal signal and the actual correctness of each step, while a knowledge-accuracy reward keeps the reasoning chain faithful to an annotated reference chain. The paper’s empirical results show average reductions in confidence calibration error of 7.5% and up to 8.3% improvement in task performance across A-OKVQA, ScienceVQA, and CulturalVQA.","pith_inferences":["The paper does not test decoding-time use, but its natural-language confidence tags could be consumed during generation to re-sample or revise any sentence the model marks uncertain before continuing the chain.","The same per-step calibration recipe could be applied to text-only language models or to agentic loops where each tool call is a link in the chain, though the paper only demonstrates visual question answering.","A stress test the paper does not run is to train with deliberately corrupted internal confidence labels: if calibration gains persist, the reward design, not the internal signal, is doing the work.","The five-level statement pools are a discrete interface; a wider or continuous confidence vocabulary might capture more of the internal signal, but it would make the statement-to-score mapping harder."],"forward_implications":["If MMBoundary works as reported, MLLMs can be trained to flag uncertain steps during inference, letting a system retry or revise those steps before finalizing an answer.","Unlike whole-response confidence scores, the per-step statements give a direct handle on where a reasoning chain begins to go wrong, not just that it ended wrong.","The method transfers to a different backbone model and to an out-of-distribution dataset, indicating the calibration behavior is not tied to one model.","Even without the reinforcement learning stage, the supervised warm-up alone improves over baselines; the three rewards add further calibration gains.","Reasoning-chain F1 and human ratings of faithfulness and granularity improve alongside calibration, so the model is not trading accuracy for better-sounding uncertainty."],"supporting_citations":[{"why":"Supplies the mean token entropy uncertainty signal used in the confidence estimation module.","marker":"Fomicheva et al., 2020"},{"why":"Supplies tokenSAR, the relevance-weighted token likelihood signal for per-sentence uncertainty.","marker":"Duan et al., 2024"},{"why":"Supplies CLIPScore, the cross-modal image–sentence agreement signal.","marker":"Hessel et al., 2021"},{"why":"SaySelf is the closest baseline and the source of the expected calibration reward, which MMBoundary extends from whole-response to sentence-level confidence.","marker":"Xu et al., 2024"},{"why":"Provides the PPO algorithm used in the reinforcement learning stage.","marker":"Schulman et al., 2017"},{"why":"Supplies A-OKVQA, one of the in-domain evaluation datasets.","marker":"Schwenk et al., 2022"},{"why":"Supplies ScienceVQA, the other in-domain evaluation dataset.","marker":"Lu et al., 2022"},{"why":"Supplies CulturalVQA, the out-of-distribution evaluation dataset.","marker":"Nayak et al., 2024"},{"why":"Defines the Expected Calibration Error metric used for the main calibration results.","marker":"Guo et al., 2017"},{"why":"Defines the Reasoning Chain F1 metric used to evaluate reasoning-chain quality.","marker":"Ho et al., 2022"}],"fun_headline_variants":["Stepwise confidence calibration cuts AI calibration error by 7.5%","Per-step AI self-doubt improves multimodal reasoning accuracy","MMBoundary teaches MLLMs to hedge each step, reducing errors","Reinforcement learning for per-step confidence boosts MLLM performance","Vision-language AI calibrates confidence per step, slashes mistakes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the model’s internal signals — token likelihoods, entropy, relevance weighting, and image–text agreement — genuinely measure how certain the model is about each reasoning step, because those signals provide the training labels for every confidence statement the model learns to say.","fun_headline_variants_meta":{"raw":{"variants":["Stepwise confidence calibration cuts AI calibration error by 7.5%","Per-step AI self-doubt improves multimodal reasoning accuracy","MMBoundary teaches MLLMs to hedge each step, reducing errors","Reinforcement learning for per-step confidence boosts MLLM performance","Vision-language AI calibrates confidence per step, slashes mistakes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000424,"raw_usage":{"total_tokens":2205,"prompt_tokens":1007,"completion_tokens":1198,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":1109}},"tokens_in":623,"tokens_out":1198,"duration_ms":10250,"temperature":1.0,"reasoning_tokens":1109,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:50:05.070381+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set where each reasoning step’s correctness is known, and check whether the four internal signals rank correct steps above incorrect ones; if they do not, MMBoundary’s warm-up labels and self-calibration reward are teaching the model to express a signal that is not its true uncertainty, and the apparent calibration gains would have to be re-attributed to the reinforcement-learning structure rather than to the confidence measurement.","supporting_citations":[],"review_version":1}