{"id":"87bef1ef-8c7d-4376-a584-6434483b799b","arxiv_id":"2608.10964","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A confidence-aware reward inside GRPO, called CAR, jointly improves diagnostic accuracy and confidence calibration of a 7B medical VQA model on VQA-RAD, SLAKE, and PathVQA.","lead":"CARE is a training framework that makes a medical vision-language model state its answers with calibrated confidence, by rewarding confident correct answers and penalizing confident wrong ones. On three medical VQA benchmarks it reports the best accuracy, lowest expected calibration error, and lowest hallucination rate among the compared models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmark numbers may not measure held-out generalization: DCoT is synthesized from the same VQA datasets used for evaluation, and no train/test split is disclosed.","rationale":"The paper's central claim is an empirical one: CARE outperforms all baselines on three benchmarks. For that claim to be meaningful, the evaluation must be on samples the model did not train on. The paper's Section 2.2 describes synthesizing CoT trajectories from a general Medical-VQA dataset using the ground-truth answer as conditioning, and Section 3.1 lists the evaluation benchmarks. No split is defined anywhere in Section 2, Section 3, or the implementation details. This leaves open the possibility that DCoT contains the very questions and images used in Table 1. If so, the accuracy and calibration gains are largely an artifact of seeing the answers during training, and the hallucination-rate reduction could also reflect memorized answer patterns rather than improved reasoning. I considered other concerns: the CAR reward directly optimizes a quantity similar to ECE, the hallucination metric depends on an LLM judge, and no seeds or error bars are reported. These are real but secondary; they would affect the strength or interpretation of the results, whereas unresolved data leakage would invalidate the results entirely. The reader's weakest assumption points to the same issue, and I agree. The appropriate disposition is unchanged: the paper should remain conditional until the train/test split is disclosed and verified. The proposed concrete check is straightforward given the paper's promise to release code and data, and it would settle the concern either way.","tokens_in":7667,"tokens_out":3496,"duration_ms":37143,"concrete_test":"Compare the sample IDs (image/question pairs) in the released DCoT corpus and the SFT/GRPO training scripts against the official test splits of VQA-RAD, SLAKE, and PathVQA. If any test sample appears in training data, rerun Table 1 using only samples absent from DCoT, and report the revised ACC/ECE/HR. If no overlap exists, the concern is resolved and the headline claims stand as reported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that Table 1 reports held-out test performance. Section 2.2 describes synthesizing DCoT from a \"standard Medical-VQA dataset\" DVQA using ground-truth answers Yi as input to the generator and verifier (Eq. 1), and Section 3.1 evaluates on VQA-RAD, SLAKE, and PathVQA. The paper never states the train/test split used for DCoT synthesis, nor does it confirm that no evaluation sample appears in the SFT or GRPO training corpus. If DCoT was built from the full benchmarks and the model was evaluated on the same full benchmarks, the accuracy, ECE, and HR numbers in Table 1 would reflect memorization of answers and answer-specific confidence patterns rather than generalization. This concern is not an accusation of fraud; it is an unresolvable ambiguity in the experimental protocol as written. It also affects all three headline metrics simultaneously and precedes any concerns about LLM-judge bias or ECE being optimized directly, so it is the most load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CARE, a two-stage medical VQA training framework that combines synthetic Medical-CoT data for supervised fine-tuning with GRPO reinforcement learning using a Confidence-Aware Reward (CAR). The reward is intended to align answer-level confidence with diagnostic correctness: correct answers with high confidence are rewarded, while incorrect answers with high confidence are penalized. Experiments on VQA-RAD, SLAKE, and PathVQA report that CARE-7B achieves the best diagnostic accuracy, the lowest Expected Calibration Error, and the lowest Hallucination Rate among the compared medical reasoning MLLMs, including Med-R1, MedVLM-R1, MedMO, Lingshu-7B, MedVLThinker, and Fleming-VL. The central claim is that accuracy and calibration can be jointly improved without a trade-off.","tokens_in":7817,"tokens_out":3188,"duration_ms":31268,"significance":"If the reported results are valid, CARE would be a practically useful contribution: it demonstrates a concrete reward design for reducing confidence miscalibration in medical VQA, a safety-relevant issue that existing reasoning-focused medical MLLMs do not directly address. The paper includes a reasonable ablation of the two training stages, and the code is promised publicly. The claimed improvements are large and consistent across three benchmark datasets, which makes the result potentially important for the medical multimodal LLM community. However, the validity of the central claim rests on two load-bearing assumptions that are not currently established in the manuscript: that evaluation is performed on held-out test samples disjoint from the training data used to synthesize DCoT, and that the reported ECE reduction reflects a genuine improvement in calibration rather than an artifact of optimizing the same confidence measure used for evaluation. The method itself is simple and reproducible, and the weaknesses identified here appear addressable with additional experiments and protocol disclosure rather than being fatal to the approach.","major_comments":[{"comment":"The source dataset D_VQA is never concretely identified, and the paper never states whether DCoT was synthesized from training splits or from the full VQA-RAD, SLAKE, and PathVQA benchmarks. Since DCoT is used for both SFT and GRPO and the same benchmarks are used for evaluation, the accuracy, ECE, and HR gains in Table 1 could reflect memorization of ground-truth answers and answer-specific confidence patterns rather than held-out generalization. Please specify the exact datasets used for synthesis, disclose the train/test split for each benchmark, and confirm that no evaluation sample appears in either training phase.","section":"Section 2.2, Eq. (1) and Section 3.1, Table 1"},{"comment":"The reported ECE improvement is substantially by construction. Eq. (5) defines answer-level confidence C(a_i), Eq. (6) uses C(a_i) to reward confident correct predictions and penalize confident incorrect predictions, and Eq. (7) computes ECE from the same confidence measure. Consequently, the training objective directly minimizes the quantity used for evaluation. To support the calibration claim, the paper should provide an independent test of calibration, for example by evaluating ECE with temperature scaling, a held-out confidence estimator, or token-level probabilities that are not part of the CAR objective.","section":"Section 2.4, Eqs. (5)-(7)"},{"comment":"The hallucination rate metric depends entirely on a single VLM judge, Lingshu-32B, with no validation of that judge against human annotation or against the ground-truth answers; the HR formula in Eq. (8) is also not fully specified because the normalization constant C and the role of the token-count weighting t_i are not defined precisely. Since Table 1 reports HR as a headline result, please either validate the judge's reliability on a sample of human-labeled trajectories or report simpler raw hallucination rates in addition to the weighted score.","section":"Section 3.1, Eqs. (7)-(8), Table 1"}],"minor_comments":[{"comment":"The notation D_VQA is introduced as \"a standard Medical-VQA dataset,\" but the actual dataset name is never given; please state which dataset or datasets were used for Medical-CoT synthesis and clarify the relation to the three evaluation benchmarks.","section":"Section 2.2"},{"comment":"The format reward R_form is mentioned but never formally defined; please specify exactly how the <think> and <answer> delimiters are checked and how this reward is applied to open-ended answers with variable lengths.","section":"Section 2.4"},{"comment":"The open-ended Recall metric is not defined; please describe how coverage of the ground-truth answer Y within the predicted answer a_i is computed and how this interacts with the exact-match indicator for closed-ended questions.","section":"Section 3.1"},{"comment":"The confidence histograms in Figure 3 lack axis labels and units, and it is not clear whether the \"pre\" and \"post\" distributions come from the exact checkpoints used in Table 1; please clarify the checkpoint selection and make the figure self-contained.","section":"Section 3.3, Figure 3"},{"comment":"The formula for HR contains an apparent formatting artifact (\"C E h\") and the hat notation is unexplained; please restate the equation cleanly and define all symbols.","section":"Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the journal and the proposed method is plausible, but the missing train/test disclosure is a serious protocol ambiguity that affects all three headline metrics. The ECE-by-construction concern is also central to the paper's main contribution. Both issues are fixable through additional experiments and disclosure, so I recommend major revision rather than rejection. I would also encourage the editor to ask the authors to make the evaluation code and data splits available for verification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a sensible reward-shaping idea with a clean write-up, but the main empirical claim is unverifiable as written because the authors never say where the training data came from relative to the test sets.\n\nWhat's actually new: the Confidence-Aware Reward (Eq. 6) is a genuine new combination. It conditions answer-level confidence on correctness inside GRPO—rewarding confident correct answers, penalizing confident wrong ones—which none of the cited RFT baselines do. The reverse-thinking CoT synthesis (generate reasoning from ground-truth answer, then filter with a verifier) is also a practical data construction recipe. The paper is clearly written and the framework is coherent.\n\nSoft spots, in order of severity. First and load-bearing: the paper never discloses the train/test split. Section 2.2 synthesizes DCoT from a standard Medical-VQA dataset DVQA, using ground-truth answers Yi to generate reasoning. Section 3.1 evaluates on VQA-RAD, SLAKE, and PathVQA. The text does not state whether DCoT was built from the full benchmarks or from a separate training subset, nor does it confirm that no evaluation sample appears in SFT or GRPO. If the same question-answer pairs were used for both, Table 1 would reflect memorization rather than generalization. The reader's stress-test is right to call this the most load-bearing concern; it hits all three headline metrics at once. I'm not accusing the authors of fraud—it may well be a standard split—but the protocol as written doesn't allow the reader to distinguish.\n\nSecond: the ECE improvement is partly by construction. Eq. 5 defines confidence, Eq. 6 optimizes that same confidence conditioned on correctness, and Eq. 7 measures ECE from that same confidence. That doesn't make it trivial—Rcalib is a legitimate training signal—but the \"lowest ECE\" headline is less impressive when you realize you're directly optimizing the metric.\n\nThird, minor: no error bars or seeds are reported, and the hallucination judge (Lingshu-32B) is used without validation against human ratings. These are standard requirements and easy to fix.\n\nWho this is for: researchers working on calibration in medical VQA, or on reward design for RFT. The method is plausible and worth exploring. But the empirical claims need the split disclosure and more rigorous evaluation before they can be taken at face value.\n\nMy recommendation: send it to peer review, and require the authors to report the exact train/test split, confirm no overlap, provide multiple seeds with variance, and validate the LLM judge. The idea deserves a serious referee; the current write-up just doesn't support the headline numbers.","headline":"The confidence-aware reward is a real idea, but the paper's headline numbers may be memorization because the train/test split is never disclosed.","tokens_in":8404,"tokens_out":2981,"would_cite":true,"duration_ms":25411,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CARE claims that a correctness-conditioned confidence reward lets one 7B medical VQA model lead all benchmarks in accuracy, calibration, and hallucination rate.","keywords":["medical visual question answering","confidence calibration","reinforcement fine-tuning","group relative policy optimization","chain-of-thought","expected calibration error","hallucination","multimodal large language models"],"falsifier":"Resolve the train/eval overlap question: enumerate the image-question pairs used to build DCoT, intersect them with the evaluation sets, and rerun CARE on the disjoint portion. If any benchmark query was used for training, or if the reported gains shrink on a clean split, the central claim of simultaneous improvement is not established.","tokens_in":7445,"feed_emoji":"🩺","tokens_out":7901,"duration_ms":69471,"temperature":0.7,"pith_summary":"Confidence-aware medical reasoning (CARE) is a training recipe for medical visual question answering that claims to remove the usual trade-off between accuracy and calibration. The paper argues that the reason current reinforcement-fine-tuned medical models are untrustworthy is not lack of reasoning but a gap between expressed confidence and actual correctness. It therefore builds a two-stage pipeline: a scalable synthesis of verified chain-of-thought traces for supervised fine-tuning, then GRPO with a reward that rewards confident correct answers and penalizes overconfident wrong ones. On VQA-RAD, SLAKE, and PathVQA, the trained 7B model reports the highest accuracy together with the lowest expected calibration error and hallucination rate among the compared models. If the claim holds, confidence can be optimized as a first-class training objective rather than measured after the fact.","feed_headline":"One reward aligns medical VQA confidence with accuracy","feed_subtitle":"CARE's two-stage training beats six baselines on accuracy, calibration, and hallucination on three benchmarks.","key_machinery":"The Confidence-Aware Reward (CAR) is the object that carries the argument: it computes answer-level confidence $C(a_i) = \\frac{1}{|a_i|} \\sum_{j=1}^{|a_i|} \\pi_\\theta(t_j \\mid V, Q, t_{<j})$ over only the tokens inside the `<answer>` span, then combines it with the correctness indicator through $R_{\\mathrm{calib}} = R_{\\mathrm{out}} C(a_i) - \\lambda (1-R_{\\mathrm{out}}) C(a_i)$. CAR plugs into the GRPO advantage estimator, replacing scalar accuracy rewards with a composite reward $R_{\\mathrm{form}} + R_{\\mathrm{out}} + R_{\\mathrm{calib}}$. Its role is to make calibration a differentiable, optimizable target inside policy gradient, while the auxiliary Medical-CoT synthesis provides the structured cold start that makes answer tokens extractable and verifiable.","core_discovery":"The paper's central discovery is that tying the model's confidence to its correctness in the reward signal lets a single 7B model win on accuracy, calibration, and hallucination simultaneously, where every baseline trades one off against another. The key result is Eq. (6): for each candidate answer, confidence $C(a_i)$ is the mean probability of the tokens in the answer span, and the calibration reward is $R_{\\mathrm{calib}} = R_{\\mathrm{out}} C(a_i) - \\lambda (1-R_{\\mathrm{out}}) C(a_i)$ with $\\lambda = 0.5$. A correct high-confidence answer is rewarded; an incorrect high-confidence answer is penalized. Inside GRPO, this composite reward is normalized into group-relative advantages, so the policy learns to say \"I am confident\" only when it is right. Reported numbers across VQA-RAD, SLAKE, and PathVQA show accuracy 0.767, 0.873, and 0.689, ECE 0.202, 0.115, and 0.290, and hallucination rates 0.048, 0.070, and 0.059, all best among the compared models. Ablations further show that closed-ended questions improve most from RL alone while open-ended questions need the SFT cold start.","pith_inferences":["The same conditional calibration reward could transfer to any verifiable question-answering setting outside medicine, such as general science or legal QA, whenever a correctness check is available; the paper sketches this direction in its conclusion but does not test it.","The answer-span average token probability is one possible confidence estimator; replacing it with a calibrated variant, such as sampling-based agreement or verbalized confidence, would test whether CARE's gains are tied to this specific definition.","The hallucination metric relies on a judge model's normalized score, so a natural follow-up experiment is to have expert clinicians rate the same outputs and tell whether the lower hallucination rate reflects factual grounding rather than adherence to a preferred output format."],"forward_implications":["A single training objective can lift accuracy and calibration together, contradicting the usual observation that strong medical reasoning models sacrifice one for the other.","Open-ended diagnostic questions rely on the synthesized chain-of-thought cold start; without it, RL alone does not reach the same open-ended accuracy.","Closed-ended questions benefit most from the RL phase alone, where the constrained answer space lets confidence-aware exploration converge without SFT initialization.","Reasoning traces generated backward from the ground-truth answer and then filtered by a verifier reduce hallucination at the data-construction stage.","The framework's reward design needs only a verifiable correctness signal, so it is portable to any medical VQA benchmark with ground-truth answers."],"supporting_citations":[{"why":"Supplies the Qwen2.5-VL base model used as both the CoT synthesizer and the policy model.","marker":"[1]"},{"why":"Defines the GRPO objective whose group-relative advantages carry the Confidence-Aware Reward.","marker":"[16]"},{"why":"Defines Expected Calibration Error, the calibration metric CARE reports and optimizes against.","marker":"[6]"},{"why":"The VQA-RAD radiology benchmark used for evaluation.","marker":"[11]"},{"why":"The SLAKE semantically labeled benchmark used for evaluation.","marker":"[14]"},{"why":"The PathVQA pathology benchmark used for evaluation.","marker":"[7]"},{"why":"MedVLM-R1, a key medical RFT baseline whose confidence miscalibration motivates the paper and that CARE is compared against.","marker":"[15]"},{"why":"Med-R1, a reinforcement-fine-tuned medical reasoning baseline included in the comparisons.","marker":"[9]"}],"fun_headline_variants":["Medical VQA: one reward aligns confidence with accuracy","CARE: confidence-aware reward wins on medical VQA","Tie confidence to correctness: new medical VQA reward","Single reward fixes medical VQA confidence gap","CARE: reward confidence only when the answer is right"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the accuracy and calibration numbers come from held-out test questions, not from the same VQA-RAD, SLAKE, and PathVQA instances that were used to synthesize the training reasoning trajectories.","fun_headline_variants_meta":{"raw":{"variants":["Medical VQA: one reward aligns confidence with accuracy","CARE: confidence-aware reward wins on medical VQA","Tie confidence to correctness: new medical VQA reward","Single reward fixes medical VQA confidence gap","CARE: reward confidence only when the answer is right"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001012,"raw_usage":{"total_tokens":4310,"prompt_tokens":1015,"completion_tokens":3295,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":3218}},"tokens_in":631,"tokens_out":3295,"duration_ms":24831,"temperature":1.0,"reasoning_tokens":3218,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:17:26.005955+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Resolve the train/eval overlap question: enumerate the image-question pairs used to build DCoT, intersect them with the evaluation sets, and rerun CARE on the disjoint portion. If any benchmark query was used for training, or if the reported gains shrink on a clean split, the central claim of simultaneous improvement is not established.","supporting_citations":[{"cited_title":"In: International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Defines Expected Calibration Error, the calibration metric CARE reports and optimizes against."},{"cited_title":"Scientific Data5(180251) (2018)","cited_arxiv_id":null,"evidence_quote":"The VQA-RAD radiology benchmark used for evaluation."},{"cited_title":"In: IEEE 18th International Symposium on Biomedical Imaging","cited_arxiv_id":null,"evidence_quote":"The SLAKE semantically labeled benchmark used for evaluation."}],"review_version":1}