{"id":"24101d29-ee21-47d1-b833-fedc1532992a","arxiv_id":"2412.13333","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Fine-tuned vision-language models make more correct predictions, but a larger share of those correct predictions are based on invalid evidence than zero-shot models.","lead":"Fine-tuning vision-language models improves accuracy but can make correct predictions rely more often on invalid image evidence. The study introduces two metrics to separate being right from being right for the right reasons, and finds both trade-offs hold across datasets and corruptions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RMA > 0.5 valid-evidence threshold is not object-size calibrated; for bounding-box masks covering more than half the image it labels uniform attention as valid, so the PT/IR comparisons may be an artifact.","rationale":"The reader's weakest-assumption analysis correctly identifies the fixed 0.5 RMA threshold as the most load-bearing premise. The central claim that fine-tuning 'led to more correct predictions based on invalid evidence' depends entirely on the validity labels assigned to explanations. Since RMA is a mass ratio, the threshold's meaning is determined by the ground-truth mask's area fraction; a uniform heatmap crosses 0.5 whenever the mask covers more than half the image. This is not a minor calibration issue: ImageNet and Stanford-Dogs use bounding boxes, which are often large, and the method's own metric therefore cannot distinguish 'attended to the object' from 'attended to the whole image.' The absence of error bars and the presence of several Table 2 exceptions to the claimed PT decrease further weaken the inference, but the threshold artifact is the primary threat because it can change the sign of the comparison. I do not recommend rejection because the issue is empirically checkable: a mask-area-normalized threshold or stratification by mask size can determine whether the qualitative ordering survives. Since the reader already issued a CONDITIONAL verdict and my concern is the same concern, no verdict change is needed.","tokens_in":17299,"tokens_out":7356,"duration_ms":71206,"concrete_test":"Re-run the full Table 2 pipeline with a chance-normalized validity rule: for each image compute p = |mask|/|image| and define valid evidence as RMA > p + delta (or equivalently use normalized RMA = (RMA-p)/(1-p) with a sweep of deltas such as 0.1, 0.25, 0.5), instead of the fixed RMA > 0.5. Additionally, stratify PT and IR by mask-area bins (p<0.25, 0.25-0.5, >0.5) for each method. If the average ordering ZS > LP/FLCP/FT for PT and FT > ZS for IR persists across all bins and thresholds, the threshold concern is not load-bearing; if the ordering flips or vanishes in large-mask bins, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the binary mapping from RMA to 'valid evidence' in 'Our Proposed Evaluations': RMA(H,M) = sum(H*M)/sum(H) >= 0.5 is called valid. RMA is a mass ratio, so for a constant heatmap it equals the fraction p of image pixels inside the ground-truth mask. For ImageNet and Stanford-Dogs the masks are bounding boxes, and when a box covers more than half the image a non-informative uniform explanation is automatically scored as valid evidence. This makes RW (right prediction, wrong rationale) nearly impossible for those images by construction, inflating PT for every method; for small-mask images (e.g., CUB segmentations) the same 0.5 rule can mark reasonable localizations as invalid. Because the correct-prediction sets of ZS, LP, FLCP, and FT differ, the PT and IR comparisons are not comparisons of evidence quality but are confounded by the distribution of mask sizes across each method's correct and wrong predictions. The paper's own supplementary note that results come from a single run means there is no variance estimate to help separate this artifact from a real effect. If a fine-tuned model's accuracy gain is concentrated in large-mask images, its PT is inflated; if concentrated in small-mask images, its PT is deflated. The observed ordering could flip once the threshold is adjusted for object size.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two metrics, Prediction Trustworthiness (PT) and Inference Reliability (IR), to evaluate whether vision-language model predictions are grounded in valid evidence. PT is the fraction of correct predictions whose attention heatmap has RMA ≥ 0.5; IR is the fraction of predictions that are correct among those with RMA ≥ 0.5. Across CLIP, ALBEF, and BLIP models on ImageNet, CalTech-101, Stanford-Dogs, and CUB-200-2011, and for zero-shot, linear probing, FLCP, and full fine-tuning, the paper reports that fine-tuning lowers PT but raises IR, and claims these patterns persist under ImageNet-C corruption and across optizer, learning-rate, explanation-method, prompt-tuning, and adapter-tuning ablations.","tokens_in":17533,"tokens_out":6722,"duration_ms":56876,"significance":"If the central finding is robust, the paper adds a useful evaluation dimension to VLM fine-tuning: accuracy gains may come with a trade-off in the rationality of the evidence, a point that is relevant to safety-critical deployments. The paper contributes two simple, easy-to-compute metrics, releases code, and includes a broad experimental matrix with multiple models, methods, datasets, ablations, and OOD tests. However, the quantitative claim currently rests on an arbitrary validity threshold and a single run per configuration, so its significance is contingent on the sensitivity and uncertainty analyses described in the major comments.","major_comments":[{"comment":"The binary valid/invalid evidence label is assigned by the single threshold RMA ≥ 0.5, with no calibration to object size. For the bounding-box masks used in ImageNet, ImageNet-C, and Stanford-Dogs, RMA of a uniform heatmap equals the fraction of image pixels inside the box; when that fraction exceeds 0.5, a non-informative explanation is automatically scored as valid evidence. Conversely, for the small segmentation masks in CUB-200-2011, a diffuse but object-covering heatmap can fall below 0.5. Because the correct-prediction sets of ZS, LP, FLCP, and FT differ, the PT and IR differences in Tables 2–4 are confounded by the distribution of mask sizes within each method's correct and wrong prediction sets. The paper does not provide a sensitivity analysis over thresholds, mask-size normalization, or the distribution of RMA scores. I request that the authors report PT and IR as functions of the RMA threshold (e.g., 0.3, 0.4, 0.6, 0.7), normalize RMA by the mask fraction, or otherwise demonstrate that the ordering of methods is not an artifact of the uncalibrated 0.5 cut-off.","section":"Our Proposed Evaluations / Eq. (1)"},{"comment":"The supplementary states 'We provide the results from a single run.' No confidence intervals, error bars, or significance tests appear in any table or figure in the paper. Because the central claims are quantitative comparisons (for example, ImageNet PT differences of 4.07–6.40 percentage points in Table 2), a single run cannot distinguish the reported pattern from optimization or evaluation noise. Please provide multiple seeds with error bars, or a paired statistical test over the same images, for at least the main tables.","section":"Supplementary: More VLM Finetuning Details"},{"comment":"Heatmaps are generated for the ground-truth class ('based on the index given by the annotations instead of predictions'), not for the predicted class. This is inconsequential for PT, where correct predictions have predicted label equal to the ground truth, but it matters for IR, which includes WR samples (wrong prediction with 'right' rationale). For a wrong prediction, the heatmap for the ground-truth class does not necessarily reflect the evidence the model actually used to make its incorrect prediction. Please either generate heatmaps for the predicted class, which is the natural target for 'evidence for the prediction,' or provide a justification and show that the IR conclusions are unchanged under both choices.","section":"Explanation Heatmap Generation"},{"comment":"The abstract asserts without qualification that 'the well-adopted fine-tuning methods led to more correct predictions based on invalid evidence.' Yet Table 2 contains multiple settings where PT increases after fine-tuning (e.g., ALBEF-ViT-B/16 FLCP on CUB: 64.33 vs 49.31 for ZS; CLIP-ViT-B/32 FT on CalTech-101: 80.01 vs 73.44). The body acknowledges these as 'rare exceptions' and attributes them to low zero-shot accuracy, but the headline claim is broader than the data. Please qualify the abstract and conclusion (e.g., 'in most settings') and quantify how often PT decreases across the full experimental matrix.","section":"Abstract / Weaknesses of Fine-tuning"}],"minor_comments":[{"comment":"The acronym FLCP is used in the main text, but the supplementary refers to 'the FLYP method' in the paragraph about the contrastive loss temperature. Please use a single, consistent acronym throughout.","section":"Supplementary: More VLM Finetuning Details"},{"comment":"The phrase 'deteriorate the prediction ratinoality' contains a typo; it should read 'rationality.'","section":"Supplementary: Additional Visualizations"},{"comment":"The phrase 'high-stack applications' in the conclusion should be 'high-stakes applications.'","section":"Conclusion"},{"comment":"The caption lists IoU and RMA values for (a) and (b) but does not state which models or explanation methods produced these heatmaps; please clarify so the example can be interpreted.","section":"Figure 1"},{"comment":"The table layout with separate PT and IR blocks is easy to misread; consider adding explicit row labels to every row (e.g., a 'Metric' column) or using a multi-level header.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the proposed PT/IR metrics are simple and likely to be adopted if the empirical claims hold. The main risk is the uncalibrated RMA threshold and the absence of uncertainty estimates, both of which are addressable with additional experiments. I do not see evidence of citation manipulation or duplicate publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know about this paper because it makes a claim that, if true, matters for anyone fine-tuning VLMs for high-stakes use: the usual methods (LP, FLCP, FT) improve accuracy and Inference Reliability (correct predictions given valid evidence) but lower Prediction Trustworthiness (correct predictions without valid evidence). The result is plausible and consistent with shortcut-learning intuitions, and it is presented as a systematic comparison across four models and four datasets, plus OOD and ablations. The two proposed metrics are simple conditional proportions of correctness and \"valid evidence,\" so there is no fitted circularity. The paper also ships code. That part is fine.\n\nThe soft spot is the definition of valid evidence. They threshold RMA at 0.5 uniformly, with no sensitivity analysis. Since RMA is the fraction of heatmap mass inside the ground-truth mask, for ImageNet and Stanford-Dogs—where the masks are bounding boxes—any image whose box covers more than half the pixels will have a uniform heatmap counted as valid evidence. That means the PT/IR comparisons across methods are potentially confounded by how each method's correct predictions distribute over object sizes. The stress-test note is right that the ordering could shift if the threshold were object-size calibrated. The authors don't seem to have considered this.\n\nAlso, the supplementary states results come from a single run; \"we did not observe any variation\" is not a substitute for error bars. And the abstract's \"we found that ... fine-tuning methods led to more correct predictions based on invalid evidence\" is stronger than the tables, which show a few exceptions (e.g., ALBEF and BLIP on CUB) where PT rises after fine-tuning. They acknowledge those in the text, but the takeaway oversells consistency.\n\nNone of this makes the question invalid; it makes the current evidence conditional. The paper deserves a serious referee, because the phenomenon is important and the failure mode is fixable. I would send it to review with a request for sensitivity analysis of the threshold (or a per-dataset calibrated definition), multiple seeds with variance reporting, and an abstract that reflects the exceptions. I'd bring it to a reading group mainly to discuss the methodological trap.","headline":"Plausible and important claim that fine-tuning hurts prediction rationality, but the uncalibrated RMA threshold and single-run results make the evidence conditional, not conclusive.","tokens_in":18054,"tokens_out":3178,"would_cite":true,"duration_ms":30843,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning a vision-language model improves accuracy while increasing the share of correct predictions that rest on invalid evidence, a gap that standard accuracy metrics hide.","keywords":["prediction trustworthiness","inference reliability","vision-language model fine-tuning","explanation heatmaps","relevant mass accuracy","prediction rationality","out-of-distribution robustness"],"falsifier":"Recompute PT and IR while varying the validity threshold between roughly 0.3 and 0.7, or replacing bounding-box masks with pixel-level segmentation masks on ImageNet; if for some threshold in that range the fine-tuned methods no longer show a PT drop relative to zero-shot, the paper's central finding is an artifact of the chosen cutoff.","tokens_in":17083,"feed_emoji":"🎯","tokens_out":6126,"duration_ms":52752,"temperature":0.7,"pith_summary":"Fine-tuning a vision-language model such as CLIP on a downstream classification task reliably improves accuracy, but the paper argues that this gain is accompanied by a hidden cost: a larger fraction of the model's correct answers rest on evidence that would not survive human scrutiny. To make this visible, the paper proposes two metrics. Prediction Trustworthiness measures, among all correct predictions, the share whose explanation heatmap concentrates on the target object rather than on background; Inference Reliability measures, among all cases where the model's attention lands on the target, the share of correct predictions. Across four VLM backbones, four datasets, and out-of-distribution corruptions, fine-tuning consistently lowers Prediction Trustworthiness while raising Inference Reliability relative to zero-shot. The message is that accuracy alone overstates the quality of fine-tuned VLMs.","feed_headline":"Fine-tuned VLMs guess right on bad evidence","feed_subtitle":"Accuracy gains hide a trust problem: tuned models lean on background cues, yet focus helps them answer.","key_machinery":"The machinery has three parts. First, explanation heatmaps are produced by gradient-weighted attention, a method that combines attention maps with their gradients to the output; these heatmaps stand in for the model's evidence. Second, the Relevant Mass Accuracy (RMA) score, defined as $\\text{RMA}(H,M)=\\frac{\\sum H\\odot M}{\\sum H}$, measures what fraction of heatmap mass falls inside the ground-truth target mask, and a hand-set threshold of 0.5 labels a prediction's evidence as valid or invalid. Third, the four quadrant counts RR, RW, WR, and WW define Prediction Trustworthiness as $\\text{PT}=\\frac{RR}{RR+RW}$ and Inference Reliability as $\\text{IR}=\\frac{RR}{RR+WR}$. The separation of PT from IR is what distinguishes 'right for wrong reasons' from 'reliable when focused.'","core_discovery":"The central claim is that mainstream fine-tuning methods, including linear probing, finetune-like-CLIP-pretrain, and full fine-tuning, produce more correct predictions based on invalid evidence than zero-shot does, as measured by the paper's Prediction Trustworthiness metric. Concretely, for CLIP-ViT-B/16 on ImageNet-1K, PT drops by 6.4%, 5.65%, and 4.07% for LP, FLCP, and FT, while accuracy rises; on CalTech-101 with ALBEF, the drops are 17.2, 13.85, and 27.31 percentage points. The same tuned models show higher Inference Reliability: given that valid evidence is found, they are more often correct, with CLIP-ViT-B/16 on ImageNet improving IR by 12.6, 8.67, and 16.92 percentage points. Both findings persist on ImageNet-C and across ablation settings, including different optimizers, learning rates, explanation methods, and prompt- and adapter-based tuning.","pith_inferences":["The authors do not test it, but the PT/IR decomposition suggests a concrete fine-tuning objective: adding a term that penalizes attention mass outside the target region should improve PT, and the paper's metrics provide the evaluation recipe.","Because RMA uses ground-truth boxes as masks, the valid-evidence label is easiest to earn on datasets where the box covers much of the image; a reader should recheck rankings on fine-grained masks before relying on exact numbers.","A practical audit protocol follows directly: in a safety-critical deployment, log heatmaps alongside predictions and reject correct answers whose RMA is below the threshold, which would recover some of the trust that fine-tuning loses.","The finding that IR rises with fine-tuning suggests the failure mode is attention selection rather than classification itself, so interventions aimed at focusing attention may be more fruitful than changing the classifier."],"forward_implications":["Reporting accuracy alone will overstate the value of fine-tuning a VLM for a safety-critical task.","A model can become simultaneously more accurate and less trustworthy, so the two properties need to be tracked independently.","Fine-tuning methods that improve accuracy are not interchangeable: the paper shows that LP, FLCP, and FT differ in how much PT drops and how much IR rises.","On corrupted images the same pattern holds, so the trustworthiness drop is not a quirk of clean evaluation data.","The four-way RR/RW/WR/WW accounting gives a template for auditing any classifier's evidence, not just VLM fine-tuning."],"supporting_citations":[{"why":"Supplies the Relevant Mass Accuracy (RMA) score used to judge whether explanation heatmaps count as valid evidence.","marker":"(Brandt, Raatjens, and Gaydadjiev 2023)"},{"why":"Supplies the gradient-weighted attention method that generates the explanation heatmaps studied as the model's evidence.","marker":"(Chefer, Gur, and Wolf 2021)"},{"why":"Provides the CLIP vision-language model and zero-shot evaluation protocol that the fine-tuning comparison builds on.","marker":"(Radford et al. 2021)"},{"why":"Defines the FLCP fine-tuning method and the accuracy-only conclusion that the paper's rationality metrics contest.","marker":"(Goyal et al. 2023)"},{"why":"Provides ImageNet-1K, the main benchmark, including the bounding-box annotations used as ground-truth masks.","marker":"(Russakovsky et al. 2015)"},{"why":"Provides ImageNet-C corruptions used to test whether the PT and IR findings persist under distributional shifts.","marker":"(Hendrycks and Dietterich 2018)"},{"why":"Provides CUB-200-2011 with pixel-level segmentation masks, giving the paper a finer-grained check on evidence validity.","marker":"(Wah et al. 2011)"}],"fun_headline_variants":["Fine-tuning boosts VLM accuracy but on invalid evidence","Tuned VLMs: right answers, wrong reasons","Accuracy up, trust down: fine-tuning's evidence problem","When VLMs get it right for the wrong reasons"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on labeling a prediction's evidence as valid whenever its RMA score is at least 0.5, and this single threshold is applied uniformly across all images, datasets, and models.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuning boosts VLM accuracy but on invalid evidence","Tuned VLMs: right answers, wrong reasons","Accuracy up, trust down: fine-tuning's evidence problem","When VLMs get it right for the wrong reasons"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1379,"prompt_tokens":936,"completion_tokens":443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":379}},"tokens_in":552,"tokens_out":443,"duration_ms":4772,"temperature":1.0,"reasoning_tokens":379,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:13:25.379871+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute PT and IR while varying the validity threshold between roughly 0.3 and 0.7, or replacing bounding-box masks with pixel-level segmentation masks on ImageNet; if for some threshold in that range the fine-tuned methods no longer show a PT drop relative to zero-shot, the paper's central finding is an artifact of the chosen cutoff.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-weighted attention method that generates the explanation heatmaps studied as the model's evidence."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the FLCP fine-tuning method and the accuracy-only conclusion that the paper's rationality metrics contest."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides ImageNet-C corruptions used to test whether the PT and IR findings persist under distributional shifts."}],"review_version":1}