{"id":"a13c0c34-8900-4b0b-a9c2-4af8ec338c1b","arxiv_id":"2501.09294","paper_version":1,"verdict":"REJECT","confidence":"LOW","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"HiCA, a hierarchical contrastive fine-tuning method for large vision-language models, is claimed to achieve state-of-the-art few-shot medical image classification, but the paper lacks the experimental detail needed to verify this claim.","lead":"This paper proposes HiCA, a two-stage fine-tuning approach that uses hierarchical contrastive learning to adapt large vision-language models to few-shot medical image classification. The authors report state-of-the-art accuracy and AUC on Chest X-ray and Breast Ultrasound benchmarks, but the paper omits code, error bars, and the key baseline it cites.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central SOTA claim is untestable as presented: Table I omits the cited prior method [5] that HiCA claims to surpass, and no significance statistics back the point estimates despite Section IV-A promising them.","rationale":"The reader's REJECT verdict is justified, but I would locate the load-bearing problem one step before the segmentation dependency. For the central claim of consistent SOTA superiority, the decisive evidence is the comparison in Table I. That table compares HiCA against CLIP zero-shot, a supervised CNN, and an ImageNet-transfer ResNet, but never against [5], the very method the introduction names as the prior work being surpassed. Since [5] is a published few-shot medical vision-language method using simple text descriptors, its omission means Table I does not establish the claimed SOTA status. In addition, the setup section announces significance testing, but no significance statistic appears in the paper; every reported number is a point estimate with no error bar or confidence interval. The phrase 'consistently outperforms' cannot be supported by single-run numbers. The reader's flagged weakness, the unnamed segmentation model behind Eq. (4), is genuine and prevents replication, as does the unspecified clustering step in Stage 1 and the missing hyperparameters (lambda_1, lambda_2, tau, delta). However, those are secondary to the comparison itself: even a fully specified method would need a controlled comparison to [5] with uncertainty quantification before the central claim can be accepted. There is no independent supporting artifact in the paper, such as released code, public data, or a formal proof, that could substitute for this check. If the proposed head-to-head test shows HiCA's margins are outside the noise, the central claim survives; otherwise, the paper as written does not support it. The verdict therefore remains REJECT, unchanged from the reader, because the required evidence is absent rather than merely imperfect.","tokens_in":8484,"tokens_out":8542,"duration_ms":76734,"concrete_test":"Run a controlled head-to-head evaluation on identical train/validation splits of Chest X-ray and Breast Ultrasound, comparing HiCA against the Byra et al. baseline [5] at 1, 5, 10, and 20 shots per class, using the same base LVLM and text descriptors, with at least 5 random seeds per condition; report mean plus/minus standard deviation (or 95% CI) for accuracy and AUC and a paired significance test (bootstrap or Wilcoxon). If the HiCA margin over [5] and over Transfer Learning is within the seed-to-seed spread, or fails the significance test at alpha = 0.05, the 'consistently outperforms' claim is unsupported as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is empirical: HiCA 'consistently outperforms state-of-the-art baselines' in few-shot and zero-shot medical image classification. The only evidence for this is Table I, and two features of that table prevent the claim from being evaluated. First, the most relevant baseline is absent: Section I explicitly cites Byra et al. [5] as the prior method HiCA surpasses, but [5] is not included among the four competing methods in Section IV-A or in Table I. An SOTA claim cannot be checked without its cited SOTA comparator. Second, Section IV-A states that 'Statistical significance testing is conducted to verify the observed improvements,' yet no test statistic, p-value, confidence interval, or standard deviation appears anywhere in Tables I-VI; all entries are single point estimates. Without variance estimates or repeated-seed results, 'consistently outperforms' is asserted rather than demonstrated. These omissions attack the minimal condition for the central claim: that the reported margins reflect a stable, reproducible advantage rather than split or seed noise. The unspecified segmentation model and clustering algorithm are real but secondary reproducibility gaps; even if fully specified, the SOTA claim would still require a controlled comparison to [5] with uncertainty quantification.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HiCA (Adaptive Vision-Language Fine-tuning with Hierarchical Contrastive Alignment), a two-stage fine-tuning method for few-shot medical image classification. Stage 1 adapts a vision-language model's visual and text encoders with pseudo-label clustering and a medical text corpus; Stage 2 applies a hierarchical contrastive loss combining global image-text alignment, local ROI-text alignment, and cross-category separation. The authors report experiments on Chest X-ray and Breast Ultrasound datasets, claim state-of-the-art accuracy and AUC in few-shot and zero-shot settings, and include ablation, human evaluation, generalization, robustness, and efficiency analyses.","tokens_in":8674,"tokens_out":3860,"duration_ms":39945,"significance":"If the empirical claims were substantiated, HiCA would be a useful and well-motivated recipe for adapting large vision-language models to medical imaging: the three-level loss decomposition is conceptually clean, the method is described with explicit equations, and the paper addresses a practically important problem. The manuscript also provides a human evaluation, robustness testing, and an efficiency comparison, which are commendable. However, as presented the central claim that HiCA 'consistently outperforms state-of-the-art baselines' is not supported by the evidence: the key state-of-the-art comparator cited in the introduction is absent, no uncertainty quantification or significance statistics appear despite an explicit statement that they were conducted, and several components of the method are underspecified to the point of irreproducibility. The framework may well be defensible, but the paper currently falls short of the evidence bar for its main claim.","major_comments":[{"comment":"Section IV-A states that experiments cover few-shot settings 'with labeled data ranging from 1-shot to 20-shot per class' and that 'statistical significance testing is conducted to verify the observed improvements,' yet Table I reports only a single point estimate per method per dataset, with no per-shot breakdown, no standard deviations, no confidence intervals, and no p-values. Without variance or repeated-seed information, the claim that HiCA 'consistently outperforms' baselines is an assertion rather than a demonstrated result. This is the central load-bearing evidence for the paper's main conclusion.","section":"Section IV-A and Table I"},{"comment":"The introduction specifically cites Byra et al. [5] as the prior work that HiCA surpasses (e.g., 'surpassing existing methods by a substantial margin [5]'), but [5] does not appear among the four compared methods listed in Section IV-A nor in Table I. A state-of-the-art claim cannot be evaluated if the stated state-of-the-art comparator is missing from the empirical comparison. The authors need to include [5] (and ideally other recent medical VLM few-shot methods) with the same experimental protocol.","section":"Section I and Section IV-A/Table I"},{"comment":"The local alignment loss in Eq. (4) depends on ROIs 'extracted using a segmentation model,' but the segmentation model is never named, and its training data or validation are not described. Likewise, Stage 1 uses 'pseudo-labels generated via unsupervised clustering' and a 'domain-specific corpus of medical texts,' but the clustering algorithm, number of clusters, text corpus, and the base LVLM backbone are all unspecified. These details are required for reproducibility and to rule out that the reported local-alignment gains stem from an unstated segmentation oracle or from overlap between the pseudo-labeling data and the evaluation sets.","section":"Section III-B2 and Section III-C"},{"comment":"The ablation study in Table II is reported only on the Chest X-ray dataset, not on Breast Ultrasound, and the entries are single point estimates without error bars or significance tests. The conclusion in Section IV-C that 'all components are essential for achieving the best performance' is therefore not supported across datasets and could reflect split or seed noise. The authors should provide ablations on both datasets with multiple runs and uncertainty quantification.","section":"Section IV-B, IV-C, and Table II"}],"minor_comments":[{"comment":"The abstract and conclusion claim strong performance in 'zero-shot scenarios,' but the experimental section describes only few-shot settings; Table I includes a CLIP zero-shot baseline, but no HiCA zero-shot result is reported. The claim should be either supported with HiCA zero-shot experiments or removed.","section":"Abstract/Conclusion"},{"comment":"The human evaluation is described as involving three radiologists and 50 samples per dataset, but the paper does not report inter-rater agreement, the selection criteria for the 50 samples, whether the radiologists were blinded to method, or any variance in the scores. These details are necessary to interpret the reported interpretability and clinical-validity scores.","section":"Section IV-D and Table III"},{"comment":"The 'unseen categories' generalization experiment is unclear for the two binary datasets used (pneumonia vs. normal and benign vs. malignant); with only two classes, splitting into seen and unseen classes leaves only one class in each set. The setup needs clarification, e.g., whether unseen categories are different pathologies from a larger pool or whether a different dataset construction is used.","section":"Section IV-E1 and Table IV"},{"comment":"The local alignment loss indexes the text embedding as z_text^k, but the notation for text descriptors and class indices is inconsistent with the global loss (where t_i is paired with image x_i). Please clarify the indexing so that the positive and negative pairs in the local loss are unambiguous.","section":"Eq. (4)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an early-stage draft: the core method is plausible, but the experimental reporting is too incomplete to support the central claim. The fixes needed (including the missing [5] baseline, per-shot results with uncertainty, and full specification of the segmentation and clustering components) are substantial but well within the scope of a revision, so I would not reject outright. I recommend the editor invite a major revision with clear expectations that these points be addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a coherent combination of known pieces, but the central claim is not checkable. Table I omits Byra et al. [5], the very baseline the introduction says HiCA surpasses, and Section IV-A promises significance testing that never appears in any table. The stress-test note lands.\n\nWhat's actually new: the package. Global CLIP-style alignment, ROI-level local alignment with a margin-based cross-category loss, plus the two-stage pseudo-label pretraining, is a sensible composition for medical few-shot VLMs. Equations (3)-(5) are clean, and the ablation, human eval, noisy-text robustness check, and efficiency table show real effort. I don't doubt the authors know what they want to do.\n\nSoft spots, in order of severity. First, the SOTA claim is structurally unsupported: no [5], no other recent few-shot medical VLM, no error bars, no standard deviations, no p-values or confidence intervals anywhere in Tables I-VI. 'Consistently outperforms' is asserted, not demonstrated. Second, reproducibility: the ROI extraction uses an unnamed segmentation model, the clustering algorithm is not described, and lambda1, lambda2, tau, delta, and K are not reported. Third, potential data leakage: Stage 1 pseudo-label clustering is run on the same datasets used for final evaluation; the authors never state that test images were excluded from clustering. That needs an explicit split. Fourth, the human evaluation has three radiologists and 50 samples with no agreement or significance statistic, so 'significantly better' is not supported. These are fixable, but together they make the current submission a high-level outline, not a verifiable paper.\n\nWho this is for: researchers working on medical vision-language adaptation might get a useful loss-decomposition idea, and the paper is a good case study for reproducibility standards. I would not cite the numbers.\n\nRecommendation: I would send it to peer review with a clear request for major revision rather than desk-reject. The method is plausible, the equations are well-formed, and the missing items are a concrete checklist: include [5], report uncertainty, name the segmentation model and clustering, release code, and state the data split. If those come back, it could be a decent paper.","headline":"A coherent method paper whose central SOTA claim is unverifiable because the key baseline and all uncertainty estimates are missing.","tokens_in":9264,"tokens_out":4197,"would_cite":false,"duration_ms":42809,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HiCA's two-stage, three-level contrastive alignment yields the best few-shot accuracy and AUC on two medical imaging benchmarks.","keywords":["few-shot learning","medical image classification","vision-language models","contrastive learning","hierarchical alignment","chest X-ray","breast ultrasound","domain adaptation"],"falsifier":"Measure the segmentation masks used for $L_{\\text{local}}$ against expert segmentations on a held-out subset of each dataset. If the ROI masks have low overlap with anatomical structures yet the reported AUC still holds, the local-loss gains are not caused by anatomical alignment. Conversely, retrain HiCA with only $L_{\\text{global}} + L_{\\text{cross}}$; if Breast Ultrasound AUC does not drop substantially, the central contribution of the local alignment term is unsupported.","tokens_in":8185,"feed_emoji":"🩻","tokens_out":5429,"duration_ms":51776,"temperature":0.7,"pith_summary":"The paper proposes HiCA, a two-stage fine-tuning strategy that adapts a large vision-language model to medical image classification using very few labeled examples. Stage one adapts the visual and text encoders to medical data using pseudo-labeled images and a medical text corpus; stage two aligns images and text at three levels: global image-text, local region-text, and cross-category separation. The paper reports that HiCA outperforms CLIP zero-shot, a supervised CNN, and ImageNet-transfer learning on Chest X-ray and Breast Ultrasound, reaching an AUC of 0.92 in the 20-shot breast ultrasound setting. If this holds, pretrained vision-language models become practical for medical domains where annotated data is scarce.","feed_headline":"HiCA lifts few-shot medical image AUC to 0.92","feed_subtitle":"A two-stage vision-language fine-tune beats CLIP and transfer baselines on Chest X-ray and Breast Ultrasound.","key_machinery":"The load-bearing object is the hierarchical contrastive loss itself: three additive terms that jointly enforce whole-image alignment, fine-grained region-level alignment, and separation between categories. The global term is the standard image-text contrastive loss, the local term repeats it on regions of interest extracted by a segmentation model, and the cross-category term is a margin loss on mismatched pairs. The ablation study is what makes this machinery central: removing any one term lowers Chest X-ray accuracy from 86.3 to between 83.5 and 84.7, so the paper credits the hierarchy, rather than any single loss, with the improvement.","core_discovery":"HiCA's central claim is that the gap between pretrained vision-language models and medical images is largely an alignment problem, and that aligning image and text embeddings at three levels closes it. The method fine-tunes the visual encoder on pseudo-labeled medical images and the text encoder on medical text, then optimizes $L_{\\text{HiCA}} = L_{\\text{global}} + \\lambda_1 L_{\\text{local}} + \\lambda_2 L_{\\text{cross}}$, where the global term matches whole images to class descriptors, the local term matches segmented regions of interest to fine-grained descriptors, and the cross-category term pushes mismatched pairs apart by a margin. The paper reports that this combination raises accuracy and AUC above CLIP zero-shot, supervised CNN, and ImageNet-transfer baselines on both datasets, with the largest gain an AUC of 0.92 in the 20-shot Breast Ultrasound setting.","pith_inferences":["My inference: if the local-alignment term works because ROIs capture clinically meaningful structure, the same loss should transfer to histopathology or retinal imaging, where fine-grained regions are similarly diagnostic.","My inference: because Stage 1 pseudo-labels come from unsupervised clustering, the method's ceiling likely depends on cluster quality; varying the cluster count or using a different clustering algorithm would expose that sensitivity.","My inference: the descriptor-generation process is underspecified, so a natural test is to replace the hand-written descriptors with LLM-generated reports and measure whether the noisy-descriptor robustness persists."],"forward_implications":["On the two tested datasets, HiCA beats CLIP zero-shot, a supervised CNN, and ImageNet-transfer learning in both few-shot and zero-shot accuracy and AUC.","Ablations show that the global, local, and cross-category losses each contribute; dropping any one lowers Chest X-ray accuracy from 86.3 to 83.5-84.7.","On held-out unseen categories, HiCA generalizes better than all baselines, reaching 75.6 accuracy versus 71.8 for the best baseline.","Under noisy textual descriptors, HiCA degrades gracefully, losing 4.2 accuracy points on Chest X-ray compared with clean text.","In the paper's human evaluation with three radiologists, HiCA predictions scored higher than transfer learning on both interpretability and clinical validity."],"supporting_citations":[{"why":"Supplies the CLIP vision-language base model that HiCA fine-tunes and whose zero-shot performance is the starting baseline.","marker":"[1]"},{"why":"Supplies ALIGN, the noisy-text-supervised model that motivates the need for domain-specific text-image alignment.","marker":"[2]"},{"why":"Supplies the text-descriptor few-shot medical classification approach whose results HiCA directly extends and compares against.","marker":"[5]"},{"why":"Provides the concept-bottleneck interpretability line that motivates the paper's interpretability claims and human evaluation.","marker":"[9]"},{"why":"Supplies a multimodal medical model context that frames the goal of generalizable medical vision-language representations.","marker":"[14]"},{"why":"Provides an empirical assessment of medical LLM and VLM adaptation that frames whether fine-tuning gains are real progress.","marker":"[17]"}],"fun_headline_variants":["Two-stage vision-language fine-tune beats CLIP on chest X-ray","Hierarchical contrastive learning lifts few-shot medical AUC","HiCA aligns image-text at three levels for medical imaging","Few-shot medical image analysis hits 0.92 AUC with HiCA","Adaptive vision-language fine-tuning improves few-shot medical images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's success rests on an unnamed segmentation model that extracts the regions of interest; if that model is inaccurate or was trained on the evaluation datasets, the reported local-alignment gains could be artifacts rather than a property of hierarchical contrastive learning.","fun_headline_variants_meta":{"raw":{"variants":["Two-stage vision-language fine-tune beats CLIP on chest X-ray","Hierarchical contrastive learning lifts few-shot medical AUC","HiCA aligns image-text at three levels for medical imaging","Few-shot medical image analysis hits 0.92 AUC with HiCA","Adaptive vision-language fine-tuning improves few-shot medical images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1344,"prompt_tokens":888,"completion_tokens":456,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":370}},"tokens_in":504,"tokens_out":456,"duration_ms":4655,"temperature":1.0,"reasoning_tokens":370,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:05:47.887202+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the segmentation masks used for $L_{\\text{local}}$ against expert segmentations on a held-out subset of each dataset. If the ROI masks have low overlap with anatomical structures yet the reported AUC still holds, the local-loss gains are not caused by anatomical alignment. Conversely, retrain HiCA with only $L_{\\text{global}} + L_{\\text{cross}}$; if Breast Ultrasound AUC does not drop substantially, the central contribution of the local alignment term is unsupported.","supporting_citations":[{"cited_title":"Learning transferable visual models from na tural language supervision,","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP vision-language base model that HiCA fine-tunes and whose zero-shot performance is the starting baseline."},{"cited_title":"Scaling up visual and vision-language representation learning with noisy text s upervision,","cited_arxiv_id":null,"evidence_quote":"Supplies ALIGN, the noisy-text-supervised model that motivates the need for domain-specific text-image alignment."},{"cited_title":"Few-shot medical image classification with simple shape and texture text descriptors using vision-language models","cited_arxiv_id":"2308.04005","evidence_quote":"Supplies the text-descriptor few-shot medical classification approach whose results HiCA directly extends and compares against."},{"cited_title":"Medica l adaptation of large language and vision-language models: Are we making pr ogress?","cited_arxiv_id":null,"evidence_quote":"Provides an empirical assessment of medical LLM and VLM adaptation that frames whether fine-tuning gains are real progress."}],"review_version":1}