{"id":"b485e05b-240e-41e6-b0c8-8a5da6a7fd2b","arxiv_id":"2603.27176","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"MEDIC-AD adds anomaly-aware and difference tokens to a medical VLM, claiming SOTA lesion detection, temporal tracking, and visual grounding; the zero-shot claim is undermined by likely train/test overlap.","lead":"MEDIC-AD is a medical image-language model that adds three abilities to a medical VLM: spotting lesions, tracking whether a condition is getting better or worse between scans, and producing heatmaps that show where the model looked. A generalist might read it because it claims state-of-the-art results on clinical benchmarks, though the 'zero-shot' evaluation appears to train on data from the same benchmarks used to test.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Zero-shot anomaly detection claims are unverifiable because Stage 1 training uses BMAD/ChestX-Det without any documented exclusion of the four test datasets, and Stage 3 evaluates on the same collections used for training.","rationale":"The reader's weakest assumption correctly identifies the central risk: the zero-shot anomaly detection claim depends on the test datasets being unseen, and the paper does not establish this. I agree this is the most load-bearing concern. I mark agreement as partial because the reader asserts these datasets are 'likely' contained in BMAD, whereas the stronger and safer framing is that the paper provides no evidence of exclusion, and BMAD's own scope is described broadly enough to cover brain MRI and chest X-ray modalities. The same structural problem extends to Stage 3, where the same benchmark families are used for both training and evaluation. The architecture itself is plausible and the stage-wise idea is interesting, but the empirical evidence as presented does not support the headline SOTA claims. Since the reader already recommends REJECT and my analysis supports that position without adding a new reason to change the verdict, the appropriate output is UNCHANGED.","tokens_in":16442,"tokens_out":6865,"duration_ms":67710,"concrete_test":"Obtain the exact Stage 1 training manifests and perform an image/patient-level overlap audit between BMAD/ChestX-Det training samples and the four zero-shot test sets (Brain MRI, Head CT, Br35h, COVID-19), using filenames, image hashes, or patient IDs. If any overlap exists, retrain Stage 1 after removing all overlapping samples and recompute Table 1 F1 scores. If the average F1 drops materially (e.g., more than 2 points) or the relative ranking against Lingshu/Citrus-V changes, the zero-shot SOTA claim fails; if no overlap is found, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that the four 'zero-shot' test sets—Brain MRI [29], Head CT [31], Br35h [18], and COVID-19 [13]—are entirely unseen during Stage 1 training. Section 3.2 states Stage 1 trains on BMAD and ChestX-Det, and Section 2.2 describes BMAD as a consolidated benchmark covering brain MRI and chest X-ray modalities. The paper never provides the exact BMAD/ChestX-Det subset manifests, never states that the four test datasets were excluded, and releases no code or data to audit this. If any of the zero-shot test images or patients appear in Stage 1 training, the reported average F1 of 91.6 in Table 1 is not zero-shot, and the SOTA claim over Lingshu and Citrus-V is invalidated. The same issue affects Table 3: Stage 3 is trained on 'selected subsets of BMAD and ChestX-Det' and evaluated on 'a subset of BMAD and ChestX-Det' (BraTS2021, RESC, BTCV+LiTs, ChestX-Det) with no held-out split described, meaning the segmentation results may reflect in-distribution memorization rather than explainability. Even without confirmed overlap, the onus is on the paper to demonstrate exclusion; its own assertion in Section 4.1 that test datasets are 'entirely unseen' is unsupported by the training description. This is decisive because the headline empirical contribution—zero-shot SOTA detection and segmentation—rests on it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"MEDIC-AD is a 7B medical vision-language model built on Lingshu and augmented with three trainable modules: <Ano> anomaly tokens (Stage 1), <Diff> difference tokens for temporal comparisons (Stage 2), and a ConvNeXt-based heatmap head (Stage 3). The paper reports state-of-the-art results on zero-shot anomaly detection on Brain MRI, Head CT, Br35h, and COVID-19 chest X-ray (Table 1), on MMXU disease-progression tracking (Table 2), and on visual grounding/segmentation over BMAD and ChestX-Det (Table 3), together with a 300-patient hospital validation.","tokens_in":16788,"tokens_out":6599,"duration_ms":68713,"significance":"If the empirical claims are trustworthy, the staged token design is a useful step toward explainable medical VLMs: it explicitly separates abnormality, temporal difference, and grounding mechanisms while preserving general medical QA. The ablations, hyperparameter sensitivity study, and real-hospital evaluation are assets. However, the paper's key 'zero-shot' and grounding comparisons are currently unverifiable because training-set/test-set disjointness is not established. The contribution should be judged only after this issue is resolved.","major_comments":[{"comment":"Stage 1 trains on 'BMAD, ChestX-Det' and §4.1 asserts the four test sets (Brain MRI, Head CT, Br35h, COVID-19) are 'entirely unseen during training.' BMAD is described in §2.2 as a consolidated medical anomaly detection benchmark covering Brain MRI and chest X-ray, and the four test datasets fall in exactly these modalities. The paper gives no BMAD subset manifest, no list of excluded patients/images, and no code to check overlap. If any of the test images or even patient cohorts appear in Stage 1, the average F1=91.6 is not zero-shot and the SOTA claim over Lingshu/Citrus-V is invalid. Please provide exact dataset components and explicit exclusion checks, or re-run with rigorously disjoint splits.","section":"§3.2 / §4.1, Table 1"},{"comment":"Stage 3 is trained on 'selected subsets of BMAD and ChestX-Det' and evaluated in Table 3 on BraTS2021, RESC, BTCV+LiTs, and ChestX-Det. No train/validation/test split is described. The reported AUC/mIoU (e.g., BraTS2021 mIoU 87.6, RESC AUC 100) may therefore reflect in-distribution fitting rather than the ability to ground new anomalies. Specify the exact splits used for Stage 3, state whether the evaluation masks were seen during training, and report performance on a truly held-out subset.","section":"§4.3 / Stage 3, Table 3"},{"comment":"All results are single point estimates; no standard deviations, confidence intervals, or significance tests are reported. This is especially important in Table 2, where the main gain is 0.655 vs 0.620, and in Table 1 where several F1 gaps between MEDIC-AD and Lingshu are small. Please report variance over at least three runs or bootstrap confidence intervals so the SOTA claims can be evaluated statistically.","section":"Tables 1–3"}],"minor_comments":[{"comment":"The text says pooling size 4 and 10 soft prompts are selected, but Table 9 shows pooling size 8 gives avg F1 91.7 vs 91.6, and 5 soft prompts gives 91.8 vs 91.6. Please justify the choice or adjust the default.","section":"Figure 5 / Table 9"},{"comment":"Stage 1 is described as optimized with the standard LLM cross-entropy loss, but it is unclear how the anomaly attention map and <Ano> tokens receive anomaly-specific supervision. Clarify whether image-level or pixel-level anomaly labels are used to train the cross-attention modules.","section":"§3.2"},{"comment":"The '<Diff> tokens only' row has no Avg. F1 value, so the ablation is incomplete for the anomaly-detection column. Please provide that number.","section":"Table 4"},{"comment":"Minor typos and formatting issues: 'LLaV A-MED', 'explainablity', duplicated axis labels in Figure 5, and the inconsistent use of 'prior/current' vs 'baseline/follow-up' terminology.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the central soundness risk is the unverified train/eval overlap in the zero-shot detection and grounding evaluations. If the authors can show explicit disjoint splits and provide code/data to audit them, the staged token framework could be a valuable contribution. As written, the SOTA claims are premature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know a few things about arXiv:2603.27176. The paper proposes a stage-wise medical VLM that adds anomaly tokens, difference tokens, and a heatmap decoder to the Lingshu backbone. The individual components aren't new — anomaly tokens look like AnomalyGPT, difference tokens like image-difference captioning, heatmaps like Citrus-V — but the stage-wise combination in a medical VLM is a legitimate extension, and the writing is clear. The real-world evaluation on 300 patient chest X-ray pairs is a nice touch, though it compares only against Lingshu and uses automatic metrics.\n\nThe problem is the evaluation. The zero-shot anomaly detection claims rest on the assertion that Brain MRI, Head CT, Br35h, and COVID-19 are entirely unseen during training. But Stage 1 trains on BMAD and ChestX-Det, and BMAD is described as covering brain MRI and chest X-ray modalities. Those four datasets are very likely inside BMAD. The paper never states that they were excluded, and no code or data is released to check. Without that exclusion, the average F1 of 91.6 is not zero-shot; it's at best an in-distribution result, and the SOTA claim collapses.\n\nStage 3 has the same problem: it's trained on selected subsets of BMAD and ChestX-Det and evaluated on a subset of BMAD and ChestX-Det, with no held-out split described. The segmentation numbers are therefore not convincing evidence of explainability.\n\nThere are also smaller issues. No error bars or significance tests anywhere. Hyperparameters like pooling size and soft prompt count are tuned on the test metrics, which inflates results. Table 4 is internally inconsistent: the same configuration (MEDIC-AD with intermediate features and <Ano>) shows MMXU 0.635 in the feature selection section and 0.655 in the <Ano> ablation. That needs to be reconciled.\n\nI don't think this is a dishonest paper; the architecture is plausible and the authors engage with the literature. But the central empirical claim is not supported as written. This deserves a serious referee because the idea is worth testing, but the paper needs major revision: explicit exclusion lists, held-out splits, error bars, and a corrected Table 4.\n\nI'd pass on citing it in its current form.","headline":"Sensible architecture, but the zero-shot evaluation is unverifiable because training and test sets may overlap — the empirical claims need major revision.","tokens_in":17323,"tokens_out":3973,"would_cite":false,"duration_ms":36717,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MEDIC-AD claims that injecting learnable <Ano> and <Diff> tokens into a 7B medical VLM, plus a heatmap decoder, yields state-of-the-art zero-shot anomaly detection, temporal symptom tracking, and anomaly segmentation, outperforming closed-s","keywords":["medical VLM","anomaly detection","temporal reasoning","visual grounding","learnable tokens","zero-shot","chest X-ray","heatmap"],"falsifier":"Check the exact dataset composition of BMAD and ChestX-Det used in Stage 1: if Brain MRI, Head CT, Br35h, or the COVID-19 dataset appears in those training files, then re-run the zero-shot evaluation with those datasets removed. A drop back toward the 88.7 F1 of the backbone would disprove the zero-shot generalization claim. A second check is to perturb paired images with global illumination/translation and see whether <Diff> tokens still say 'no change'.","tokens_in":16313,"feed_emoji":"🩻","tokens_out":6944,"duration_ms":64003,"temperature":0.7,"pith_summary":"This paper tries to establish that a medical vision-language model can be turned into a clinically useful tool by teaching it, in stages, to spot abnormalities, compare two scans over time, and point at the evidence. The proposed model, MEDIC-AD, adds two kinds of learnable tokens to a 7B medical VLM: <Ano> tokens that encode lesion-related visual features, and <Diff> tokens that encode what changed between a prior and a current image. A third stage trains a heatmap decoder so the model can localize the regions behind its answer. The paper reports that this design beats closed-source and medical-specialized baselines on zero-shot anomaly detection (average F1 91.6), symptom tracking on the MMXU benchmark (overall 0.655), and anomaly segmentation grounding (e.g., mIoU 79.8 on ChestX-Det vs 12.4 for Citrus-V). If right, it would mean a modest 7B model with roughly 128M additional parameters can deliver lesion detection, temporal monitoring, and visual explanations in a real clinical workflow.","feed_headline":"Stage-wise tokens lift medical VLM to 91.6 zero-shot F1","feed_subtitle":"Anomaly, difference, and heatmap tokens beat GPT-4o and medical baselines on detection, tracking, and grounding.","key_machinery":"The load-bearing mechanism is a pair of learnable sequence tokens inserted into the LLM's input. <Ano> tokens are generated by an anomaly processor: learnable abnormal/normal system tokens cross-attend to four intermediate vision-encoder layers, a sigmoid-based anomaly map modulates the visual features, and a Q-Former plus MLP compresses the result into a few tokens that the LLM reads alongside the image. <Diff> tokens are produced by a second Q-Former that compares the anomaly-modulated features of two images, isolating lesion-specific change. The heatmap decoder is the third piece: it fuses <Ano> tokens with visual features through a ConvNeXt segmentation head to output a spatial map. Toge","core_discovery":"MEDIC-AD's central claim is that explicit, learnable anomaly tokens and difference tokens give a medical VLM the inductive biases it needs for three clinical tasks that generic VLMs handle poorly. Stage 1 produces <Ano> tokens by comparing abnormal and normal system tokens across multi-scale visual features, using the resulting anomaly attention map to modulate the visual features themselves; Stage 2 builds <Diff> tokens from those modulated features of two images through a separate Q-Former, so temporal change is separated from static context; Stage 3 fuses <Ano> tokens with visual features in a ConvNeXt head to generate heatmaps. The paper argues these stages mirror the clinical workflow—d","pith_inferences":["If the zero-shot claim survives a clean dataset-overlap check, the same token recipe could be applied to other medical modalities with temporal pairs—such as longitudinal CT for tumor response or OCT for retinal disease—where current VLMs simply concatenate images.","The design suggests a general strategy: rather than fine-tuning a large medical VLM end-to-end, one can insert small task-specific token generators in front of a frozen LLM, which may be cheaper to adapt and easier to audit in regulated settings.","A natural stress test would be to feed the model paired images with artificially induced global shifts (brightness, translation) to see whether <Diff> tokens correctly report 'no change'; the paper's claim that it separates pathological change from acquisition noise is directly checkable this way.","The 300-patient hospital study is a single site; confirming the same stability across multiple institutions and scanner vendors would turn the usability claim into an operational one."],"forward_implications":["The reported 91.6 average zero-shot F1 across brain MRI, head CT, brain tumor, and COVID-19 X-ray suggests anomaly-token supervision transfers across modalities and diseases.","On the MMXU symptom-tracking benchmark, the 0.655 overall accuracy over the 0.620 backbone implies that explicit difference tokens improve temporal reasoning rather than just image concatenation.","The heatmap decoder's mIoU gains over a SAM2-based grounding baseline (e.g., 79.8 vs 12.4 on ChestX-Det) indicate that anomaly tokens can serve as a spatial grounding signal without a heavy segmentation model.","Because only small modules are updated (about 128M parameters), the approach is compatible with keeping general medical QA knowledge intact, as the appendix's comparisons to full fine-tuning suggest.","The real-hospital 300-patient follow-up study, if the evaluation methodology holds, points toward deployment in monitoring workflows where a radiologist reviews model-generated change captions and heatmaps."],"fun_headline_variants":["Anomaly tokens turn medical VLM knowledge into clinical action","Stage-wise tokens boost medical VLM detection, tracking, explainability","MEDIC-AD: learnable tokens improve lesion, progress, heatmaps","State-of-the-art medical VLM with anomaly and diff tokens"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the four zero-shot test datasets—Brain MRI, Head CT, Br35h, and COVID-19 X-ray—were truly absent from Stage 1 training; since Stage 1 trains on BMAD and ChestX-Det, which are broad medical anomaly collections that likely include those very datasets, the paper needs to show an explicit exclusion to keep the 91.6 F1 claim zero-shot.","fun_headline_variants_meta":{"raw":{"variants":["Anomaly tokens turn medical VLM knowledge into clinical action","Stage-wise tokens boost medical VLM detection, tracking, explainability","MEDIC-AD: learnable tokens improve lesion, progress, heatmaps","State-of-the-art medical VLM with anomaly and diff tokens"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001018,"raw_usage":{"total_tokens":4135,"prompt_tokens":750,"completion_tokens":3385,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":3310}},"tokens_in":494,"tokens_out":3385,"duration_ms":24837,"temperature":1.0,"reasoning_tokens":3310,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T17:12:58.340745+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check the exact dataset composition of BMAD and ChestX-Det used in Stage 1: if Brain MRI, Head CT, Br35h, or the COVID-19 dataset appears in those training files, then re-run the zero-shot evaluation with those datasets removed. A drop back toward the 88.7 F1 of the backbone would disprove the zero-shot generalization claim. A second check is to perturb paired images with global illumination/translation and see whether <Diff> tokens still say 'no change'.","supporting_citations":[],"review_version":1}