{"id":"e8111923-e817-4f1a-8e14-a728bc3c99b0","arxiv_id":"2607.26885","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"CRC-tuned PMC-LLaMA plus offline asymmetric alignment and an anatomy-negation penalty yields SOTA medical image-report retrieval and zero-shot classification on MIMIC-CXR, CheXpert, and IU X-Ray.","lead":"SCALPEL turns a medical LLM into a text encoder for chest X-ray reports, then aligns it to images with a loss that penalizes left/right and negation mistakes. It reports stronger retrieval and zero-shot clinical matching than prior medical CLIP-style models on standard public benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"ANAO gains rest on unvalidated external NER/negation labels whose error rate is never measured against the claimed SOTA margins.","rationale":"The reader correctly isolates the external-metadata dependency of ANAO as the weakest assumption behind the SOTA claim. My stress test reaches the same locus: without a quantified error rate on a_i / n_i, the additive penalties in Eqs. 7–8 are an uncalibrated regularizer whose sign may be wrong on a non-trivial fraction of pairs. The rest of the stack (CRC + LoRA + offline cache) is standard and adequately ablated; the load-bearing novelty and the reported margins both ride on ANAO being clinically faithful. Because the paper already acknowledges cascading-error risk yet supplies neither NER accuracy nor a gold-label control, the CONDITIONAL verdict is unchanged—release of the exact metadata pipelines plus the radiologist-validated ablation above would be required before the margins can be treated as settled. No stronger internal inconsistency was found; the concern is empirical grounding, not logical collapse.","tokens_in":13511,"tokens_out":643,"duration_ms":13066,"concrete_test":"On a random 500-report subset of the MIMIC-CXR test split, have two radiologists independently annotate laterality (6-way) and the K negation bits used by ANAO; compute pipeline F1 against this gold set. Then re-train the Stage-2 model twice—once with pipeline labels, once with gold labels—and re-evaluate Table 1 I2T/T2I R@1. If pipeline F1 < 0.85 or the R@1 gap versus gold exceeds 1.5 points, the ANAO contribution (and thus the SOTA margin) is not reliably supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central SOTA claim for SCALPEL* (Table 1 I2T R@1 23.17%, T2I R@1 19.38%; Table 2 leading ACC/F1/VQA) is attributed in large part to the Anatomy-Negation Aware Objective (Eqs. 6–8). Those penalties are constructed from anatomical labels a_i and negation vectors n_i that are produced by external clinical NER pipelines [23,21] and then treated as ground truth. The paper never reports precision/recall of laterality or negation extraction on MIMIC-CXR (or any held-out sample), nor does it ablate noisy versus clean metadata. Section 5 itself flags “cascading errors” from this step, yet the ablation in Fig. 3 only contrasts “InfoNCE vs ANAO” under the same (uncharacterized) labels. If the pipelines systematically mis-tag laterality or negation—common failure modes on free-text radiology reports—then L_anat and L_neg push the shared embedding space in the wrong direction, and the reported gains over MedProbCLIP / MMedPO cannot be attributed to clinically correct supervision. This is the single least-secure condition under which the strongest claim holds.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"SCALPEL is a two-stage medical vision-language pre-training framework that converts a generative medical LLM (PMC-LLaMA-13B) into a discriminative isotropic text encoder via Clinical Report Contrastive (CRC) fine-tuning (bidirectional MNTP + SimCSE with LoRA), then performs asymmetric cross-modal alignment by freezing the LLM, caching text embeddings offline, and training only the vision encoder (DINOv2 ViT-B/16) plus a lightweight projector under an Anatomy-Negation Aware Objective (ANAO). ANAO augments InfoNCE with hinge penalties on pairs whose externally extracted anatomical laterality labels or negation vectors disagree (Eqs. 6–8). Experiments on MIMIC-CXR retrieval, zero-shot classification (IU X-Ray, CheXpert), and medical VQA (SLAKE, VQA-RAD) report state-of-the-art or consistently improved numbers versus MedCLIP, BioViL-T, BiomedCLIP, MedProbCLIP, MMedPO and related baselines, with ablations (Fig. 3) and hyperparameter sweeps (Table 3) supporting the contribution of CRC, caching, and ANAO.","tokens_in":13862,"tokens_out":1438,"duration_ms":35989,"significance":"The work targets three genuine, well-motivated bottlenecks in medical VLP: limited context of BERT-scale encoders on long radiology reports, anisotropic collapse of generative LLM embeddings under contrastive losses, and clinical hallucinations around laterality and negation. The asymmetric caching design yields a clear efficiency win (reported ~76% FLOPs reduction vs. a CXR-BERT baseline while retaining a large LLM text tower), and the modular “+SCALPEL” results show the components can be dropped into existing pipelines. If the gains are robust, the paper offers a practical recipe for bringing billion-parameter medical LLMs into contrastive VLP without full end-to-end joint training. Strengths include multi-benchmark evaluation, component ablations, efficiency numbers, and an explicit hyperparameter sensitivity table.","major_comments":[{"comment":"The central attribution of gains to ANAO (Eqs. 6–8, §3.3) rests on anatomical labels a_i and negation vectors n_i produced by external clinical NER/negation pipelines [23,21] that are treated as ground truth. No precision/recall, error analysis, or inter-annotator comparison of laterality or negation extraction is reported on MIMIC-CXR (or any held-out sample), and Fig. 3 only contrasts InfoNCE vs. ANAO under the same uncharacterized labels. Section 5 itself flags cascading errors from this step. Because L_anat and L_neg directly reshape the embedding space, systematic NER mistakes (common on free-text reports) would push representations in the wrong direction; the SOTA margins in Tables 1–2 cannot be confidently credited to clinically correct supervision without a noise study, clean-vs-noisy ablation, or at least reported extraction accuracy on a labeled subset.","section":"§3.3, Eqs. (6)–(8); Fig. 3; §5"},{"comment":"Tables 1–2 and Fig. 3 report point estimates only; there are no error bars, multi-seed means/stds, or statistical significance tests. Several absolute gains are modest (e.g., +1–2 points R@1 on already competitive baselines; occasional metric drops such as BioViL-T T2I R@5 and IU X-Ray F1 under +SCALPEL). Without variance estimates it is difficult to judge whether the claimed SOTA ranking of SCALPEL* is stable or sensitive to initialization and data order, especially given that α, λ_a, λ_n are selected for peak table numbers (Table 3).","section":"Tables 1–2; Fig. 3; Table 3"},{"comment":"The ablation in Fig. 3a jumps across simultaneous changes of text encoder (Transformer / CXR-BERT / PubMedBERT / PMC-LLaMA), vision backbone (ViT-B/16 vs DeiT3-L), and loss (InfoNCE vs ANAO). A fuller factorial or staged isolation (CRC alone, caching alone, ANAO alone, then combinations) on a fixed backbone would make the contribution of each claimed ingredient clearer and would better support the narrative that CRC resolves anisotropy and ANAO resolves hallucinations.","section":"Fig. 3(a); §4.4"}],"minor_comments":[{"comment":"Fig. 1’s similarity-contribution bar chart is hard to read in grayscale and the exact embedding-component decomposition method is not specified in the caption or text.","section":"Fig. 1"},{"comment":"Notation: S_ij is defined with temperature τ in Eq. 1, yet τ_s appears only in the SimCSE term; a single temperature table or explicit statement of which temperature is used at inference would help reproducibility.","section":"§3.1–3.2"},{"comment":"Implementation details give LoRA rank/α and training steps for CRC but omit the exact mean-pooling layer, whether [EOS]/last-token or mean-over-tokens is used for h_i, and the projector hidden sizes—small but necessary for exact reimplementation.","section":"§4.2"},{"comment":"Related work could more clearly position against concurrent LLM2CLIP-style caption-contrastive LLM encoders and against other medical negation-aware losses; a short paragraph on how ANAO differs from soft-target or ontology-based methods (MedCLIP, MedKLIP) would strengthen novelty framing.","section":"§2"},{"comment":"Minor typos / formatting: “F ramework” spacing in §3.1 heading; “V alidation” in §4.3; arXiv-style “Title Suppressed Due to Excessive Length” headers should be cleaned for camera-ready.","section":"§3.1; §4.3"}],"recommendation":"major_revision","confidential_remarks":"The NER-supervision gap is the single load-bearing weakness; if the authors can add even a modest validation of laterality/negation extraction accuracy or a controlled noise ablation, the paper becomes much stronger. Otherwise the SOTA claim over-reaches. Scope is appropriate for a solid medical CV / MICCAI-style venue; novelty is incremental but useful rather than transformative. No integrity red flags beyond ordinary hyperparameter selection on reported metrics."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a solid engineering recipe for CXR image–report alignment, not a conceptual leap. CRC (bidirectional MNTP + SimCSE on PMC-LLaMA), offline text caching, and the anatomy/negation penalties (ANAO) together give consistent gains on MIMIC retrieval and the usual zero-shot/VQA suites. The distinctive bit is ANAO plus the medical-report adaptation of the LLM2CLIP-style encoder conversion; everything else is known parts put together carefully.\n\nWhat it does well: it names three real bottlenecks (LLM anisotropy, memory, laterality/negation blindness) and attacks each with a concrete stage. Tables 1–2 show the full stack and the +SCALPEL plug-in both help most baselines. Ablations and the efficiency plot are honest enough—ViT-B/16 + cached LLM beats heavier vision backbones on their metrics, and FLOPs drop hard because text is frozen and cached. Hyperparameter sweeps exist. Citation pattern is fine; they credit LLM2CLIP, SimCSE, MedCLIP, BioViL, etc.\n\nSoft spots, in proportion: the stress-test lands. ANAO’s L_anat and L_neg are built from external NER/negation tags treated as ground truth, with zero reported precision on laterality or negation for MIMIC free text. Section 5 even flags cascading errors, yet Fig. 3 only swaps InfoNCE vs ANAO under the same uncharacterized labels. If those tags are noisy, the penalties push the space the wrong way and the SOTA margins are harder to trust. Minor issues: no multi-seed error bars, a few metric drops under +SCALPEL get post-hoc architectural excuses, and reproducibility is limited without code or the exact metadata pipeline. None of that sinks the paper; it just caps how hard you should lean on the headline numbers.\n\nWho it’s for: people building or benchmarking medical VLP on CXR who want a practical LLM-text-encoder path and an explicit laterality/negation knob. Worth a serious referee. I’d bring it to reading group only if someone is actively doing radiology retrieval or negation-aware contrastive work; otherwise skim the method and tables. Engage if that is your lane; treat the ANAO gains as provisional until NER quality is measured.","headline":"Competent medical VLP systems paper: real failure modes, known pieces assembled cleanly, SOTA tables that partly rest on unvalidated NER labels.","tokens_in":14539,"tokens_out":565,"would_cite":false,"duration_ms":18755,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"SCALPEL converts medical LLMs into isotropic encoders and aligns them to chest X-rays with anatomy- and negation-aware penalties, reaching state-of-the-art retrieval, zero-shot classification, and VQA.","keywords":["vision-language pre-training","medical LLMs","contrastive learning","cross-modal retrieval","anatomy-negation aware objective","radiology reports","zero-shot classification","medical VQA"],"falsifier":"Replace the external NER-derived laterality and negation labels with deliberately flipped or random labels and check whether retrieval R@1 and zero-shot F1 on MIMIC-CXR / CheXpert drop sharply; if performance is unchanged, ANAO is not doing the claimed work.","tokens_in":14352,"feed_emoji":"🩺","tokens_out":904,"duration_ms":20740,"temperature":0.7,"pith_summary":"Medical vision-language models usually rely on short-context BERT-style text encoders that cannot fully absorb long, terminology-heavy radiology reports. Plugging in large medical language models should help, but generative LLMs collapse into anisotropic embeddings under ordinary contrastive loss, joint training is memory-prohibitive, and vanilla InfoNCE ignores laterality and negation, producing clinical hallucinations. SCALPEL first contrastively fine-tunes a medical LLM on reports so its embeddings become isotropic and bidirectional, then freezes it and caches text features offline so only the vision side and a light projector train under large batches. An Anatomy-Negation Aware Objective adds explicit penalties whenever high similarity is assigned to pairs that disagree on laterality or on whether a finding is present or absent. On MIMIC-CXR, CheXpert, IU X-Ray, SLAKE and VQA-RAD the resulting system leads reported baselines in cross-modal retrieval, zero-shot disease classification and medical visual question answering.","feed_headline":"Medical LLM encoder beats prior X-ray–report models","feed_subtitle":"CRC fine-tuning plus anatomy-negation penalties lift retrieval, zero-shot diagnosis and VQA","key_machinery":"Anatomy-Negation Aware Objective (ANAO): standard InfoNCE plus additive penalties that drive down similarity for batch pairs whose extracted anatomical laterality labels differ or whose negation vectors disagree on radiological findings.","core_discovery":"A two-stage LLM-powered pipeline—Clinical Report Contrastive fine-tuning that turns a generative medical LLM into an isotropic bidirectional encoder, followed by asymmetric offline-cached alignment under an Anatomy-Negation Aware Objective—resolves anisotropy, memory cost and clinical hallucination and yields state-of-the-art medical vision-language representations.","pith_inferences":["If NER quality is the binding constraint, replacing the external pipeline with an end-to-end LLM that jointly extracts laterality/negation and produces the embedding could remove the cascading-error failure mode the authors flag.","The same anisotropy-plus-negation problem likely appears in other long-document medical modalities (pathology, discharge summaries); CRC+ANAO-style adaptation may transfer there with little change.","Freezing the LLM after CRC leaves early cross-modal attention unexplored; parameter-efficient unfreezing under the same ANAO loss is a natural next measurement."],"forward_implications":["Existing medical VLP pipelines can swap in a CRC-tuned LLM encoder plus ANAO and obtain measurable gains in I2T/T2I recall without redesigning the vision backbone.","Zero-shot disease classification and medical VQA improve when the shared space is forced to respect laterality and negation, reducing clinically dangerous mismatches.","Offline text-feature caching makes billion-parameter medical LLMs practical for large-batch contrastive training on standard hardware.","The same modular recipe can be dropped into other medical image-report collections beyond chest X-ray."],"fun_headline_variants":["SCALPEL turns generative medical LLMs into isotropic X-ray report encoders","Anatomy-negation penalties curb laterality errors in LLM vision-language models","CRC fine-tuning plus cached alignment lifts MIMIC-CXR retrieval to SOTA","Asymmetric offline caching lets medical LLMs train without huge batch memory","Two-stage SCALPEL pipeline fixes anisotropy and clinical hallucinations"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The anatomy and negation labels used by the penalty terms are assumed correct; they come from external clinical NER pipelines whose errors would push the embedding space the wrong way.","fun_headline_variants_meta":{"raw":{"variants":["SCALPEL turns generative medical LLMs into isotropic X-ray report encoders","Anatomy-negation penalties curb laterality errors in LLM vision-language models","CRC fine-tuning plus cached alignment lifts MIMIC-CXR retrieval to SOTA","Asymmetric offline caching lets medical LLMs train without huge batch memory","Two-stage SCALPEL pipeline fixes anisotropy and clinical hallucinations"]},"model":"grok-4.5","effort":"low","cost_usd":0.00421,"raw_usage":{"total_tokens":1304,"prompt_tokens":849,"num_sources_used":0,"completion_tokens":99,"cost_in_usd_ticks":42104000,"prompt_tokens_details":{"text_tokens":849,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":356,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":849,"tokens_out":99,"duration_ms":6463,"temperature":1.0,"reasoning_tokens":356,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T18:08:11.895907+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Replace the external NER-derived laterality and negation labels with deliberately flipped or random labels and check whether retrieval R@1 and zero-shot F1 on MIMIC-CXR / CheXpert drop sharply; if performance is unchanged, ANAO is not doing the claimed work.","supporting_citations":[],"review_version":1}