{"id":"7c41581b-9849-4229-93f8-314b628ca5db","arxiv_id":"2509.03011","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A ResNet, CBAM, Grad-CAM and T5-based framework produces lesion-aware captions and MES classification for ulcerative colitis endoscopy images, claiming 84.7% MES accuracy and BLEU-4 0.87.","lead":"The paper presents a machine-learning system that writes clinical captions for ulcerative colitis endoscopy images, combining standard vision features with attention maps and clinical metadata. It claims to outperform an existing benchmark method on both caption quality and Mayo Endoscopic Subscore classification.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paired bootstrap significance against Valencia et al. cannot be computed from reported aggregate baseline metrics; p<0.01 is unsubstantiated unless per-sample baseline outputs are provided.","rationale":"The reader's weakest assumption focused on test-split comparability. I agree partially, but the sharper problem is the statistical-significance protocol. The paper never says it re-ran Valencia et al.; it only cites their published aggregate metrics. A paired bootstrap cannot compare aggregate numbers; it needs per-sample paired scores. This makes the p<0.01 claim structurally unsupported as written. The central claim therefore remains conditional: it could be true, but the evidence required to verify it has not been supplied. I do not see an internal mathematical contradiction in the model design, and the ablations are directionally plausible, so I would not reject the paper. I would keep the reader's CONDITIONAL verdict, with the condition now explicitly including release of per-sample baseline outputs and the bootstrap implementation. No code, data, or formal verification is available, so the empirical claims cannot be independently checked.","tokens_in":6094,"tokens_out":5967,"duration_ms":66558,"concrete_test":"Request the evaluation artifact from the authors: exact test-set identifiers, per-sample MES predictions and captions for both the proposed model and a re-implemented Valencia et al. baseline, plus the bootstrap script. Recompute the paired bootstrap (1,000 resamples) from these per-sample outputs. If the per-sample baseline outputs do not exist, the p<0.01 claim is not reproducible. If they exist, verify the test split matches the split used to produce the published baseline numbers and that BLEU-4/ROUGE-L are computed with identical caption references and tokenization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that the proposed model outperforms Valencia et al. [15] with p<0.01—rests on a paired bootstrap described in §4.1: 'a paired bootstrap resampling with 1,000 iterations, yielding a p-value < 0.01 for both classification and captioning improvements over the baseline.' A paired bootstrap requires per-sample outputs from both models on the identical test examples. The paper quotes only aggregate baseline numbers from [15] (77.8% MES accuracy, BLEU-4 0.77, ROUGE-L 0.72) and never states that the baseline was re-implemented or evaluated on the same 15% split. If the baseline was not re-run, the reported p-value cannot be produced by the stated procedure; it is not merely an omitted detail but an impossible calculation from the evidence presented. Even under the generous assumption that the baseline was re-run, the paper must disclose the exact test-split indices, caption references, and metric tokenization to make the pairing meaningful. Without these, the headline superiority claim is unsupported. The ablation study does not address this, since its 'No Attention Fusion' variant is not the Valencia baseline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a lesion-aware image captioning framework for ulcerative colitis endoscopy. The architecture uses a dual-branch ResNet-50: one branch performs MES classification and produces Grad-CAM heatmaps, while the other extracts visual features that are refined by CBAM and multiplied by a Grad-CAM-derived spatial mask. A T5 decoder with visual cross-attention generates captions, optionally conditioned on clinical metadata formatted as natural-language prompts. The authors report MES classification accuracy of 84.7%, BLEU-4 of 0.87, and ROUGE-L of 0.85, compared with 77.8%, 0.77, and 0.72 for Valencia et al. (2023), and claim p<0.01 from a paired bootstrap. They also present ablations removing CBAM, Grad-CAM, clinical prompts, and attention fusion, as well as qualitative examples across MES grades.","tokens_in":6351,"tokens_out":4098,"duration_ms":46276,"significance":"If the reported results are reproducible, the combination of Grad-CAM spatial supervision, CBAM feature enhancement, and clinical prompt conditioning is a plausible, practical contribution to automated endoscopic reporting. The paper includes a useful ablation matrix and reports concrete quantitative gains. However, the central comparison with the prior state of the art is not currently verifiable: the paired-bootstrap significance claim cannot be supported by the aggregate baseline statistics quoted in Table 1, and the test-split comparability with Valencia et al. is not established. The absence of training and metric-computation details further limits reproducibility. The paper's contribution is therefore promising but not yet demonstrated to the standard required for acceptance.","major_comments":[{"comment":"The paired bootstrap significance claim is unsupported by the evidence presented. A paired bootstrap with 1,000 iterations requires per-sample outputs from both models on the identical test items. The manuscript quotes only aggregate baseline numbers from Valencia et al. (77.8%, 0.77, 0.72) and does not state that the baseline was re-run on the same 15% split. As written, the reported p<0.01 cannot be computed from the information in the paper. Please provide either (a) per-sample outputs from a re-implemented baseline evaluated on the exact same test split, with the bootstrap procedure fully specified, or (b) revise the claim to a non-statistical comparison.","section":"§4.1"},{"comment":"The fair-comparison assumption is not established. The dataset is described as introduced in [15] and split 70/15/15 with MES balance, but the paper does not state that this test split matches the evaluation split used by Valencia et al., nor that the caption references and MES labels are identical. If the split or preprocessing differs, the reported improvements in Table 1 may reflect evaluation setup rather than the method. Please specify the exact split indices or a formal justification that the splits coincide, and wherever possible re-run the baseline under the same protocol.","section":"§3.1 and Table 1"},{"comment":"Reproducibility details are insufficient for the central experiments. The paper does not specify the exact T5-base configuration, optimizer, learning rate schedule, batch size, number of training steps, warmup, max sequence length, or random seeds. The value λ=0.2 in Eq. (2) is said to be 'selected through validation' but no search range or criterion is given. These details are needed to assess whether the reported metrics are stable and to allow independent replication.","section":"§3.4–3.5"},{"comment":"There is a potential circularity involving MES labels. The clinical prompt is exemplified as 'MES-2; bleeding: yes', and MES classification accuracy is reported as a headline result. If the ground-truth MES label is used to form the prompt at test time, the 84.7% accuracy is not an independent model prediction. Please clarify whether the prompts are formed from ground-truth labels or from the model's own classifier, and evaluate the setting where prompts are predicted rather than oracle-provided.","section":"§3.4 and §4.1"},{"comment":"A 'T5-based post-processor' or 'grammar-aware refinement' stage is introduced only in the qualitative results and is not described in the methodology. It is unclear whether the BLEU-4 and ROUGE-L scores in Table 1 are computed on the raw model outputs or on the refined captions. If the refined outputs are scored, this post-processing component must be part of the method description and ablated separately; if it is not scored, its role in the reported numbers should be stated.","section":"§4.2"}],"minor_comments":[{"comment":"The dimensions in F' = CBAM(F) ⊙ (1 + αM) are inconsistent as written: M is R^{1×H×W} and F is R^{C×H×W}. Please clarify how the Grad-CAM mask is broadcast or replicated across channels.","section":"Eq. (1)"},{"comment":"The metrics 'alignment score' and 'token-level precision' are mentioned but never defined or reported. Either define them and include results, or remove the mention.","section":"§3.6"},{"comment":"The phrase 'modified T5 decoder incorporated visual cross-attention at each decoding layer' is vague. T5 already has cross-attention to encoder outputs; please specify how the visual features are injected relative to the prompt tokens, and whether the encoder also receives text or is bypassed.","section":"§3.4"},{"comment":"Ablation results are presented without error bars or significance tests. Given that the headline comparison uses p<0.01, the ablations should be accompanied by the same kind of statistical assessment or at least a statement of variance across seeds.","section":"§4.3"},{"comment":"The MES balance of the split is asserted but not quantified. Please report class frequencies in train/validation/test and state whether the balance was stratified by MES or by patient.","section":"§3.1"},{"comment":"Several references are incomplete, e.g., 'Valencia, H.e.a.' and 'Jiang, X.e.a.' lack full author lists and venue details, making it difficult for readers to locate the baseline work.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central technical idea is reasonable and the paper is in scope for a medical imaging venue. My recommendation of major revision is driven by the unverifiable paired-bootstrap comparison and the missing split-comparability and prompt-circularity clarifications, all of which are fixable rather than fundamentally flawed. If the authors cannot re-run the baseline on the identical split, they should substantially weaken the superiority claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a credible applied paper with a clear ablation story, but the headline significance claim over Valencia et al. is not supported by what is reported. The architecture is a known-components combination—ResNet, CBAM, Grad-CAM, T5—and the authors are upfront that their novelty is the first UC captioning system to put those three together. That is a legitimate incremental contribution, and the ablation study (Table 1) gives it internal coherence: each removal hurts, and the 'No Attention Fusion' variant lands near the baseline, which is what you'd expect if the added machinery does real work. The paper earns credit for the 2,355-image expert-annotated dataset and for reporting inference cost and memory usage on a modest GPU.\n\nSoft spots, in order of seriousness. First, the paired bootstrap p<0.01 claim. A paired bootstrap needs per-sample scores from both models on the same test items. The paper quotes only aggregate numbers from [15] and never states that the baseline was re-run on the same split. If it wasn't, the p-value cannot be computed from the reported evidence; it's not just an omitted detail, it's an impossible calculation. The authors need to provide per-sample baseline outputs, or rephrase the claim as an unpaired comparison. Second, the evaluation section is thin: no T5-base configuration, learning rate, batch size, tokenizer details, or exact bootstrap procedure. The test split is said to be 15% but not shown to match [15]'s split exactly. Without those, the 84.7 vs 77.8 gap is plausible but not independently verifiable. Also minor: the clinical prompt uses the same MES labels that are also the classification target, so the captioning gain from prompts is slightly circular; still, it's a legitimate conditioning choice in practice.\n\nBottom line: if the numbers replicate, this is a useful result for endoscopy reporting, and the ablation is a solid start. The paper needs the missing artifacts and a re-run of the baseline before I'd trust the superiority claim. I'd send it to peer review, but with a request for major revision and code/data release.\n\nThis is not a desk reject; it's the kind of paper a serious referee can improve.","headline":"Genuine but unverified incremental gain in UC endoscopic captioning, with an inflated statistical claim that needs per-sample baseline data.","tokens_in":6863,"tokens_out":1441,"would_cite":false,"duration_ms":15856,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that feeding Grad-CAM lesion heatmaps and structured clinical prompts into a ResNet-CBAM-T5 pipeline improves both ulcerative colitis captioning and MES severity classification over the prior state of the art on the same d","keywords":["ulcerative colitis","image captioning","Grad-CAM","CBAM","T5","lesion-aware attention","MES classification","visual-language fusion"],"falsifier":"Replace the Grad-CAM heatmap with a uniform spatial mask of the same smoothness, keeping all other components fixed; if MES accuracy stays at 84.7% and BLEU-4/ROUGE-L stay at 0.87/0.85, the lesion-specific content of the heatmap is not what produces the gain. Separately, re-running the prior baseline on the exact test split and references used here would reveal whether the margin is method or evaluation setup.","tokens_in":5980,"feed_emoji":"🩺","tokens_out":12090,"duration_ms":129263,"temperature":0.7,"pith_summary":"The paper is trying to establish that an automated ulcerative colitis endoscopy captioner produces better reports when it is explicitly made lesion-aware: Grad-CAM heatmaps from a severity-classification branch are multiplied into the visual features, CBAM reweights those features, and structured clinical metadata is fed to a T5 decoder as a prompt. On 2,355 annotated endoscopic images, it reports 84.7% MES classification accuracy, BLEU-4 of 0.87, and ROUGE-L of 0.85, versus 77.8%, 0.77, and 0.72 for the previous system on the same dataset, with a paired-bootstrap p-value below 0.01. If true, the practical payoff is a single pipeline that writes structured, interpretable endoscopy reports and grades disease severity at the same time, with built-in visual explanation of where the model is looking. The qualitative examples show the model sometimes names findings—friability, erythema, superficial ulcers—that are absent from the expert reference captions, which the paper reads as evidence that spatial lesion grounding extracts relevant detail beyond the original annotations.","feed_headline":"Lesion-aware captions beat prior ulcerative colitis reporting","feed_subtitle":"Adding Grad-CAM lesion maps and clinical prompts pushes accuracy to 84.7% and lifts caption quality.","key_machinery":"The mechanism that carries the argument is the Grad-CAM-modulated feature map used as a spatial gate: F' = CBAM(F) ⊙ (1 + αM), where M is the upsampled, normalized Grad-CAM heatmap, α is a learnable scalar, and ⊙ is elementwise multiplication. This turns a post hoc explanation tool into an explicit attention mask during training and inference, so lesion-relevant pixels are amplified before the visual features enter the language decoder. The CBAM module (a Convolutional Block Attention Module) adds channel and spatial attention; the T5 decoder (a pretrained text-to-text transformer) adds cross-attention over the projected visual features; and the clinical metadata prompt conditions the decode","core_discovery":"The central claim is that spatial lesion grounding is not just an explanation layer but a training signal that improves both the vision and language sides of the model. The architecture passes the endoscopic image through two ResNet-50 branches; one produces the MES severity class and its Grad-CAM heatmap, the other extracts features refined by CBAM. Those features are modulated by the Grad-CAM map through F' = CBAM(F) ⊙ (1 + αM), so regions the classifier treats as pathological get amplified before projection into the T5 decoder. The decoder also receives clinical metadata (MES grade, vascular pattern, bleeding, erythema, friability, ulceration) as a natural-language prompt, and training co","pith_inferences":["The same 'classifier heatmap as a training mask' recipe is a natural transfer to other tasks with small, localized lesions and ordinal severity labels, such as Crohn's disease endoscopic indices or polyp surveillance; the paper does not test this.","The metric question is left open: if generated captions add clinically valid findings not in the reference, BLEU and ROUGE will systematically under-reward them, so a clinician-preference study would be the sharper test of usefulness.","The global learned scalar α could be made spatial or per-channel to let lesion emphasis vary by severity; the paper's fixed global scaling is one simple choice among several possible designs.","The dual-loss setup presumes MES classification and captioning share visual features; an experiment that removes the classification branch and compares attention maps would reveal how much of the benefit is from joint training rather than from the Grad-CAM mask."],"forward_implications":["If the claim holds, removing any of the three designed components—CBAM, Grad-CAM spatial supervision, or clinical prompts—should cost measurable performance, exactly as the ablation table shows.","A single model can replace separate classification and reporting tools: the same forward pass yields MES grade, structured lesion tags, a free-text caption, and a Grad-CAM heatmap for clinicians to inspect.","Because generated captions sometimes contain findings absent from reference captions, lesion-aware captioning may surface clinically relevant details that human annotation workflows miss or leave implicit.","At roughly 1.45 seconds per image on a modest GPU, the pipeline is near the threshold for batch or near-real-time use in endoscopic reporting workflows."],"supporting_citations":[{"why":"Supplies the annotated ulcerative colitis dataset, captions, and prior baseline whose numbers the paper's main comparison and ablations are anchored to.","marker":"[15]"},{"why":"Defines Grad-CAM, the gradient-based localization method the paper converts into a spatial supervision mask.","marker":"[13]"},{"why":"Introduces T5, the pretrained text-to-text transformer used as the captioning decoder and prompt-conditioned language model.","marker":"[12]"},{"why":"Shows Grad-CAM-style localization and multilevel attention applied to biomedical image captioning, the lineage this paper extends.","marker":"[6]"}],"fun_headline_variants":["Grad-CAM guides captioning to sharper ulcerative colitis reports","Lesion maps sharpen both captioning and MES grading in UC endoscopy","Fusing lesion heatmaps lifts ulcerative colitis caption quality","Clinical prompts plus lesion attention improve endoscopic reporting","Lesion-aware fusion outdoes baseline in ulcerative colitis captions"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing assumption is that the comparison with the prior baseline is fair: the same test split, the same reference captions, and the same MES labels were used for both systems.","fun_headline_variants_meta":{"raw":{"variants":["Grad-CAM guides captioning to sharper ulcerative colitis reports","Lesion maps sharpen both captioning and MES grading in UC endoscopy","Fusing lesion heatmaps lifts ulcerative colitis caption quality","Clinical prompts plus lesion attention improve endoscopic reporting","Lesion-aware fusion outdoes baseline in ulcerative colitis captions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00058,"raw_usage":{"total_tokens":2513,"prompt_tokens":631,"completion_tokens":1882,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":375,"completion_tokens_details":{"reasoning_tokens":1796}},"tokens_in":375,"tokens_out":1882,"duration_ms":12285,"temperature":1.0,"reasoning_tokens":1796,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:11:17.956970+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the Grad-CAM heatmap with a uniform spatial mask of the same smoothness, keeping all other components fixed; if MES accuracy stays at 84.7% and BLEU-4/ROUGE-L stay at 0.87/0.85, the lesion-specific content of the heatmap is not what produces the gain. Separately, re-running the prior baseline on the exact test split and references used here would reveal whether the margin is method or evaluation setup.","supporting_citations":[{"cited_title":"In: Proceedings of the MICCAI (2023)","cited_arxiv_id":null,"evidence_quote":"Supplies the annotated ulcerative colitis dataset, captions, and prior baseline whose numbers the paper's main comparison and ablations are anchored to."},{"cited_title":"In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)","cited_arxiv_id":null,"evidence_quote":"Defines Grad-CAM, the gradient-based localization method the paper converts into a spatial supervision mask."},{"cited_title":"Neurocomputing378, 303–312 (2020)","cited_arxiv_id":null,"evidence_quote":"Shows Grad-CAM-style localization and multilevel attention applied to biomedical image captioning, the lineage this paper extends."}],"review_version":1}