{"id":"de905108-46a7-49bc-92eb-8bc6031cde1e","arxiv_id":"2508.04534","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"ExplainSeg obtains segmentation masks from classification-only training by converting integrated-gradient heatmaps of a fine-tuned DINO model into masks via NCut or morphology and DenseCRF.","lead":"This paper proposes ExplainSeg, a method that derives segmentation masks from a medical image classifier by fine-tuning a DINO vision transformer on image-level labels and converting its explainability heatmap into a mask. A generalist reader might care because pixel-level medical annotations are expensive, and classification labels are easier to collect.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Kvasir-SEG fine-tuning labels are derived from ground-truth masks, contradicting the paper's central 'No Masks Needed' claim.","rationale":"The reader's verdict is REJECT, and my independent read arrives at the same conclusion via a more direct route. The paper's central claim has two parts: (i) no ground-truth masks are used in fine-tuning and (ii) ExplainSeg achieves state-of-the-art performance. The first is internally falsified by the paper's own protocol. For Kvasir-SEG, patch-level labels are defined by thresholding the ground-truth masks; the fine-tuning head is therefore trained on labels that carry pixel-level spatial information. No amount of post-processing or XAI can restore the claim that mask information was absent. This is more load-bearing than the attribution-localization concern: even if IG+noise-tunnel perfectly localized classification evidence, that evidence on Kvasir-SEG is partly a byproduct of mask-derived patch labels. The SOTA claim is also compromised by Table 2, where MaskCut beats ExplainSeg on Kvasir-SEG (mIoU 41.3 vs 28.6), but the mask contradiction alone suffices for rejection. I therefore see no reason to alter the reader's REJECT verdict.","tokens_in":7897,"tokens_out":7012,"duration_ms":80135,"concrete_test":"Obtain the data-preparation code for Kvasir-SEG (or independently reimplement the patch-labeling step) and check whether each patch's label y_i is computed by intersecting the patch with the ground-truth mask. If yes, the Section 4.1 'no mask information' assertion is false. This is a direct, low-cost verification that settles the contradiction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 describes Kvasir-SEG preparation: because the dataset contains only positive samples, the authors split each image into patches and assigned labels 'based on the presence of polyp pixels in the corresponding mask.' This directly contradicts the sentence in the same section stating that 'no information from the ground truth segmentation masks is used during the classification network finetuning stage.' The fine-tuning labels for one of the three datasets are therefore derived from the very masks the method claims not to need. This is not a wording issue: patch labels encode spatial information from the mask, so the classifier is trained with localization signal on Kvasir-SEG. Consequently, the title and abstract's 'No Masks Needed' claim is unsupported for the full evaluation; at best it could hold for CBIS-DDSM and NuInsSeg. The internal contradiction alone invalidates the central claim as presented.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ExplainSeg, a pipeline that fine-tunes a DINO-pretrained Vision Transformer with a classification head on image-level labels, applies Integrated Gradients with a noise tunnel to obtain attribution maps, and converts these maps into binary segmentation masks via morphology or normalized cuts followed by DenseCRF. The method is evaluated on CBIS-DDSM, NuInsSeg, and Kvasir-SEG, with comparisons to TokenCut, MICRA-Net, and MaskCut. The authors claim state-of-the-art performance and assert that no ground-truth segmentation mask information is used during classification fine-tuning.","tokens_in":8100,"tokens_out":4382,"duration_ms":50553,"significance":"If fully substantiated, the approach would be practically valuable: it offers a way to derive pixel-level segmentations from image-level labels in medical imaging, where dense annotation is expensive. The paper uses standard, reproducible components (DINO, Integrated Gradients, NCut, DenseCRF) and reports an ablation over four variants. However, the central claims are not established as stated: one dataset's classification labels are derived directly from the ground-truth masks, the reported Kvasir-SEG comparison is won by MaskCut, and no statistical significance or variance information is provided. The baseline set is also too narrow to support a general 'state-of-the-art' claim.","major_comments":[{"comment":"This paragraph states that Kvasir-SEG patches were labeled 'based on the presence of polyp pixels in the corresponding mask,' and then states in the same section that 'no information from the ground truth segmentation masks is used during the classification network finetuning stage.' These statements directly contradict each other. Since Kvasir-SEG is one of the three evaluation datasets, the title/abstract claim 'No Masks Needed' is not supported for the full evaluation. To fix this, the authors must either construct Kvasir-SEG classification labels without using mask-derived localization (e.g., using image-level labels or external negative samples) and rerun all affected experiments, or explicitly restrict the no-mask claim to CBIS-DDSM and NuInsSeg. As written, this is a load-bearing inconsistency.","section":"Section 4.1, Kvasir-SEG paragraph"},{"comment":"On Kvasir-SEG, MaskCut achieves mIoU 41.3 and Dice 48.4, while ExplainSeg (XNCut) achieves mIoU 28.6 and Dice 41.4, a 12.7-point mIoU deficit. This contradicts the Section 1 claim that ExplainSeg 'achieves state-of-the-art performance and significantly outperforms other segmentation methods using pre-trained medical image segmentation models.' The text later acknowledges that ExplainSeg 'does not attain the top score on Kvasir-SEG,' but the abstract and introduction still make an unqualified state-of-the-art claim. The claims must be revised to match the reported results, or the scope of the claims must be narrowed.","section":"Section 4.2, Table 2"},{"comment":"No error bars, confidence intervals, or significance tests are reported; all numbers appear to come from a single run. For example, the NuInsSeg mIoU gap between ExplainSeg (13.1) and MaskCut (9.3) is modest and could easily be within run-to-run variation, especially with morphological post-processing and NCut. The phrase 'significantly outperforms' cannot be assessed without multiple seeds and dispersion measures. At minimum, report mean plus/minus standard deviation over several training seeds and, where feasible, a paired test over the evaluation images.","section":"Section 4.2, Tables 1 and 2"},{"comment":"The comparison set is too narrow to support a general state-of-the-art conclusion. The authors compare against two unsupervised object-discovery methods (TokenCut, MaskCut) and MICRA-Net, a supervised method from a different microscopy domain. Prior XAI-to-segmentation works [15, 16] are excluded because no public implementation is available, but the contribution is framed generically. The authors should either broaden the baseline set to include weakly supervised segmentation methods and other XAI-based segmentation approaches, or soften the state-of-the-art claim to a claim of competitiveness on the considered baselines and datasets.","section":"Section 4.2, Baselines Selection"}],"minor_comments":[{"comment":"The method section is incomplete: equations (1) and (2) are referenced but missing, and the text jumps from the problem formulation to equation (3). Please restore the full derivation and label equations consistently.","section":"Section 3"},{"comment":"The pseudo-code contains undefined or garbled symbols (e.g., the placeholder characters shown as '�'), and the inputs 'optional training' versus 'input image' are confusing. Use standard mathematical notation and clearly distinguish training-time and inference-time inputs.","section":"Algorithm 1"},{"comment":"Several post-processing hyperparameters are not specified: morphology kernel sizes, threshold selection, NCut segment count, and DenseCRF parameters. These are essential for reproducibility. Please provide a table of all hyperparameters or release the configuration.","section":"Section 4.1, Implementation details"},{"comment":"The 'MIOU' header is capitalized inconsistently; use 'mIoU' throughout.","section":"Table 2"},{"comment":"Some references appear only tangentially used (e.g., [17] for 'Growing a brain') and the reference list has formatting inconsistencies. Please check that every reference is cited and that citation indices match the bibliography.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the internal contradiction in Section 4.1: for Kvasir-SEG, the fine-tuning labels are literally constructed from the ground-truth masks, yet the paper claims no mask information is used. This is not a stylistic issue; it invalidates the title-level claim for one of the three datasets. I recommend that the editor require the authors to address this explicitly and, if necessary, rerun the Kvasir-SEG experiments with truly mask-free labels. The state-of-the-art claim is also unsupported by Table 2, where MaskCut wins Kvasir-SEG by a wide margin. The paper may be salvageable with narrower claims and additional experiments, but the current version is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know: the pipeline (fine-tune DINO linear head, IG noise-tunnel heatmap, fuse with ViT features, NCut/DenseCRF) is a sensible combination, and the ablation of four variants across three medical datasets is genuinely informative. The problem is the central claim. Section 4.1 says that 'no information from the ground truth segmentation masks is used during the classification network finetuning stage,' but in the same section they explain that Kvasir-SEG patch labels are assigned based on whether polyp pixels are present in the corresponding mask. That is mask-derived localization signal. The title's 'No Masks Needed' does not hold for a third of the evaluation, and the same mask-derived labels are arguably at work for NuInsSeg if image-level presence is inferred from the instance masks.\n\nThat contradiction is load-bearing because the paper's novelty is supposed to be classification-only supervision. If you set it aside, the results are modest: mIoU 13–32%, no error bars, and the 'state-of-the-art' claim in Section 1 conflicts with Table 2, where MaskCut gets 41.3 mIoU on Kvasir-SEG versus the method's 28.6. On NuInsSeg the edge over MaskCut is 13.1 vs 9.3, which could easily be noise — no significance tests are reported. The baseline set is also thin; including TokenCut and MICRA-Net is fine, but omitting weakly supervised segmentation baselines makes the comparison incomplete.\n\nCredit where it's due: the variant ablations (Fusion & Morphology vs NCut vs XAI) are a reasonable way to see which stage matters, and the authors are upfront that prior XAI-to-segmentation work exists [15,16]. The idea of fine-tuning on medical labels then explaining is a legitimate extension. But the execution does not back the claims. No code, no hyperparameter sensitivity, and the implementation details are sparse.\n\nWho is this for? Someone working on weakly supervised medical segmentation might read it to see one concrete recipe, but they should not treat the results as proof that XAI alone can replace masks. If this crossed my desk, I would not send it to external reviewers as is; the central claim is internally contradicted. I would ask for a major revision that redefines the contribution (e.g., 'XAI-based segmentation after image-level label fine-tuning' without claiming no mask information), adds error bars, and includes a stronger baseline set. As it stands, I would not cite it.","headline":"Plausible pipeline, but the 'No Masks Needed' claim collapses on Kvasir-SEG because its patch labels come from the ground-truth masks.","tokens_in":8605,"tokens_out":3702,"would_cite":false,"duration_ms":39505,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ExplainSeg derives segmentation masks from classification labels alone: fine-tune a DINO ViT, turn Integrated Gradients heatmaps into masks via normalized cuts, and beat three baselines on two of three datasets without pixel annotations.","keywords":["medical image segmentation","explainable AI","integrated gradients","image-level supervision","transfer learning","normalized cut","vision transformer","self-supervised pre-training"],"falsifier":"The direct test is a label-shuffle control: retrain the pipeline with the fine-tuning labels randomly permuted, keeping images and all other settings identical. If the derived masks still match the ground-truth masks about as well as in the reported runs, then the fine-tuned classifier is not what localizes the object and the central mechanism fails. A complementary observational check: split results by whether the classifier's prediction was correct; if images classified wrongly yield equally good masks against the ground truth, classification and localization have decoupled.","tokens_in":7786,"feed_emoji":"🩻","tokens_out":23107,"duration_ms":188901,"temperature":0.7,"pith_summary":"The paper tries to establish that pixel-level medical segmentation can be produced from image-level classification labels only, without any pixel-wise annotation at fine-tuning time. Its pipeline, ExplainSeg, fine-tunes a self-supervised Vision Transformer as a classifier on medical images, computes an explainable-AI relevance map (Integrated Gradients with a noise tunnel) for the predicted class, and converts that map into a segmentation mask using normalized cuts or morphology with DenseCRF refinement. Across three medical domains — mammography, histopathology, and polyp endoscopy — the best variant reports higher accuracy than three pre-trained-model baselines (TokenCut, MICRA-Net, MaskCut) on the first two domains and competitive accuracy on the third, with its largest margin on mammography. If the claim holds, medical segmenters could be built from the diagnostic labels hospitals already collect, sidestepping the costly expert pixel-labelling that currently bottlenecks medical AI. The crux is whether fine-tuned classification attributions localize the target structure rather than global image context; that is the premise the experiments must carry.","feed_headline":"No masks needed: XAI heatmaps turn class labels into pixel masks","feed_subtitle":"No pixel-wise labels: attribution maps from a fine-tuned classifier outdo baselines on mammography and histopathology.","key_machinery":"The load-bearing object is the attribution map: Integrated Gradients with a noise tunnel, applied to the fine-tuned classifier's prediction, assigns each pixel a relevance score for the predicted class. This is the step that turns 'why did the network predict this class' into 'where is the object'. The backbone is a DINO-pretrained Vision Transformer fine-tuned with a linear classification head, whose self-supervised features give the heatmaps spatial structure. Two mechanisms carry the rest: multiplying the relevance map by the ViT's intermediate feature map to denoise outlier pixels, and converting relevance values into a clean binary mask either by normalized-cut spectral clustering or by","core_discovery":"The paper claims that a classification network fine-tuned on image-label pairs keeps enough spatial information to yield a segmentation mask. ExplainSeg implements this: a DINO-pretrained Vision Transformer is fine-tuned on class labels only; Integrated Gradients with a noise tunnel turns the prediction into a relevance map; and normalized cuts converts that map into a binary mask, refined by DenseCRF. The best variant (XAI + NCut) reports 31.2% mIoU / 43.7% Dice on CBIS-DDSM, roughly double the best baseline's Dice, and beats all three baselines on NuInsSeg as well, staying competitive on Kvasir-SEG where MaskCut leads. The fine-tuning stage uses no ground-truth masks — that is the sense of","pith_inferences":["Editorial note: the 'no masks' framing is strictest for CBIS-DDSM and NuInsSeg. For Kvasir-SEG, Section 4.1 shows the patch-level classification labels are generated from the ground-truth polyp masks, so mask-derived supervision enters indirectly; the literal claim that holds is 'no masks during fine-tuning,' not 'no mask-derived labels anywhere.'","Editorial note: Section 4.1 states that both training and validation images are passed through the model to generate the evaluated masks, so the reported scores include in-sample images; a validation-only rerun would give the cleaner estimate of generalization.","Editorial extension: the mechanism should transfer to other medical tasks whose diagnostic label is solved from the lesion's own pixels, and should degrade where global context predicts the label; the large CBIS-DDSM gain suggests low-contrast, non-RGB modalities are the sweet spot.","Editorial note: in the text as provided, the method section jumps from the problem formulation (Section 3.1) to the feature-relevance fusion (Section 3.5); the fine-tuning objective and the integrated-gradients formula are not shown, so the exact training signal is asserted rather than demonstrated."],"forward_implications":["If the claim holds, segmentation training no longer needs pixel-wise masks: any dataset with diagnostic labels becomes a candidate for building a segmenter.","On CBIS-DDSM the best variant (31.2% mIoU, 43.7% Dice) more than doubles the best baseline, indicating the largest gains appear on low-contrast, non-RGB modalities where general-purpose unsupervised methods collapse.","The two-stage design (explanation, then post-processing) is modular: the NCut variant wins on mammograms, the morphology variant posts the top mIoU on histopathology, and the fusion-with-features variant leads on endoscopy — so the configuration can be chosen per modality.","Because the masks are literally explanations of the classification decision, the clinician sees not only where the model segments but why the model made its diagnosis, coupling segmentation with interpretability."],"supporting_citations":[{"why":"Supplies the DINO self-supervised Vision Transformer that the linear classifier is fine-tuned on top of; the backbone of the entire pipeline.","marker":"[12]"},{"why":"Integrated Gradients, the attribution method that converts the classifier's prediction into the relevance map that becomes the segmentation mask.","marker":"[13]"},{"why":"The Captum implementation of Integrated Gradients with a noise tunnel; the concrete XAI machinery used to denoise the heatmap.","marker":"[18]"},{"why":"Normalized cuts spectral clustering; the post-processing step that turns relevance maps into binary masks in the best-performing variant.","marker":"[22]"},{"why":"TokenCut, the normalized-cut-based unsupervised baseline on ViT features that ExplainSeg is compared against and outperforms on two datasets.","marker":"[4]"},{"why":"CutLER/MaskCut, the unsupervised object-discovery baseline that is ExplainSeg's strongest comparator, leading on Kvasir-SEG.","marker":"[5]"},{"why":"MICRA-Net, the medical-domain weakly supervised baseline that ExplainSeg consistently outperforms.","marker":"[29]"},{"why":"CBIS-DDSM mammography dataset, the evaluation domain where ExplainSeg's margin over baselines is largest.","marker":"[24]"},{"why":"NuInsSeg nuclei dataset, the histopathology evaluation domain where ExplainSeg beats all three baselines.","marker":"[25]"},{"why":"Kvasir-SEG polyp dataset, the endoscopy evaluation domain whose patch-level classification labels are derived from the ground-truth masks.","marker":"[26]"}],"fun_headline_variants":["No masks, no problem: XAI turns labels into segmentation","Zero-mask segmentation: fine-tuned XAI doubles Dice","Class labels only: XAI heatmaps become pixel masks","ExplainSeg: label-only training, XAI segmentation"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that, once the classifier is fine-tuned on image-level labels, its attribution heatmaps concentrate on the pixels of the target structure — and not on global cues such as breast density, background tissue, or staining variation that can also predict the class.","fun_headline_variants_meta":{"raw":{"variants":["No masks, no problem: XAI turns labels into segmentation","Zero-mask segmentation: fine-tuned XAI doubles Dice","Class labels only: XAI heatmaps become pixel masks","ExplainSeg: label-only training, XAI segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000529,"raw_usage":{"total_tokens":2344,"prompt_tokens":655,"completion_tokens":1689,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":399,"completion_tokens_details":{"reasoning_tokens":1633}},"tokens_in":399,"tokens_out":1689,"duration_ms":14457,"temperature":1.0,"reasoning_tokens":1633,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:54:45.852159+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The direct test is a label-shuffle control: retrain the pipeline with the fine-tuning labels randomly permuted, keeping images and all other settings identical. If the derived masks still match the ground-truth masks about as well as in the reported runs, then the fine-tuned classifier is not what localizes the object and the central mechanism fails. A complementary observational check: split results by whether the classifier's prediction was correct; if images classified wrongly yield equally good masks against the ground truth, classification and localization have decoupled.","supporting_citations":[{"cited_title":"Emerging properties in self-supervised vision transformers,","cited_arxiv_id":null,"evidence_quote":"Supplies the DINO self-supervised Vision Transformer that the linear classifier is fine-tuned on top of; the backbone of the entire pipeline."},{"cited_title":"Axiomatic attribution for deep networks,","cited_arxiv_id":null,"evidence_quote":"Integrated Gradients, the attribution method that converts the classifier's prediction into the relevance map that becomes the segmentation mask."},{"cited_title":"Captum: A unified and generic model interpretability library for pytorch,","cited_arxiv_id":null,"evidence_quote":"The Captum implementation of Integrated Gradients with a noise tunnel; the concrete XAI machinery used to denoise the heatmap."},{"cited_title":"Normalized cuts and image segmentation,","cited_arxiv_id":null,"evidence_quote":"Normalized cuts spectral clustering; the post-processing step that turns relevance maps into binary masks in the best-performing variant."},{"cited_title":"Self- supervised transformers for unsupervised object discovery using normalized cut,","cited_arxiv_id":null,"evidence_quote":"TokenCut, the normalized-cut-based unsupervised baseline on ViT features that ExplainSeg is compared against and outperforms on two datasets."},{"cited_title":"Cut and learn for unsupervised object detection and instance segmentation,","cited_arxiv_id":null,"evidence_quote":"CutLER/MaskCut, the unsupervised object-discovery baseline that is ExplainSeg's strongest comparator, leading on Kvasir-SEG."},{"cited_title":"Microscopy analysis neural network to solve detection, enumeration and segmentation from image-level annotations,","cited_arxiv_id":null,"evidence_quote":"MICRA-Net, the medical-domain weakly supervised baseline that ExplainSeg consistently outperforms."},{"cited_title":"A curated mammography data set for use in computer-aided detection and diagnosis research,","cited_arxiv_id":null,"evidence_quote":"CBIS-DDSM mammography dataset, the evaluation domain where ExplainSeg's margin over baselines is largest."},{"cited_title":"Nuinsseg: A fully annotated dataset for nuclei instance segmentation in h&e-stained histological images,","cited_arxiv_id":null,"evidence_quote":"NuInsSeg nuclei dataset, the histopathology evaluation domain where ExplainSeg beats all three baselines."},{"cited_title":"Kvasir-seg: A segmented polyp dataset,","cited_arxiv_id":null,"evidence_quote":"Kvasir-SEG polyp dataset, the endoscopy evaluation domain whose patch-level classification labels are derived from the ground-truth masks."}],"review_version":1}