{"id":"4a730794-2b0f-4fc0-ac84-f6d51d2d0b00","arxiv_id":"2506.12747","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DSM integrates k-means attention, Mamba state-space layers, diffusion-guided boundary refinement, and CLIP text prompts to segment seen organs and unseen tumors in CT images.","lead":"This paper combines diffusion, state-space models, and text-guided queries to segment tumors in CT scans, including tumor types never seen during training. It reports the best results on three public medical datasets for both seen and unseen segmentation tasks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MSD 'unseen' set is used for ablation and fine-tuning (Tables 4-5), so the 8-of-9 zero-shot superiority may be a model-selection artifact rather than evidence of zero-shot generalization.","rationale":"The reader's identified weakest assumption, the reliability of the negative maximal organ-query response for OOD mask prompts, is a legitimate limitation and could explain failure on unstudied tumor types. However, it is secondary to the evaluation-protocol issue: even if the OOD signal is reliable for the three tested MSD tumor types, the numerical 8-of-9 claim can be inflated by using MSD in ablation and fine-tuning. Conversely, if the clean-holdout check passes, the OOD limitation would affect only generalization beyond the studied categories, not the validity of the reported comparisons. I therefore identify the MSD-informed model selection as the most load-bearing concern. The paper's data-description inconsistencies, such as the training-set counts in Section 5.1, are also present but are minor relative to the evaluation issue. I keep the reader's conditional verdict, but for a stronger reason: the condition should be a pre-registered clean MSD evaluation, not merely clearer ablations and revised data descriptions. Nothing in this critique impugns the authors' intent; the concern is about the protocol as described, which permits selection bias.","tokens_in":16664,"tokens_out":8211,"duration_ms":97810,"concrete_test":"Pre-register a clean zero-shot protocol: freeze all component choices and hyperparameters using only the seen LiTS validation set, then evaluate the frozen DSM exactly once on a held-out MSD split that was never inspected during development. Compare the resulting AUROC, FPR95, and DSC across the three MSD tumor types against ZePT. If DSM does not lead in at least 8 of 9 terms, or if the margin shrinks substantially, the current superiority claim should be attributed to model selection on the MSD test set rather than to the method itself.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central zero-shot claim relies on MSD being a clean, held-out testbed. The paper's own protocol violates this. Section 6.4 (Table 5) selects the four architectural components using colon tumor segmentation on MSD, explicitly labeled as 'MSD (Unseen)'. Section 6.3 (Table 4) further fine-tunes DSM on MSD with 396 training images and 74 test images. No separate MSD validation split or statement that component choices were frozen before inspecting MSD metrics is provided. With six ablation variants compared on MSD metrics, selecting the best-performing row and then reporting the same MSD numbers as zero-shot superiority inflates the apparent advantage even if the components are neutral. The baselines in Table 2 were not tuned on MSD labels in the same way, so the comparison is not apples-to-apples. This evaluation-protocol issue directly affects the reported 8-of-9 statistic, not just generalization to future tumor types.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DSM, a two-stage framework for CT-based organ and tumor segmentation. Stage 1 trains organ queries through a k-Means Mask Mamba decoder that couples the kMaX-DeepLab query-grouping idea with an SSM layer. Stage 2 adds tumor queries, anomaly mask visual prompts derived from negative maximal organ-query responses, diffusion-guided boundary enhancement, and CLIP text-embedding alignment. The authors report state-of-the-art zero-shot segmentation of unseen MSD tumor categories (best in 8 of 9 metrics in Table 2), strong results on seen organs/tumors (Table 3), competitive fully supervised fine-tuning on MSD (Table 4), and a four-component ablation (Table 5).","tokens_in":16855,"tokens_out":10914,"duration_ms":124598,"significance":"If validated, the central claim is clinically valuable: a single model that segments known organs and detects and segments previously unseen tumor categories in CT volumes without per-task retraining. The framework is ambitious and integrates several currently popular mechanisms, and the authors release code, evaluate on a broad set of standard datasets against multiple baseline families (open-vocabulary, OOD detection, prompt-based medical segmentation), and report standard metrics with standard errors. The main barrier is not the method's plausibility but the evaluation protocol: the dataset presented as the unseen testbed is also used for component selection and fine-tuning, which would invalidate the zero-shot interpretation as stated.","major_comments":[{"comment":"The central zero-shot claim is compromised by the use of MSD for model selection and fine-tuning. Section 5.1 defines MSD as the inference set for unseen tumor types, but Section 6.4 explicitly evaluates the ablation on 'Colon Tumor in MSD (Unseen)' and Section 6.3 fine-tunes DSM on 396 MSD images with a 396/74 split. Since the final architecture reported in Table 2 is the configuration selected in Table 5 using MSD labels, the Table 2 results cannot be read as independent evidence of zero-shot generalization; the 'best in 8 of 9' statistic may reflect model selection on the test distribution rather than a transferable capability. The authors should either freeze all architecture and hyperparameter choices before inspecting any MSD metric and state this explicitly, or re-run the zero-shot evaluation on a genuinely held-out set of tumor types that was never used for ablations, fine-tuning, or checkpoint selection.","section":"Sections 5.1, 6.3, 6.4; Tables 4 and 5"},{"comment":"The anomaly mask visual prompt is central to the method, but the notation for the query response is internally inconsistent. Equation (9) defines R_i as an argmax over organ queries, which is a hard assignment (e.g., one-hot) rather than an affinity score; Equation (14) then uses the 'negative maximal query response' as an anomaly score, which only makes sense if R_i contains continuous similarities or logits. As written, A_i in Eq. (14) would be binary and would not encode the degree to which a location is out-of-distribution. The authors need to clarify what R_i actually is, define the computation unambiguously, and provide evidence (e.g., score distributions or failure cases) for the underlying assumption that unseen tumors consistently produce lower organ-query affinity than healthy tissue.","section":"Section 4.2.1, Eqs. (9) and (14)"},{"comment":"The baseline protocol for the key zero-shot comparison is underspecified. The text says that 'Swin UNETR and ZePT ... are fine-tuned on the inference dataset since they were not originally pretrained on this data', which, if taken literally, would mean that the main zero-shot baselines in Table 2 were fine-tuned on the MSD test distribution. It is also not explained how 2D open-vocabulary methods such as ZegFormer, OpenSeg, OVSeg, and Freeseg are applied to 3D CT volumes (e.g., slice-by-slice inference, volume aggregation, or some adaptation). The authors should state precisely which configuration of each baseline is used in each table and describe the adaptation of 2D methods to 3D data.","section":"Section 5.4"}],"minor_comments":[{"comment":"The training-set description contains a numeric inconsistency: the text says Stage 1 uses 2,668 images from 8 public datasets, while the sum of the training rows in Table 1 is 2,688. The sentence about '40% of BTCV, LiTS, and KiTS images (192 images)' is also confusing and should be rewritten to make clear that 20/52/120 images from those three datasets are used for training.","section":"Section 5.1, Table 1"},{"comment":"The ablation table is difficult to read in the current formatting: the checkmark columns are not clearly aligned with the module names, so the reader cannot tell which component is active in each row. Please reformat the table so each row unambiguously shows the subset of kMMM, AMVP, DQR, and CLIP being evaluated.","section":"Table 5"},{"comment":"The sentence 'we calculate the cosine similarity between the text embeddings rather than relying solely on the vision embedding' is confusing because Eq. (20) computes cosine similarity between each text embedding and the projected query embedding. Please rephrase to match the actual computation.","section":"Section 4.2.3"},{"comment":"Equation (8) uses an unnormalized argmax over queries with no mention of how gradients flow through the hard assignment. If this is meant to be the kMaX-DeepLab-style masked attention, please state the exact operation (e.g., straight-through estimation or a separate soft assignment for training).","section":"Section 4.2.1, Eq. (8)"},{"comment":"There is a typo in the class prompt list: 'Celiac Truck' should be 'Celiac Trunk'.","section":"Section 4.2.3, class prompt list"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the evaluation protocol: the MSD 'unseen' set is used in ablations and fine-tuning, which undermines the headline zero-shot claim. This is fixable by re-running the experiments with a clearly held-out set or by explicitly freezing all design choices before inspecting MSD. The method itself appears technically plausible, and the code release and breadth of baselines are strengths."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou should know two things about this one. First, DSM is a sensible integration of existing parts: kMaX-DeepLab's k-means attention, Mamba SSM layers, Perona-Malik diffusion refinement, MaxQuery-style OOD scores, and CLIP text alignment. The reported gains over strong baselines are consistent across tables, and the two-stage query design is new as a combination. Second, the zero-shot evaluation on MSD is contaminated. Section 6.4 uses colon tumor segmentation on MSD, explicitly labeled MSD Unseen, to select the four architectural components, and Section 6.3 fine-tunes DSM on MSD with a 396/74 split. Comparing six ablation variants on the same dataset and then reporting that dataset as the zero-shot test set in Table 2 means the 8-of-9 superiority claim is a model-selection artifact, not evidence of generalization. This is the load-bearing problem.\n\nThe paper does well elsewhere. The seen-organ and seen-tumor results on BTCV, LiTS, and KiTS are clean, and the improvements, while a few DSC points, are uniform. The code is public, which is a plus. The use of CLIP text embeddings for class prompts is standard but the cosine similarity formulation is a reasonable tweak.\n\nSoft spots beyond the protocol issue: the ablation table is ambiguous because the checkmarks in Table 5 are not clearly aligned with the component columns, and the text's description of which rows isolate kMMM does not match the table. The training-set description is internally inconsistent: it says 40 percent of BTCV, LiTS, and KiTS images (192 images) are used for training, but the listed counts are 20, 52, and 120 scans, so 40 percent of that is about 77, not 192. The OOD mask-prompt assumption, that unseen tumors always produce lower organ-query affinity than healthy tissue, is not stress-tested with failure cases. That is a real limitation but might be acceptable if the MSD protocol were fixed.\n\nThe central architectural claim, that the components help, holds up. The zero-shot claim as stated does not. This paper is for researchers working on open-vocabulary medical segmentation who care about evaluation hygiene. I would send it to peer review, but I would expect a major revision requiring the authors to freeze component choices before looking at MSD, or to hold out a separate dataset or split for true zero-shot testing, and to rewrite the ablation and data descriptions.","headline":"Convincing integration of existing parts, but the MSD zero-shot evaluation is contaminated by model selection on the same data, so the 8-of-9 headline does not stand.","tokens_in":17383,"tokens_out":4124,"would_cite":false,"duration_ms":40851,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage query framework using k-Means Mask Mamba and diffusion-guided refinement claims state-of-the-art zero-shot segmentation of unseen tumor categories in CT volumes, beating open-vocabulary and OOD baselines on 8 of 9 metrics.","keywords":["diffusion-guided segmentation","state space models","Mamba","open-vocabulary segmentation","zero-shot tumor segmentation","out-of-distribution detection","medical image segmentation","CT imaging"],"falsifier":"One could test on a set of low-contrast tumors known to resemble their host organ in CT attenuation (e.g., some pancreatic adenocarcinomas or small liver metastases): if the anomaly mask from the negative maximal organ-query response fails to cover these lesions, the zero-shot segmentation would degrade even though organ segmentation remains accurate.","tokens_in":16471,"feed_emoji":"🩻","tokens_out":7636,"duration_ms":80570,"temperature":0.7,"pith_summary":"The paper aims to show that one model can segment organs and also detect and segment tumor types it never saw during training, on CT volumes. It does this by first learning organ queries, then deriving anomaly prompts from the regions where those queries fail, and refining tumor queries with a diffusion-guided decoder plus CLIP text alignment. On three unseen tumor categories from the MSD dataset, the proposed DSM model claims the best score in 8 of 9 evaluation terms, including AUROC and Dice, against open-vocabulary and out-of-distribution baselines. If correct, this would allow clinicians to query a model for a new tumor class by name without retraining for each rare lesion type.","feed_headline":"DSM model tops zero-shot tumor segmentation on 8 of 9 metrics","feed_subtitle":"Organ queries seed anomaly prompts, letting the model segment unseen pancreas, lung, and colon tumors.","key_machinery":"The mechanism that carries the argument is the two-stage query system. Stage 1 trains organ queries in a k-Means Mask Mamba (kMMM) decoder, which combines query-wise argmax grouping, as in k-means clustering, with a state space model (SSM) layer to preserve long-range memory across 3D volumes. Stage 2 builds tumor queries: the negative maximal organ-query response is min-max normalized with a 0.5 threshold to form anomaly mask prompts, which are added to the attention logits so tumor queries focus on out-of-distribution regions. A Diffusion-guided Query Refinement (DQR) decoder then applies a nonlinear diffusion process (Perona–Malik style) to enhance boundary features, and CLIP text embeddings are used to compute a cosine similarity map for class alignment.","core_discovery":"The central claim is that the negative maximal organ-query response — the regions where no learned organ matches the tissue — is a reliable visual prompt for locating novel tumors, and that tumor queries refined by this prompt can segment those tumors zero-shot. The paper further claims that adding a k-Means Mask Mamba layer for long-range query memory, a diffusion-guided query refinement decoder for sharper boundaries, and cosine alignment with CLIP text embeddings is what lets the model outperform existing open-vocabulary, prompt-based, and OOD-detection baselines on pancreas, lung, and colon tumors from the Medical Segmentation Decathlon.","pith_inferences":["Because the anomaly prompt is a simple threshold on the negative maximal response, replacing it with a calibrated likelihood ratio could yield confidence scores for uncertain regions, a natural clinical extension the paper does not explore.","If the training organ taxonomy is incomplete, healthy structures that resemble tumors could be flagged as anomalies; testing on datasets with additional normal anatomy (e.g., vessels, bowel) would reveal this failure mode.","The diffusion-guided fusion is a learned anisotropic smoothing; an ablation that fixes the diffusivity to a hand-crafted edge-stopping function would show how much of the gain comes from learning rather than from the diffusion prior itself.","The paper evaluates only CT; the same query-anomaly mechanism should be testable on MRI or ultrasound, where boundary ambiguity is even more pronounced, and the zero-shot benefit might be larger."],"forward_implications":["A model can be asked, via a text class name, to segment a tumor type it has never seen, without any fine-tuning.","The organ queries double as a general out-of-distribution detector, so the approach could flag any unexpected anatomy, not just the three tested tumor families.","Diffusion-guided boundary enhancement transfers to seen organs and tumors, improving Dice on all tested structures, which suggests the boundary-sharpening idea is task-general.","The kMMM layer's SSM memory stabilizes training on long 3D sequences, which should carry over to other volumetric medical tasks with long-range dependencies."],"supporting_citations":[{"why":"supplies the query-wise argmax / k-means grouping used by the kMMM decoder.","marker":"(Yu et al., 2022)"},{"why":"provides the Mamba SSM layer that maintains long-term memory in query updates.","marker":"(Gu and Dao, 2024)"},{"why":"gives the diffusion-guided feature fusion and boundary enhancement mechanism.","marker":"(Tan et al., 2022)"},{"why":"CLIP text embeddings enable open-vocabulary class alignment.","marker":"(Radford et al., 2021)"},{"why":"ZePT is the strongest zero-shot pan-tumor baseline that DSM compares against and improves upon.","marker":"(Jiang et al., 2024)"},{"why":"MaxQuery is the best OOD-detection baseline whose query-based OOD localization DSM extends.","marker":"(Yuan et al., 2023)"},{"why":"the Universal model supplies the partial labeled strategy and is a CLIP-driven baseline.","marker":"(Liu et al., 2023)"},{"why":"MaskFormer's masked attention decoder is the architectural base for both query stages.","marker":"(Cheng et al., 2021)"}],"fun_headline_variants":["Diffusion and state space model segments unseen tumors zero-shot","Negative organ response locates novel tumors for segmentation","DSM: diffusion and state space for zero-shot tumor segmentation","k-Means Mask Mamba plus diffusion refines tumor queries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach depends on unseen tumors always producing lower organ-query affinity scores than healthy organ tissue, so that the negative maximal response marks tumor regions.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion and state space model segments unseen tumors zero-shot","Negative organ response locates novel tumors for segmentation","DSM: diffusion and state space for zero-shot tumor segmentation","k-Means Mask Mamba plus diffusion refines tumor queries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2813,"prompt_tokens":841,"completion_tokens":1972,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":1905}},"tokens_in":457,"tokens_out":1972,"duration_ms":18494,"temperature":1.0,"reasoning_tokens":1905,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:42:39.229469+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One could test on a set of low-contrast tumors known to resemble their host organ in CT attenuation (e.g., some pancreatic adenocarcinomas or small liver metastases): if the anomaly mask from the negative maximal organ-query response fails to cover these lesions, the zero-shot segmentation would degrade even though organ segmentation remains accurate.","supporting_citations":[{"cited_title":", author Gu, A","cited_arxiv_id":null,"evidence_quote":"provides the Mamba SSM layer that maintains long-term memory in query updates."},{"cited_title":", author Kim, J.W","cited_arxiv_id":null,"evidence_quote":"CLIP text embeddings enable open-vocabulary class alignment."},{"cited_title":", author Huang, Z","cited_arxiv_id":null,"evidence_quote":"ZePT is the strongest zero-shot pan-tumor baseline that DSM compares against and improves upon."},{"cited_title":", author Xia, Y","cited_arxiv_id":null,"evidence_quote":"MaxQuery is the best OOD-detection baseline whose query-based OOD localization DSM extends."},{"cited_title":", author Zhang, Y","cited_arxiv_id":null,"evidence_quote":"the Universal model supplies the partial labeled strategy and is a CLIP-driven baseline."},{"cited_title":", author Schwing, A","cited_arxiv_id":null,"evidence_quote":"MaskFormer's masked attention decoder is the architectural base for both query stages."}],"review_version":1}