{"id":"65e223f3-93c2-4583-a74c-dc3fe5a6647a","arxiv_id":"2505.02753","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Internal features of a frozen medical diffusion model, combined with text-guided attention maps and pseudo-healthy inpainting, enable zero-shot tumor segmentation across CT and MRI.","lead":"DiffuGTS segments tumors in CT and MRI scans without ever training on the tumor type being tested, by repurposing the internal features of a frozen medical diffusion model. It reports large, consistent gains over existing zero-shot segmentation methods on four datasets, which could reduce the need for per-tumor annotated training data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MAISI's MSD pretraining overlap with the evaluation sets leaves the zero-shot claim unverified: Supp. F argues against leakage but provides no control experiment, so the reported MSD gains could come from memorized appearance.","rationale":"I agree with the reader's weakest-assumption identification. The central claim is zero-shot generalization, and the most direct threat is that the frozen foundation model was pretrained on the same MSD data used for evaluation. I examined the method for internal inconsistencies (e.g., AOVA aggregation, residual learning) and found the pipeline plausible; the ablations in Tables 4-6 and Supp. C are internally consistent. The leakage concern is therefore the main soft spot. My proposed check is a concrete non-overlap evaluation on LiTS for the liver-tumor leave-one-out setting. If that check shows comparable performance, the concern is resolved; if not, the reported margins on MSD are overstated. This does not change the reader's CONDITIONAL verdict, since the requested control experiment is exactly what the conditional requires.","tokens_in":18041,"tokens_out":8293,"duration_ms":97285,"concrete_test":"Take the released DiffuGTS checkpoint (trained under the leave-one-out protocol, with MSD liver excluded) and evaluate it on the public LiTS liver-tumor CT dataset, which does not overlap MAISI's MSD pretraining, using identical preprocessing, prompt template ('An abnormal CT scan of Liver Tumor'), and the same mask-refinement pipeline. Compare the resulting DSC/NSD with the reported MSD liver scores (63.23/73.58). If the LiTS DSC is materially lower (e.g., >5 points) while the ZePT/Malenia baselines keep their relative ordering, this indicates MSD-specific memorization in the frozen MAISI features/refiner inflates the reported zero-shot gains; if the scores are comparable, the leakage concern is substantially refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that MAISI's pretraining on MSD (acknowledged in Supp. F) does not leak useful appearance or label information into the frozen VAE features or the diffusion-model refinement. The paper's zero-shot superiority is quantified primarily on MSD: five of six evaluated categories in Table 1 and MSD-Brain in Table 2 come from MSD, and the leave-one-out design leaves out an MSD category each time. Supp. F argues that the VAE was trained for reconstruction and that the repaint paradigm is out-of-distribution, but these are arguments, not controls. No experiment compares MSD test performance against a non-overlapping dataset with the same tumor categories, and no probe checks whether the VAE or diffusion model behaves differently on MSD versus non-MSD images. If MAISI has memorized MSD-specific tumor appearances, the AOVA cross-attention and residual maps can exploit that memory, converting a claimed zero-shot task into a near-duplicate retrieval task. The non-MSD results (KiTS23, BraTS23, in-house) are suggestive but do not control for MSD-specific inflation, because they involve different categories, modalities, and/or are private. Without such a control, the headline claim that DiffuGTS 'surpasses current state-of-the-art models across multiple zero-shot settings' is not independently established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DiffuGTS, a framework for zero-shot tumor segmentation that repurposes the frozen VAE encoder and diffusion decoder of MAISI, a medical foundation diffusion model. It first builds anomaly-aware open-vocabulary attention (AOVA) maps by aligning multi-scale VAE features with text prompts for normal and abnormal categories, then refines the resulting masks through training-free latent-space inpainting that synthesizes pseudo-healthy images, followed by pixel-level and feature-level residual learning. The method is evaluated on MSD, KiTS23, BraTS23, and an in-house MRI dataset under leave-one-out and cross-modality zero-shot settings, reporting consistent improvements over SAM-based, vision-language, and anomaly-detection baselines. Ablations support the contribution of each component, and the authors release code.","tokens_in":18303,"tokens_out":2703,"duration_ms":29688,"significance":"If the reported results hold, the paper makes a substantive contribution: it demonstrates that a frozen medical foundation diffusion model can serve as a highly parameter-efficient backbone for zero-shot tumor segmentation across organs and modalities, and it introduces a novel mask-refinement mechanism via latent-space inpainting that goes beyond simple residual maps. The work is well motivated, the ablations are informative, and the experiments cover a broad set of tumor types and both CT and MRI. The authors also ship code and provide detailed dataset documentation. However, the central empirical claim depends on the absence of leakage from MAISI's pretraining on MSD, and the paper asserts statistical significance without reporting the supporting intervals or p-values; these issues need to be resolved before the zero-shot superiority claim can be accepted.","major_comments":[{"comment":"The central zero-shot claim is primarily established on MSD datasets, and MAISI was pretrained on MSD. The rebuttal in Supp. F argues that the VAE was trained for reconstruction and that the repaint paradigm is out-of-distribution, but it provides no control experiment. If MAISI's features have memorized MSD-specific tumor appearances, the AOVA maps and residual refinement could exploit that memory, converting an allegedly zero-shot task into near-duplicate retrieval. This is load-bearing because five of six categories in Table 1 and MSD-Brain in Table 2 come from MSD. Please add a control that does not rely on argument, for example: (a) evaluate the same tumor categories on non-overlapping datasets, or (b) train a probe to check whether the frozen VAE or the diffusion model behaves differently on MSD versus non-MSD images, or (c) retrain MAISI on a disjoint corpus. Without such a control, the headline claim that DiffuGTS 'surpasses current state-of-the-art models across multiple zero-shot settings' is not independently verified.","section":"Supplementary Material F; Tables 1 and 2"},{"comment":"The paper states that '95% CIs were calculated, and a p-value cutoff of less than 0.05 was used to define statistical significance,' but no confidence intervals, p-values, or significance tests appear anywhere in the main text or supplementary material. This makes the assertion of statistical significance unsupported and prevents the reader from judging whether the reported margins over baselines are meaningful. Please either report the CIs and the results of the significance testing, or remove the claim. This is a load-bearing issue because the abstract and conclusions rely on 'surpassing' state-of-the-art methods.","section":"Section 4, Evaluation Metrics"},{"comment":"The prompt setup for SAM-based baselines is unspecified. The text says these methods 'need manual or automatic-generated prompts during testing' but does not state what prompts were used for SAM 2, SaLIP, and H-SAM, or whether prompts were derived from ground-truth annotations. If the baselines received weaker prompts than the text prompts used by DiffuGTS, the comparison would be biased. Please specify the exact prompt protocol (e.g., number of points, boxes, or masks, and whether they come from ground truth) for each SAM-based baseline, or explicitly acknowledge the limitation in the comparison.","section":"Section 4, Competing Methods and Baselines"}],"minor_comments":[{"comment":"There is a typo in the prompt template: 'A nomal CT scan/MRI of {organ name}' should read 'A normal CT scan/MRI of {organ name}'.","section":"Section 3.1, Textual Prompt Composition"},{"comment":"The caption contains 'competting methods'; this should be 'competing methods'.","section":"Table 3 caption"},{"comment":"The notation for the noise schedule is confusing: the text defines α as a product over timesteps using β_s, but the equation uses α_t in the Gaussian distribution. Clarify the relationship between α_t, α, and β_s to avoid ambiguity.","section":"Section 3.2, Eq. (3)"},{"comment":"The robustness analysis with misleading prompts is useful, but it would be stronger if it also reported quantitative results (e.g., false-positive rates) rather than only showing that no foreground mask is predicted in two examples.","section":"Supplementary Material D"}],"recommendation":"major_revision","confidential_remarks":"The leakage concern is the main risk to the paper's central claim. The authors' rebuttal in Supp. F is reasonable but not a substitute for a control experiment; I would encourage the editor to require an additional experiment on non-overlapping data with the same tumor categories. The missing statistical reporting is also a significant omission that must be fixed. The self-citations to ZePT and Malenia are standard baseline comparisons and do not constitute circularity. The paper is otherwise technically sound and the method is novel; a major revision with the requested controls and statistics could bring it to an acceptable level."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's central claim—that frozen features from a medical foundation diffusion model can be repurposed for zero-shot tumor segmentation—is plausible and largely supported by the ablations. The new thing is the package: anomaly-aware open-vocabulary attention maps built from frozen MAISI VAE features, followed by latent inpainting to synthesize pseudo-healthy images and residual learning to refine masks. Individually the ingredients are familiar, but the combination is original, and the authors are careful to ablate each piece. The supplementary control—adding MAISI to ZePT and Malenia improves them but doesn't close the gap—is good evidence that the gains come from the specific designs, not just from having a big diffusion model in the loop.\n\nThe main soft spot is the evaluation confound. MAISI was pretrained on MSD, and five of the six categories in Table 1 plus MSD-Brain in Table 2 come from MSD. The authors respond with a reasoned argument in Supp. F: the VAE was trained for reconstruction, and the repaint-based pseudo-healthy generation is out-of-distribution for MAISI. That's plausible, but it's an argument, not a control. A direct experiment on a disjoint dataset, or a per-dataset breakdown comparing MSD vs. non-MSD performance, would have settled it. That said, the non-MSD numbers (KiTS23, BraTS23, private in-house MRI) also show large gains, so even if the MSD results were partly inflated by leakage, the generalizable trend holds.\n\nTwo smaller issues. The paper states that 95% CIs were calculated and p<0.05 was used, but never reports any CIs or p-values. That's sloppy and easy to fix. Also, the prompt setup for SAM-based baselines is undisclosed, which hurts reproducibility.\n\nProportionately: this is a revision-ready paper, not a desk reject. The ablations are solid, the writing is clear, the code is promised, and the central argument holds up well enough to survive the leakage concern. The authors should add a leakage control, report the promised statistics, and specify the SAM prompt protocol. The computational cost is high (12T FLOPs) but acknowledged as a limitation.\n\nThis paper is for anyone working on zero-shot medical segmentation or transferring medical foundation models. It deserves a serious referee, and I'd bring it to a reading group.","headline":"A well-constructed zero-shot tumor segmentation paper with large reported gains; the MAISI/MSD training overlap is a real caveat but not fatal, and the paper deserves a serious referee.","tokens_in":18862,"tokens_out":2591,"would_cite":true,"duration_ms":29797,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The internal representations of a frozen medical foundation diffusion model can be repurposed into zero-shot tumor segmenters across organs and modalities.","keywords":["zero-shot tumor segmentation","frozen diffusion models","open-vocabulary attention maps","anomaly detection","pseudo-healthy image synthesis","latent space inpainting","3D medical imaging","text-prompt segmentation"],"falsifier":"Rerun the leave-one-out and cross-modality benchmarks with the same foundation diffusion model trained on a corpus that excludes MSD, and compare results on MSD test cases against results on datasets never used in pretraining; if the large margins over prior zero-shot and anomaly-detection baselines mostly evaporate on the non-overlapping MSD cases while persisting elsewhere, the apparent generalization is driven by pretraining overlap rather than by the attention-map and inpainting machinery.","tokens_in":17816,"feed_emoji":"🩻","tokens_out":8344,"duration_ms":87858,"temperature":0.7,"pith_summary":"This paper aims to show that a frozen medical foundation diffusion model, trained to generate 3D CT and MRI anatomy, already carries enough spatial knowledge to segment tumors it was never trained to segment. The proposed framework, DiffuGTS, builds anomaly-aware open-vocabulary attention maps from the model's internal VAE features and text prompts naming a normal organ or an abnormal finding, then refines those maps by inpainting the suspected tumor region to a pseudo-healthy version and comparing the two at pixel and feature levels. The authors report large zero-shot margins over prior text-guided and anomaly-detection methods across seven tumor categories, four datasets, and both CT and MRI, including a 21-point Dice similarity gain when moving from CT training to MRI test data. If the claim holds, a single frozen generative model could serve as a parameter-efficient backbone for segmenting unseen diseases without retraining.","feed_headline":"Frozen diffusion model segments tumors it was never trained on","feed_subtitle":"Attention maps plus pseudo-healthy inpainting beat zero-shot baselines across CT and MRI.","key_machinery":"The load-bearing machinery is the anomaly-aware open-vocabulary attention (AOVA) map: cross-attention matrices between pixel-level queries from the frozen VAE encoder's multi-scale features and text attribution keys projected from normal/organ and abnormal/disease prompts are resized and summed across feature levels to localize anomalies in a category-free, text-prompt-driven way. A second mechanism is training-free latent space inpainting: the suspected tumor region, enlarged by a coefficient, is re-sampled from the frozen diffusion model while known healthy regions are copied from the input latent, and the resulting pseudo-healthy image is compared with the original through pixel-level and feature-level residuals that are combined and thresholded into a final mask. Only lightweight adapters, MLP projections, and attention layers are trained; the diffusion model and the VAE stay frozen.","core_discovery":"The central discovery is that the internal representations of a frozen medical foundation diffusion model—specifically the multi-scale features of its 3D variational autoencoder—can be repurposed as zero-shot semantic cues for tumor localization, and that the same frozen model can then be used to clean up those cues. Text embeddings of normal and abnormal prompts are projected into attribution keys that attend over the VAE's pixel features, producing anomaly-aware open-vocabulary attention maps that mark tumor candidates without a preset category list. A training-free latent inpainting step replaces the candidate region by sampling from the diffusion model, producing a pseudo-healthy volume that preserves non-tumor anatomy; subtracting the original volume from this pseudo-healthy version at both pixel and latent-feature levels yields the final segmentation. On leave-one-out and cross-modality tests, the authors report that this combination outperforms existing zero-shot segmentation and anomaly-detection methods, with the largest gains coming from the mask-refinement stage rather than from the attention maps alone.","pith_inferences":["If the same attention-plus-inpainting recipe transfers to other generative foundation models with a latent anatomy prior, it could convert any such model into an anomaly segmenter without retraining it.","The reported efficiency is a trade-off: the method trades high inference FLOPs for few trainable parameters, so distilling the frozen model's features into a compact segmenter is a natural next step that the paper leaves open.","The cleanest test of genuine zero-shot generalization would be to compare performance on evaluation datasets that overlap the foundation model's pretraining with performance on non-overlapping datasets, since the paper's supplementary data-leakage discussion argues, but does not experimentally demonstrate, that overlap leaks no label information.","A wider taxonomy of anomalies—cysts, hemorrhage, inflammation, or imaging artifacts—could be probed with the same category-free attention maps, because the method does not rely on a closed training category list."],"forward_implications":["A new tumor type can be added at test time by writing a text prompt; no training examples or labels for it are needed.","Cross-modality generalization is a direct corollary: the framework passes from CT-trained attention maps to MRI tumor segmentation with the same weights, with much larger gains than text-only baselines.","The method is parameter-efficient in the sense that only small adapter and projection modules are trained, although inference FLOPs are high because the generative backbone runs at test time.","Mask quality improves because the pseudo-healthy comparison is anchored to the input anatomy; replacing direct organ synthesis with inpainting preserves healthy regions and sharpens tumor boundaries."],"supporting_citations":[{"why":"Supplies the frozen medical foundation diffusion model whose VAE features and generative decoder carry the anatomical prior used for feature extraction and mask refinement.","marker":"[16]"},{"why":"Provides the training-free inpainting formulation that the paper adapts to latent space for pseudo-healthy image synthesis.","marker":"[30]"},{"why":"Supplies the latent-space adaptation of inpainting used to align the enlarged anomaly mask with VAE latent dimensions.","marker":"[12]"},{"why":"Defines the zero-shot pan-tumor segmentation baseline the paper compares against and builds on for mask-text alignment.","marker":"[22]"},{"why":"Provides the mask-attribute alignment baseline and the strongest text-driven zero-shot competitor the paper exceeds.","marker":"[23]"},{"why":"Supplies the contrastive vision-language alignment objective used to tie attention-map class embeddings to text embeddings.","marker":"[34]"},{"why":"Provides the 404 pathology-free CT scans used as normal samples for anomaly detection training.","marker":"[40]"},{"why":"Supplies the Dice loss used to supervise attention maps and feature-level residual maps with partial segmentation labels.","marker":"[31]"}],"fun_headline_variants":["Zero-shot tumor segmentation via frozen diffusion attention maps","Frozen diffusion model learns tumor segmentation on the fly","Anomaly-aware attention from frozen diffusion finds tumors","Pseudo-healthy residual maps boost zero-shot tumor masks","Open-vocabulary prompts guide frozen diffusion to segment tumors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise, stated in the supplementary material's data-leakage analysis rather than proven by experiment, is that the foundation model's pretraining on partly overlapping public evaluation datasets does not leak useful appearance or label information into the variational-autoencoder features or the pseudo-healthy images; if that overlap helps the scores, the zero-shot claim is overstated.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot tumor segmentation via frozen diffusion attention maps","Frozen diffusion model learns tumor segmentation on the fly","Anomaly-aware attention from frozen diffusion finds tumors","Pseudo-healthy residual maps boost zero-shot tumor masks","Open-vocabulary prompts guide frozen diffusion to segment tumors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000589,"raw_usage":{"total_tokens":2762,"prompt_tokens":941,"completion_tokens":1821,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":1744}},"tokens_in":557,"tokens_out":1821,"duration_ms":16184,"temperature":1.0,"reasoning_tokens":1744,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:42:24.936894+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the leave-one-out and cross-modality benchmarks with the same foundation diffusion model trained on a corpus that excludes MSD, and compare results on MSD test cases against results on datasets never used in pretraining; if the large margins over prior zero-shot and anomaly-detection baselines mostly evaporate on the non-overlapping MSD cases while persisting elsewhere, the apparent generalization is driven by pretraining overlap rather than by the attention-map and inpainting machinery.","supporting_citations":[{"cited_title":"Repaint: Inpainting using denoising diffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"Provides the training-free inpainting formulation that the paper adapts to latent space for pseudo-healthy image synthesis."},{"cited_title":"Latentpaint: Image inpainting in latent space with diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the latent-space adaptation of inpainting used to align the enlarged anomaly mask with VAE latent dimensions."},{"cited_title":"Zept: Zero-shot pan-tumor segmentation via query-disentangling and self-prompting","cited_arxiv_id":null,"evidence_quote":"Defines the zero-shot pan-tumor segmentation baseline the paper compares against and builds on for mask-text alignment."},{"cited_title":"Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment","cited_arxiv_id":"2410.15744","evidence_quote":"Provides the mask-attribute alignment baseline and the strongest text-driven zero-shot competitor the paper exceeds."},{"cited_title":"V-net: Fully convolutional neural networks for volumetric medical image segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the Dice loss used to supervise attention maps and feature-level residual maps with partial segmentation labels."}],"review_version":1}