{"id":"2ec6f2a5-6b6e-4b12-9f0d-0dba2e0d0a3a","arxiv_id":"2506.02854","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"HSP-SAM adds learned abstract prompt pairs to SAM, achieving prompt-free medical image segmentation with reported zero-shot improvements of up to 14.04 percent Dice.","lead":"Researchers built a version of the Segment Anything Model that segments medical images without any clicks, boxes, or other manual prompts. It learns abstract task tokens that tell the model what to look for, and it reports large gains on unseen medical datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paper's central claim that learned abstract Q&A prompts drive the reported zero-shot gains is not supported by its own ablation: removing the Q&A module changes average Dice by only ~0.97 (78.78 to 77.81), and no positional-prompt control exists.","rationale":"The reader's weakest assumption and my concern point to the same gap: the paper has not shown that the learned Q&A prompts, rather than the accompanying architectural changes, drive the reported zero-shot improvements. My pass sharpens this with an internal inconsistency: the paper's own ablation assigns most of the generalization gain to hierarchical decoding and skip connections, not to the Q&A module. This is a load-bearing issue for the paper's conceptual claim, but not for the framework as an engineering contribution. The fix is concrete and inexpensive: a within-framework positional-prompt baseline and backbone-matched comparisons. If the control shows no benefit for abstract prompts, the paper should be revised to make hierarchical decoding and skip connections the primary claimed contributions. This warrants a conditional acceptance rather than rejection, because the reported tables are internally consistent and the missing control is straightforward to run.","tokens_in":15210,"tokens_out":8542,"duration_ms":100407,"concrete_test":"Add a positional-prompt control on the S3→T3 task: keep LoRA, hierarchical decoding, skip connections, training data, and schedule identical, but replace the Q&A vector pairs with c learned positional embeddings (e.g., a center point or fixed grid) appended at the same encoder and decoder positions; report T3 Dice and HD. If this control reaches the full model's ~80.00 Dice, or if the abstract-vs-positional gap is smaller than ~1 Dice point, the abstract-prompt interpretation fails. As a secondary check, rerun the headline comparisons with HSP-SAM using a ViT-B backbone so all baselines are backbone-matched.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that replacing positional prompts with learned abstract Q&A prompts is what produces the large zero-shot gains, e.g., 80.00 vs. 65.96 Dice on T3 in Table 4. This attribution is not tested. In the ablation study in §4.4 (labeled Table 5 in the arXiv text), the full model with Q&A pairs, hierarchical decoding, and skip connections reaches 78.78 average Dice on the S→T generalization tasks; removing only the Q&A prompt pairs (Ablation_5) reaches 77.81, and hierarchy-only without Q&A or skip (Ablation_4) reaches 77.87. By the paper's own numbers, the Q&A module contributes less than 1 Dice point to the domain-generalization average, while the hierarchical-decoding and skip-connection modules contribute about 3.4–4 points. Comparisons to prior positional-prompt self-prompting methods (SAM-SP, H-SAM, ESP-MedSAM) simultaneously change backbone, LoRA configuration, decoder depth, and skip connections, so the 14.04% gain over ESP-MedSAM cannot be uniquely attributed to prompt type. Moreover, the no-Q&A ablations are still prompt-free, so the framework does not require abstract prompts to eliminate manual prompting. Section 4.5's prompt-count invariance argument is indirect and does not clarify how the c output tokens are aggregated into a final prediction. The conceptual novelty—abstract prompts, not positional prompts—is plausible but currently unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HSP-SAM, a prompt-free adaptation of the Segment Anything Model for medical image segmentation. The method freezes the SAM image encoder and adds LoRA adapters, learned abstract 'Q&A' prompt pairs that are attached to both the encoder and decoder, and a hierarchical U-shaped decoder with skip connections. The authors report results on polyp segmentation, skin lesion segmentation, and a five-modality domain-generalization protocol with zero-shot evaluation on unseen target datasets. They claim state-of-the-art or highly competitive Dice and Hausdorff distance scores, including gains of up to 14.04% Dice over prior prompt-free SAM methods on a challenging unseen dataset, and they argue that abstract task-guided prompts generalize better than positional prompts. The paper is written as a systems/application contribution with a strong conceptual framing, but the central attribution of the gains to abstract prompts is not directly tested in the reported experiments.","tokens_in":15483,"tokens_out":4739,"duration_ms":54937,"significance":"If the claims hold, HSP-SAM would be a useful prompt-free medical segmentation baseline with practical domain-generalization value, and its zero-shot evaluations across several medical modalities are a genuine strength. The proposed recipe is simple and likely to be adopted by practitioners. However, the paper's more ambitious conceptual claim—that learned abstract prompts, rather than positional prompts, drive the large zero-shot gains—is currently under-supported. The key comparison is confounded by simultaneous changes in backbone, LoRA configuration, decoder depth, and skip connections, and the paper's own ablation attributes less than one Dice point to the Q&A module. This is a load-bearing issue for the stated novelty. The manuscript also defers experimental details to appendices that are not present and does not report uncertainty estimates. With a same-architecture positional-prompt control and proper statistical reporting, the contribution would be substantially stronger.","major_comments":[{"comment":"The central claim that abstract Q&A prompts outperform positional prompts and account for the reported zero-shot gains is not supported by the manuscript's own ablation. In Table 5, removing only the Q&A pairs from the full model (Ablation_3 vs. Ablation_5) changes average Dice from 78.78 to 77.81, a drop of less than one point, while removing both Q&A pairs and skip connections (Ablation_4) leaves 77.87. At the same time, the comparisons to SAM-SP, H-SAM, and ESP-MedSAM in Table 4 change multiple architectural components simultaneously: backbone, LoRA rank, decoder depth, and skip connections. The 14.04-point gap on T3 therefore cannot be uniquely attributed to replacing positional prompts with abstract prompts. Please add a control that keeps the full HSP-SAM architecture but substitutes learned positional prompts for the Q&A pairs, and report per-dataset Dice and HD for that control.","section":"§4.4, Table 5; §4.3, Table 4"},{"comment":"The prompt-count invariance analysis in Fig. 3 is indirect and does not clarify the aggregation mechanism. The paper defines c Q&A pairs and states that each pair corresponds to a segmentation target, but it never states how the c output tokens from the mask decoder are combined into the final single prediction. Without knowing whether the outputs are averaged, concatenated, or decoded through separate heads, the observed invariance to c could be an artifact of the aggregation operation rather than evidence that the prompts encode task-level abstractions. Please specify the aggregation and, if possible, report the actual final-segmentation behavior for each prompt count.","section":"§3.3, Eq. (1)-(2); §4.5, Fig. 3"},{"comment":"All performance comparisons are reported as single point estimates without error bars, confidence intervals, or significance tests. Several headline improvements are small; for example, the Dice gains on ISIC-2017 and ISIC-2018 in Table 2 are 0.44 and 0.93 points, which are within typical run-to-run variation for medical segmentation models. The conclusion that HSP-SAM outperforms previous methods, especially on the classic tasks in Sections 4.1 and 4.2, is therefore not statistically supported. Please report means and standard deviations over at least three independent runs, or bootstrap confidence intervals, for the key comparisons.","section":"Tables 1-4"},{"comment":"The experimental setup is not fully checkable from the submitted manuscript. The text repeatedly defers dataset split definitions, training hyperparameters, and implementation details to Appendix A.1-A.3, but these appendices are not present in the manuscript, and the code is stated to be released only upon acceptance. This is particularly important for the zero-shot generalization protocol in Section 4.3, because the exact construction of the source-target pairs and the 7:3 splits determines whether the evaluation is truly out-of-sample. Please include the appendix content in the version under review or provide a public anonymized code/data supplement.","section":"§4.1-§4.3, Appendices A.1-A.3"}],"minor_comments":[{"comment":"The dataset name 'CVC-ClonDB' appears to be a typo; elsewhere in the manuscript and in the literature it is 'CVC-ColonDB'.","section":"§4.1"},{"comment":"The SAM-SP method is cited as [58] in Section 2.3 but as [59] in the reference list and in Tables 1 and 2; please make the citation numbering consistent.","section":"§2.3 and References"},{"comment":"The text refers to 'Ablatio_4' in the setup paragraph; this should be 'Ablation_4'. Also, the ablation table is labeled Table 5 in the manuscript but is referred to as 'Table 6' in the text; please reconcile the numbering.","section":"§4.4"},{"comment":"The caption for Fig. 3 states that source results are marked with triangles and target results with squares, but the figure as rendered has no legend and no axis labels; please add a legend and label the axes for the figure to be interpretable.","section":"§4.5, Fig. 3"},{"comment":"The phrase 'we are the first to introduce learning abstract prompts during the self-prompting process' is a strong novelty claim that cannot be verified from the cited related work alone; please soften it or provide a more systematic comparison with any existing abstract-prompt self-prompting methods.","section":"Abstract and Conclusion"}],"recommendation":"major_revision","confidential_remarks":"This paper has competitive empirical results and a plausible application-level contribution, but the framing as a conceptual finding about abstract versus positional prompts currently outruns the evidence. The missing same-architecture positional baseline is the key issue: without it, the claimed mechanism is not established. The lack of appendices and code also makes the zero-shot protocol difficult to audit. I would recommend requiring the additional control experiments and full reproducibility materials before considering the paper further; if those cannot be provided, the paper may more appropriately be framed as a prompt-free segmentation system without the stronger conceptual claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful prompt-free SAM adapter, and the Q&A prompt-pair mechanism is a genuine twist on the self-prompting line. But the paper's central conceptual claim—that learned abstract prompts, not positional ones, drive the zero-shot gains—is not supported by its own ablation, and the authors will need to do real work to make it stick.\n\nWhat's actually new: prior self-prompting SAM variants (SAM-SP, H-SAM, ESP-MedSAM) all generate positional prompts (boxes, noise-based masks). HSP-SAM replaces those with learned Q&A prompt pairs, split between the encoder and decoder, connected through a bottleneck MLP. That's a clean, transportable idea, and the architecture is specified in enough detail that a competent lab could rebuild it. The evaluation is also genuinely broad: five source/target modality pairs, classic polyp and skin-lesion benchmarks, and zero-shot transfers on unseen datasets. The zero-shot numbers, if reproducible, are competitive with or better than prior prompt-free methods. All of that is worth credit.\n\nThe soft spot is the attribution. Table 6 (labeled Table 5 in the text) shows the full model at 78.78 average Dice on the generalization tasks; drop Q&A pairs and you get 77.81. That's under one point. Hierarchy plus skip connections contribute roughly 3.4–4 points. The comparisons against positional-prompt self-prompting methods change backbone, LoRA, decoder depth, and skip connections simultaneously, so the 14-point gain over ESP-MedSAM cannot be pinned on prompt type. There is also no same-architecture positional-prompt baseline, which is the one experiment that would actually test the paper's thesis. The Section 4.5 prompt-count invariance argument is indirect: it shows the number of prompts doesn't matter much, which is consistent with abstract prompts, but it doesn't rule out that the prompts are just acting as a fixed bias. And the paper never explains how the c output tokens are aggregated into the final prediction.\n\nMinor stuff, in proportion: no error bars or significance tests on the headline numbers; code and appendices are promised but absent; the 'first to abandon positional prompts' framing is stronger than the literature justifies, since learnable tokens in frozen ViTs are common in prompt learning. None of that is fatal. The central empirical result—that this adapter generalizes well zero-shot—is plausible and the tables are internally consistent. The conceptual story just needs to be matched to the evidence.\n\nWho this is for: anyone adapting SAM to medical imaging, especially clinical batch workflows. It deserves a serious referee; I'd send it to review, but the authors should be asked to add a same-framework positional-prompt baseline, report variance, release code, and scale back the claim that abstract prompts themselves are what produce the generalization.","headline":"A genuinely useful prompt-free SAM adapter, but the paper's central claim that abstract Q&A prompts drive its zero-shot gains is contradicted by its own ablation, which shows they contribute less than one Dice point.","tokens_in":16048,"tokens_out":2584,"would_cite":true,"duration_ms":27657,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HSP-SAM replaces SAM's manual prompts with learned abstract Q&A prompts and reports up to 14.04% higher Dice on unseen medical datasets.","keywords":["medical image segmentation","Segment Anything Model","prompt-free segmentation","self-prompting","abstract prompts","LoRA fine-tuning","domain generalization","polyp segmentation"],"falsifier":"Find a source-trained HSP-SAM and a target dataset from the same modality but with a substantially different object distribution, then train a positional-prompt variant (for example, a self-generated box or point version) under identical conditions; if the positional variant matches or exceeds HSP-SAM's zero-shot Dice, the abstract-prompt claim is not supported. Equivalently, on a dataset where the object count per image varies widely, if Dice improves monotonically as the number of Q&A prompts grows beyond the maximum object count, the prompts are behaving as object-specific slots rather than task-level abstractions.","tokens_in":14984,"feed_emoji":"🩻","tokens_out":4239,"duration_ms":39041,"temperature":0.7,"pith_summary":"The paper proposes HSP-SAM, a framework that adapts the Segment Anything Model (SAM) to medical image segmentation without any manual prompts. Its central claim is that SAM's prompt dependence can be removed not by generating the usual positional prompts (points, boxes) but by learning abstract 'Q&A' prompt pairs that encode the segmentation task itself. The authors report that this self-prompting approach outperforms prior prompt-free SAM methods on polyp and skin lesion segmentation, and achieves zero-shot Dice gains of up to 14.04% over the previous state of the art on unseen datasets across five imaging modalities. If the claim holds, a single SAM-based model can segment diverse medical images in a fully automated, prompt-free pipeline.","feed_headline":"Prompt-free SAM beats prior best by 14% on unseen medical images","feed_subtitle":"HSP-SAM learns abstract Q&A prompt pairs that transfer across five imaging modalities without manual prompts.","key_machinery":"The central mechanism is the Q&A prompt pair: a set of learnable vectors $Q$ that are concatenated into the image encoder's input and transformed by a dimensionality-reducing map $f$ followed by task-specific MLPs into A-prompts $A$ for the mask decoder, with one pair per segmentation target count. A hierarchical feature fusion module assigns separate Q&A pairs, bottlenecks, and mask decoder blocks to each global attention layer of SAM's encoder, fusing high- and low-dimensional features through a U-shaped structure with skip connections. LoRA keeps the encoder parameter-efficient. The Q&A transform is what carries the paper's claim: the prompts are optimized to convey task-relevant cues, not to locate individual objects.","core_discovery":"On its own terms, the paper discovers that SAM does not truly require positional prompts: it requires task guidance, and positional prompts are only one possible form. HSP-SAM instantiates this by pairing learnable Q-prompts, fed into the frozen image encoder, with A-prompts, produced from the Q-prompts through a bottleneck mapping and task-specific MLPs, and fed into the mask decoder. These Q&A prompt pairs are trained during fine-tuning while the image encoder stays frozen except for LoRA adapters. The authors argue that because the prompts are abstract and task-level rather than tied to specific target locations, they generalize across datasets and modalities where positional prompting fails, such as vessel segmentation with thin structures or nuclei segmentation with dozens of objects per image.","pith_inferences":["If the abstract-prompt interpretation is correct, the same mechanism could generalize beyond medical imaging to any SAM domain where prompts are unavailable, such as satellite or video segmentation, by retraining only the prompt pairs.","The paper compares against H-SAM and ESP-MedSAM, but not against a version of HSP-SAM that uses positional prompts; such an ablation would test whether the gains come from abstraction or simply from the hierarchical decoding and skip connections.","The reported prompt-count invariance on DSB-2018 is the key evidence; reproducing it across more datasets with variable object counts would strengthen the claim that the prompts are not object-specific."],"forward_implications":["Prompt-free SAM adaptation becomes a viable default for medical imaging; clinicians could segment lesions without supplying points or boxes.","The same frozen SAM image encoder, equipped with learned abstract prompts, can be transferred zero-shot across five imaging modalities, suggesting a path to a universal medical segmentation model.","The Q&A prompt design implies that the number of prompts need not match the number of objects: performance stayed stable when prompt count varied from 1 to 16 on nuclei images with dozens of targets.","Because LoRA keeps the image encoder frozen and only trains small prompt and decoder modules, the framework adds only about 24 million parameters, making deployment cheap."],"supporting_citations":[{"why":"The prior self-prompting approach that HSP-SAM extends; its positional-prompt generation is the method the paper argues against.","marker":"[59]"},{"why":"A competing prompt-free SAM adaptation with hierarchical decoding, used as a baseline in experiments.","marker":"[8]"},{"why":"A competing prompt-free method using self-prompting and knowledge distillation, the main state-of-the-art baseline on generalization tasks.","marker":"[50]"},{"why":"The Segment Anything Model whose prompt dependence HSP-SAM removes; the image encoder and mask decoder are reused.","marker":"[25]"},{"why":"Provides the parameter-efficient low-rank adaptation method for fine-tuning the frozen image encoder.","marker":"[16]"},{"why":"Inspiration for linking prompts between branches; here adapted to pair Q-prompts in the encoder with A-prompts in the decoder.","marker":"[23]"},{"why":"Represents the manual-prompt fine-tuning paradigm that HSP-SAM contrasts with.","marker":"[27]"}],"fun_headline_variants":["Self-prompting SAM boosts unseen medical segmentation by 14%","Prompt-free SAM: abstract prompts beat manual clicks in medical imaging","HSP-SAM learns abstract prompts, outdoes SOTA on unseen medical data","No prompts needed: SAM variant leaps 14% on unseen medical benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that the learned Q&A prompt pairs capture a task-level abstraction that transfers across datasets, rather than memorizing source-domain statistics that happen to correlate with the target datasets.","fun_headline_variants_meta":{"raw":{"variants":["Self-prompting SAM boosts unseen medical segmentation by 14%","Prompt-free SAM: abstract prompts beat manual clicks in medical imaging","HSP-SAM learns abstract prompts, outdoes SOTA on unseen medical data","No prompts needed: SAM variant leaps 14% on unseen medical benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000861,"raw_usage":{"total_tokens":3710,"prompt_tokens":893,"completion_tokens":2817,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":2739}},"tokens_in":509,"tokens_out":2817,"duration_ms":19636,"temperature":1.0,"reasoning_tokens":2739,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:14:05.947994+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find a source-trained HSP-SAM and a target dataset from the same modality but with a substantially different object distribution, then train a positional-prompt variant (for example, a self-generated box or point version) under identical conditions; if the positional variant matches or exceeds HSP-SAM's zero-shot Dice, the abstract-prompt claim is not supported. Equivalently, on a dataset where the object count per image varies widely, if Dice improves monotonically as the number of Q&A prompts grows beyond the maximum object count, the prompts are behaving as object-specific slots rather than task-level abstractions.","supporting_citations":[{"cited_title":"Unleashing the potential of sam for medical adaptation via hierarchical decoding","cited_arxiv_id":null,"evidence_quote":"A competing prompt-free SAM adaptation with hierarchical decoding, used as a baseline in experiments."}],"review_version":1}