{"id":"96fd64a1-9cc6-41f8-822c-0f53d7c1bdc4","arxiv_id":"2501.01072","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"EUGIS uses evidential uncertainty estimates to guide point-prompt sampling in interactive segmentation, reporting state-of-the-art performance on three ultrasound datasets with a single click.","lead":"This paper proposes EUGIS, an interactive ultrasound image segmentation method that uses an uncertainty map to decide where a user should click next, reducing the number of prompts needed. The authors report large gains over published methods on three ultrasound datasets, which could make interactive segmentation more practical in clinical settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central single-prompt superiority claim is not supported without a direct uncertainty-guided vs. random prompt-sampling ablation.","rationale":"The reader's verdict identifies several experimental gaps and highlights the from-scratch baseline concern. I agree that this is a serious external-validity problem, but I think the most load-bearing internal concern is narrower and even more decisive: the paper's core novelty, the uncertainty-guided prompt sampling, is never isolated in any experiment. The stated contribution is that prioritizing high-uncertainty regions produces better interactive segmentation with fewer prompts (Sec. I, Sec. III-C.2), but the only ablations (Tables IV and V) remove the hybrid encoder, CEU, and multi-head selection; no ablation switches the prompt-sampling strategy. The benchmark comparisons are also compromised by the unexplained from-scratch training of SAM-family baselines, so the headline numbers cannot establish superiority over those methods. These are not internal inconsistencies in the math, and the evidential uncertainty derivation in Sec. III-B appears coherent, but the experiments as reported do not support the strongest-claim as stated. A conditional accept is appropriate: a direct prompt-sampling ablation and a corrected/fair baseline comparison would settle the matter.","tokens_in":18946,"tokens_out":1414,"duration_ms":13107,"concrete_test":"Run EUGIS on BUSI with the prompt generator in Sec. III-C.2 replaced by (a) uniform random sampling over the predicted foreground/background error mask and (b) random sampling over the full image, holding all model weights, two-stage training, encoder, and loss identical; report Dice/Jaccard/95HD at 1, 3, and 5 prompts. If the uncertainty-guided variants do not beat the random error-region sampling by a statistically significant margin under the same training budget, the paper's central single-prompt superiority claim is not attributable to its proposed mechanism.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's headline claim is that EUGIS, using uncertainty-guided prompts, outperforms state-of-the-art interactive segmentation baselines with one prompt. The mechanism that supposedly delivers this advantage is the evidential uncertainty-guided prompt sampling (Sec. III-C.2), yet no experiment isolates this mechanism. The ablations in Sec. V-C remove CEU, Multi-Seg, and the hybrid encoder, but never replace uncertainty-guided prompt sampling with random sampling or with the standard error-region sampling used by end-to-end counterparts. Without that ablation, the observed gains (e.g., 88.95% vs. 83.73% Dice at one point on BUSI, Table I) could plausibly come from the stronger hybrid encoder, the multi-head selection, or the extra two-stage training regime rather than from the uncertainty guidance. In addition, Sec. V-A states the interactive baselines were all trained from scratch without pre-trained weights, which likely handicaps the SAM-family baselines and is never justified; the reported single-prompt numbers are therefore not evidence of the claimed superiority over the baselines as intended to be used. The CEU ablation (Table V) does show a 1.46% Dice change on DDTI, but this validates calibration, not the prompt-sampling mechanism. The central contribution—simulating expert radiologist behavior via uncertainty maps—is thus asserted but never directly tested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EUGIS, a two-stage interactive segmentation framework for ultrasound images. In stage I, a hybrid encoder-decoder is trained with evidential deep learning based on Dempster-Shafer theory and Subjective Logic, together with a new calibration loss (CEU), to produce per-pixel uncertainty maps. In stage II, the model is extended with a prompt encoder; point prompts are simulated by selecting the highest-uncertainty pixels from the (initially stage-I, later updated) uncertainty map, and the model refines the segmentation over multiple iterations while selecting among K output masks. The method is evaluated on BUSI, DDTI, and EchoNet with five-fold cross-validation, comparing with U-Net, U-Net++, SegNet, TransUNet, H2Former, SAM, SAMed, MedSAM, SAM-Med2D, and Medical SAM Adapter, and reporting Dice, Jaccard, and 95HD. The central claim is that EUGIS reaches state-of-the-art performance with a single point prompt, for example 88.95% Dice on BUSI.","tokens_in":19141,"tokens_out":9123,"duration_ms":86273,"significance":"If the empirical claims were properly supported, EUGIS would be a useful contribution to interactive ultrasound segmentation: the idea of replacing random or error-region prompt simulation with evidence-based uncertainty-guided sampling is natural and could reduce the number of clicks needed. The paper also contains a standard derivation of Dirichlet-based evidential losses, an explicit calibration loss, and evaluations on three public datasets. However, the key mechanism is not isolated by an ablation, the comparison with SAM-family baselines appears to handicap those baselines, and at least one central number is reported inconsistently. These issues must be resolved before the claims can be accepted.","major_comments":[{"comment":"The paper's principal stated contribution is the use of evidential uncertainty maps to generate point prompts that simulate expert radiologist behavior. Yet the ablation study never replaces uncertainty-guided prompt sampling with random sampling or with the error-region sampling used by existing end-to-end methods. The reported ablations vary the encoder (Table IV), CEU, and multi-head selection (Table V), but the prompt-sampling mechanism is held constant throughout. Consequently, the single-prompt improvements in Tables I–III cannot be attributed to uncertainty-guided sampling; they could come from the hybrid encoder, the two-stage training schedule, CEU, or the multi-head selection. A controlled experiment that keeps all other components fixed and swaps only the prompt-sampling strategy—uncertainty-guided versus random versus error-region—is necessary to validate the core claim.","section":"§III-C.2, §V-C, Tables IV and V"},{"comment":"The text states that the interactive SAM-family baselines (SAMed, SAM-Med2D, and Medical SAM Adapter) are all trained from scratch rather than using the pre-trained weights provided. These methods are explicitly designed to leverage pre-trained SAM weights, so training them from scratch on relatively small ultrasound datasets removes their main advantage and is likely to understate their performance. The headline result that EUGIS with one click beats SAM-Med2D with five clicks is therefore not a fair comparison against these baselines as they are intended to be used. The authors should either use the official pre-trained weights and standard fine-tuning protocols, or provide a clear justification for the from-scratch training and show that the conclusions are unchanged when pre-trained initialization is used.","section":"§V-A, Tables I–III"},{"comment":"The BUSI single-prompt result is reported inconsistently: the text states that EUGIS with a single point prompt attains Dice and Jaccard scores of 87.69% and 79.31%, while Table I lists 88.95% and 81.08% for the same configuration. This is a direct numerical inconsistency in the central experimental result and must be corrected. The authors should also verify that all other numbers quoted in the text match the corresponding tables.","section":"§V-A.1 versus Table I"},{"comment":"The prompt-simulation and iterative-training protocol is not specified precisely enough to reproduce or to interpret. Eq. (14) defines Lcon as a mean-square error but calls its minimum a 'highest confidence score', and Eq. (15) is hard to parse. In addition, it is unclear how the 'k pixels exhibiting the greatest uncertainty' are converted into point clicks—for example, whether a clustering or connected-component step is used to avoid selecting adjacent pixels—how the M iterations relate to the 1/3/5 click counts in the tables, and whether prompts are re-sampled at every iteration. Please provide an algorithmic description of the interaction schedule.","section":"§III-C, Eqs. (14)–(16)"}],"minor_comments":[{"comment":"The notation X ∈ R^{N×H×W×C} is unusual; if N denotes the batch size it should be stated explicitly, and otherwise the input shape should be corrected to a standard image tensor shape.","section":"§III-B.1"},{"comment":"The text says 'pj,k is a probability vector of length k'; from the context it should be a vector of length N (the number of classes), and the simplex SN is also defined with N classes.","section":"§III-B.1, Eq. (4)"},{"comment":"The annealing coefficient α_t = α0 e^{-t/T} in Eq. (5) contains an unspecified hyperparameter α0; its value should be reported in the implementation details.","section":"§III-B.2, §IV-B"},{"comment":"For BUSI, please clarify how the 780 images are used, since the normal class has no segmentation mask; in particular, specify whether normal images are discarded or included and how the five-fold cross-validation splits are formed.","section":"§IV-A"},{"comment":"The caption of Figure 2 is grammatically incomplete ('it retains uncertainty map from the first-stage model are used as the point prompt generator') and should be rewritten for clarity.","section":"Figure 2 caption"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the central mechanism is not directly tested and the baseline comparison appears to disadvantage the SAM-family methods. The numerical inconsistency in the BUSI result also needs to be resolved. If the authors cannot supply the missing prompt-sampling ablation and fair pretrained baselines, the paper should not be published in its current form; the scope of the needed changes is substantial enough to require another review round."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper combines evidential uncertainty estimation, a calibration loss, and uncertainty-guided prompt simulation for interactive ultrasound segmentation. That exact combination is new, and the CEU calibration loss with the annealing schedule is a genuinely reasonable idea. The hybrid encoder and multi-head selection are sensible engineering. The math follows Sensoy et al. and related work cleanly; I did not see a load-bearing derivation error. The visualizations show the uncertainty maps lining up with actual segmentation errors, which is encouraging evidence for the mechanism.\n\nWhat the paper does not show is its central claim: that uncertainty-guided prompt sampling is what makes EUGIS superior with one click. There is no ablation comparing uncertainty-guided prompt simulation against random sampling, or against the standard error-region sampling used by prior end-to-end methods. The ablation in Table V only removes CEU and Multi-Seg; it never isolates the prompt-sampling mechanism. So the large reported gap over SAM-Med2D (88.95 vs. 83.73 Dice on BUSI) could plausibly come from the stronger hybrid encoder, multi-head selection, or the two-stage training, not from the uncertainty guidance. I agree with the stress-test note here.\n\nThe baseline comparison is also shaky. Section V-A states the interactive baselines were trained from scratch rather than using their pre-trained weights. That likely handicaps SAM-family models, which depend on pre-training. The authors give no justification and no training details for those baselines. Second, there is an internal inconsistency: the BUSI text reports 87.69 Dice for EUGIS at one point, but Table I reports 88.95. One of those is wrong. Third, code is only promised, not released, so the results cannot be independently reproduced yet.\n\nThese are real problems, but they are fixable. The method is coherent, the components are standard, and the idea of using calibrated evidential uncertainty to place prompts is worth testing. If the authors add the missing prompt-sampling ablation, correct the baseline protocol or justify it, fix the BUSI inconsistency, and release code and exact training settings, the paper could support its claims. As it stands, the empirical case for the headline is not established.\n\nThis should go to peer review with a request for major revision. A serious referee can push the authors on the missing ablation and baseline fairness. I would not cite the quantitative results in their current form, but I would follow the paper's progress and revisit after revision.","headline":"Plausible ultrasound interactive segmentation system with a useful calibration idea, but the paper's headline single-prompt superiority claim is not actually demonstrated because the core prompt-sampling mechanism is never ablated and the baselines were trained from scratch.","tokens_in":19696,"tokens_out":1276,"would_cite":false,"duration_ms":15158,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single point prompt, placed at the highest-uncertainty pixel, outperforms existing interactive and non-interactive segmentation methods across three ultrasound datasets.","keywords":["Ultrasound image segmentation","Interactive segmentation","Evidential uncertainty","Dempster-Shafer theory","Subjective Logic","Uncertainty calibration","Point prompting","Medical image segmentation"],"falsifier":"Retrain SAM-Med2D, Medical SAM Adapter, and SAMed with their released pre-trained weights and run the same 1-, 3-, and 5-click protocol on BUSI, DDTI, and EchoNet; if any baseline matches or beats EUGIS at equal or fewer clicks, the single-prompt superiority claim fails. As a second check, replace EUGIS's uncertainty-based prompt selection with random point sampling inside the same architecture; if the Dice gap disappears, then uncertainty guidance, not the architecture or training, is doing the work.","tokens_in":18704,"feed_emoji":"🩺","tokens_out":7032,"duration_ms":58849,"temperature":0.7,"pith_summary":"This paper claims that interactive ultrasound segmentation can be made dramatically more efficient by letting the model decide where the user should click. Its proposed EUGIS paradigm first trains an evidential model, based on Dempster-Shafer theory and Subjective Logic, to produce a per-pixel uncertainty map, then places point prompts at the most uncertain locations instead of at random or hand-chosen error regions. A trainable calibration loss sharpens the boundary between confident and uncertain predictions, and multiple segmentation heads pick the most confident mask. Across breast, thyroid, and left-ventricle ultrasound datasets, EUGIS with a single point prompt is reported to beat state-of-the-art interactive baselines that receive five clicks, and also beat non-interactive methods. The practical stake is a reduction in clinician effort and iteration count if the interaction can be guided reliably by uncertainty.","feed_headline":"One uncertainty-guided click beats five-click baselines","feed_subtitle":"EUGIS turns one high-uncertainty pixel into a prompt and hits 88.95% Dice on BUSI, beating baselines given five clicks.","key_machinery":"The central mechanism is the evidential uncertainty map built from Dempster-Shafer theory and Subjective Logic. Each pixel's network output is treated as non-negative evidence $e^n_{j,k}$ that defines a Dirichlet concentration $\\alpha^n_{j,k}=e^n_{j,k}+1$, from which the belief mass $b^n_{j,k}=e^n_{j,k}/S$ and uncertainty $u_{j,k}=N/S$ follow, where $S=\\sum_n(e^n_{j,k}+1)$. This uncertainty map drives prompt simulation: at each iteration the model picks the $k$ pixels with the highest uncertainty as point prompts. The calibration loss $L_{\\text{CEU}}$ uses an annealing factor $\\alpha_t$ to enforce that accurate predictions are confident and inaccurate ones are uncertain. Stage II initializes the interactive model from Stage I, re-generates the uncertainty map each epoch, and selects among $K$ segmentation masks the one with the highest confidence score, making the prompt guidance and the final mask selection work from the same evidential signal.","core_discovery":"On the paper's own terms, the discovery is that evidential uncertainty is a better prompt generator for interactive segmentation than the random or error-region sampling used by prior end-to-end methods. EUGIS parameterizes the model's output as a Dirichlet distribution via Dempster-Shafer theory and Subjective Logic, so each pixel carries a belief mass and an uncertainty mass, and the highest-uncertainty pixels become the next point prompts. The authors introduce a calibrated evidential uncertainty loss (CEU) that penalizes confident wrong predictions early and uncertain right predictions later, producing cleaner uncertainty maps. With this guidance, EUGIS reaches 88.95% Dice on BUSI, 91.63% on DDTI, and 94.85% on EchoNet with one click, exceeding the best interactive baseline at five clicks on each dataset (e.g., 83.73% Dice for SAM-Med2D on BUSI). The paper also reports that removing CEU lowers Dice by 1.46% on DDTI, and that a hybrid CNN-ViT encoder contributes more than the multiple segmentation heads.","pith_inferences":["If the single-click result transfers to a real radiology workflow, the practical bottleneck shifts from placing prompts to deciding when the model's uncertainty is low enough to stop iterating; an automatic stopping rule based on the uncertainty map would be a natural extension.","The same uncertainty-guided sampling could be applied to other interactive modalities such as CT or MRI, but the paper does not test that, and boundary ambiguity there differs from ultrasound speckle.","A direct test of whether EUGIS truly emulates radiologists would be to compare its chosen click locations with clicks from expert sonographers on the same images; the paper only simulates prompts from uncertainty maps.","Because the reported gains depend on training SAM-based baselines from scratch, the headline margin over those baselines may shrink under standard transfer-learning practice, even though the uncertainty-guided mechanism itself could still stand."],"forward_implications":["A clinician could plausibly segment an ultrasound lesion with a single click, because one well-placed prompt already outperforms five prompts in the reported comparisons.","The uncertainty map doubles as a quality indicator: regions where segmentation fails are the regions the model flags as uncertain, which could support automated flagging of unreliable boundaries.","Because the same paradigm works on breast, thyroid, and cardiac ultrasound, the method is not tied to one anatomy or one imaging setup.","The ablation results imply that calibration of uncertainty, not just the uncertainty estimate itself, is a major source of the gain; future interactive systems should treat uncertainty calibration as a first-class component.","On larger datasets, non-interactive methods close much of the gap, but EUGIS still leads with one click, suggesting the interaction benefit persists as data scale grows."],"supporting_citations":[{"why":"Supplies the Dirichlet-based evidential deep learning machinery that EUGIS uses to convert logits into evidence and uncertainty.","marker":"[22]"},{"why":"Provides the Dirichlet-based uncertainty calibration idea that the paper extends into its trainable CEU loss.","marker":"[23]"},{"why":"Gives Dempster-Shafer theory, the formalism for belief masses and the 'I don't know' state underlying the uncertainty score.","marker":"[46]"},{"why":"Gives Subjective Logic, which connects belief assignments to a Dirichlet distribution over class probabilities.","marker":"[47]"},{"why":"SAM-Med2D is the strongest interactive baseline at five clicks that EUGIS claims to beat with a single point prompt.","marker":"[16]"},{"why":"Medical SAM Adapter is the other leading interactive baseline across the three datasets and defines the comparison regime.","marker":"[20]"},{"why":"BUSI breast ultrasound dataset is the primary evaluation set where EUGIS reports 88.95% Dice with one click.","marker":"[48]"},{"why":"DDTI thyroid ultrasound dataset is the second evaluation set, used for the ablation study of the CEU loss.","marker":"[49]"},{"why":"EchoNet cardiac ultrasound dataset is the large-scale evaluation set testing whether the single-prompt advantage persists with over 20,000 images.","marker":"[50]"}],"fun_headline_variants":["Uncertainty-guided click cuts prompt count by 80%","Evidential uncertainty picks best prompt for ultrasound segmentation","One high-uncertainty pixel outperforms five random prompts","Calibrated evidential uncertainty boosts one-click segmentation","Single evidential prompt beats five-click interactive baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that EUGIS outperforms state-of-the-art baselines with a single prompt rests on the assumption that training the SAM-based baselines from scratch, without their pre-trained weights, is a fair comparison, since the reported gaps might be much smaller if those baselines were used as intended.","fun_headline_variants_meta":{"raw":{"variants":["Uncertainty-guided click cuts prompt count by 80%","Evidential uncertainty picks best prompt for ultrasound segmentation","One high-uncertainty pixel outperforms five random prompts","Calibrated evidential uncertainty boosts one-click segmentation","Single evidential prompt beats five-click interactive baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000346,"raw_usage":{"total_tokens":1926,"prompt_tokens":1004,"completion_tokens":922,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":843}},"tokens_in":620,"tokens_out":922,"duration_ms":8566,"temperature":1.0,"reasoning_tokens":843,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:35:34.566400+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain SAM-Med2D, Medical SAM Adapter, and SAMed with their released pre-trained weights and run the same 1-, 3-, and 5-click protocol on BUSI, DDTI, and EchoNet; if any baseline matches or beats EUGIS at equal or fewer clicks, the single-prompt superiority claim fails. As a second check, replace EUGIS's uncertainty-based prompt selection with random point sampling inside the same architecture; if the Dice gap disappears, then uncertainty guidance, not the architecture or training, is doing the work.","supporting_citations":[{"cited_title":"Evidential deep learning to quantify classification uncertainty,","cited_arxiv_id":null,"evidence_quote":"Provides the Dirichlet-based uncertainty calibration idea that the paper extends into its trainable CEU loss."},{"cited_title":"Tbrats: Trusted brain tumor segmentation,","cited_arxiv_id":null,"evidence_quote":"Gives Dempster-Shafer theory, the formalism for belief masses and the 'I don't know' state underlying the uncertainty score."},{"cited_title":"A generalization of bayesian inference,","cited_arxiv_id":null,"evidence_quote":"Gives Subjective Logic, which connects belief assignments to a Dirichlet distribution over class probabilities."},{"cited_title":"Jsang, Subjective Logic: A formalism for reasoning under uncertainty","cited_arxiv_id":null,"evidence_quote":"BUSI breast ultrasound dataset is the primary evaluation set where EUGIS reports 88.95% Dice with one click."},{"cited_title":"An open access thyroid ultrasound image database,","cited_arxiv_id":null,"evidence_quote":"EchoNet cardiac ultrasound dataset is the large-scale evaluation set testing whether the single-prompt advantage persists with over 20,000 images."}],"review_version":1}