{"id":"26fb9210-6e42-4f2f-9a98-405ab53c2480","arxiv_id":"2509.05809","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A SAM variant that samples a latent code into the prompt embedding produces diverse lung nodule masks and reports better GED, DSC, and IoU than Probabilistic U-Net on LIDC-IDRI.","lead":"This paper attaches a variational latent space to the Segment Anything Model so that one prompt can yield several plausible lung nodule segmentations. On LIDC-IDRI the authors report lower generalized energy distance and higher Dice and IoU than Probabilistic U-Net, though the baselines are not matched in backbone.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gains attributed to the CVAE latent may be driven by SAM's pretrained backbone: no deterministic SAM control is reported, and GED does not isolate diversity from mean accuracy.","rationale":"The reader's weakest assumption identifies exactly the load-bearing concern: the empirical comparison does not identify the contribution of the probabilistic module because the backbone is not controlled. The paper's strongest claim is not that SAM is a good segmenter but that the CVAE latent enables probabilistic, ambiguity-aware segmentation that outperforms probabilistic baselines. For that claim, the mechanism must be the cause of the measured improvement. Section 3.3 compares against Probabilistic U-Net, which uses a U-Net backbone, and Dropout SAM, which uses SAM with dropout in the decoder but no learned latent. Neither is a deterministic SAM control, so the GED/DSC/IoU gains could be entirely due to SAM's pretrained encoder and decoder. The GED metric makes this concern acute: generalized energy distance can be low for a peaked predictive distribution centered near the mean of the expert masks, even if the model never produces masks reflecting the spread of expert opinions. Thus the central claim is not falsified, but the reported evidence does not establish it. The proposed deterministic-SAM arm would directly settle the attribution question, and a diversity-only metric (mean pairwise Dice) would test whether the model truly captures variability rather than merely emitting a sharp consensus. I agree with the reader's conditional verdict and the recommended additions; my read does not move the verdict.","tokens_in":4550,"tokens_out":3765,"duration_ms":47948,"concrete_test":"Re-run the LIDC-IDRI experiment with a deterministic SAM control: identical image encoder, prompt encoder, and mask decoder; identical training protocol with the same BCE+Dice loss but without the KL term and without latent injection; and identical GED/DSC/IoU evaluation including the same number of inference samples (e.g., 32). If deterministic SAM reaches GED within seed-level noise of 0.2910, the gains are backbone-driven; if its GED is materially worse, the latent mechanism is doing work. Additionally report mean pairwise Dice across sampled masks for both models, since a diversity-only metric would separate a sharp consensus predictor from one that actually reflects inter-expert variability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that adding a CVAE latent space to SAM produces output distributions closer to expert annotation variability (GED 0.2910 vs. 0.3349 for Probabilistic U-Net). The load-bearing untested premise is that this improvement is caused by the latent mechanism (prior/posterior and prompt-embedding modulation) rather than by SAM's much stronger pretrained image encoder and mask decoder. Section 3.3/Table 1 compares Probabilistic SAM only against Probabilistic U-Net, Dropout U-Net, and Dropout SAM; no deterministic SAM trained/fine-tuned on the same split and prompt protocol is reported. A deterministic SAM could achieve a low GED by emitting an accurate consensus mask, because GED rewards closeness to the target distribution in expectation and does not by itself separate diversity from mean accuracy. Without a matched-backbone control, the observed gains can be fully explained by SAM's representations, and the paper's specific conclusion that the probabilistic machinery captures annotator variability is unsupported. This is not an internal inconsistency but an identification problem in the experimental design.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Probabilistic SAM, a CVAE-style extension of the Segment Anything Model (SAM) for ambiguity-aware medical image segmentation. A latent variable z is sampled from a posterior network during training and from an image-conditioned prior during inference; the sampled latent is projected and added to the sparse prompt embedding before SAM's mask decoder generates a mask. The model is trained with a reconstruction loss (BCE + Dice) plus a β-weighted KL term, with β=10. The method is evaluated on LIDC-IDRI lung nodule slices against Dropout U-Net, Probabilistic U-Net, and a Dropout SAM baseline, reporting generalized energy distance (GED), Dice, and IoU. The headline result is GED 0.2910 versus 0.3349 for Probabilistic U-Net, with notably higher DSC/IoU. The central claim is that the latent mechanism captures inter-expert variability and that the model outperforms existing probabilistic baselines on uncertainty-aware metrics.","tokens_in":4777,"tokens_out":3822,"duration_ms":44136,"significance":"If the reported gains are attributable to the probabilistic latent mechanism, the paper offers a simple and practical recipe for adding ambiguity modeling to a promptable foundation model, with code released and evaluation on a held-out test split against four external expert masks. The variational formulation is standard, and the architectural intervention—modulating prompt embeddings with a latent sample—is clean and easy to replicate. However, the current experimental design does not isolate the mechanism responsible for the gains: no deterministic SAM arm is reported, and GED conflates mean accuracy with sample diversity. The significance of the paper depends on closing this identification gap, because the conclusion that the probabilistic machinery 'captures annotator variability' is not yet supported by the evidence.","major_comments":[{"comment":"The experimental design does not include a deterministic SAM baseline trained or fine-tuned on the same split and prompt protocol. The abstract and conclusions attribute the GED/DSC/IoU improvements to the CVAE latent space, but the only SAM-based comparators are Dropout SAM and Probabilistic SAM. A deterministic SAM that emits a single accurate consensus mask could achieve a low GED because GED rewards closeness to the target distribution in expectation. Without a matched-backbone control, the observed gains are equally compatible with an explanation based on SAM's strong pretrained image encoder and mask decoder rather than on the prior/posterior networks or prompt-embedding modulation. Please add a deterministic SAM arm, or an ablation in which the latent is removed (e.g., fixed to its prior mean) while keeping the same backbone and training setup.","section":"Section 3.3, Table 1"},{"comment":"The diversity claim is not supported by GED alone. GED (generalized energy distance) includes a term that rewards closeness between the model's samples and the target annotation distribution, so a model with high mean accuracy but zero diversity can still obtain a competitive GED. Probabilistic SAM's DSC/IoU are much higher than Probabilistic U-Net's (0.8255/0.7849 vs 0.5818/0.5557), so the GED advantage may reflect accuracy differences rather than the ability to reproduce annotator variability. Please report a diversity-only metric—for example, the expected pairwise Dice/IoU among sampled masks, or the E||X−X'|| term of GED separately—and, if possible, disaggregate results per expert target or within accuracy-matched bins.","section":"Section 3.3, GED interpretation"},{"comment":"The evaluation protocol for the headline metrics is underspecified. The paper does not define the GED estimator, state the number of Monte Carlo samples used to approximate it, describe the distance function used in GED (e.g., squared Euclidean distance on flattened masks), or explain how DSC/IoU are aggregated across the 1,996 test slices and across sampled masks and expert annotations. The paired t-test details (unit of pairing, correction for multiple comparisons) are also absent. Because the claimed GED improvement is only 4.39%, this specification is necessary for reproducibility and for interpreting whether the improvement is meaningful.","section":"Section 3.2 and Section 3.3"}],"minor_comments":[{"comment":"The BCE loss in Eq. (2) has an unbalanced parenthesis: the bracket is not closed. Also, the text writes N(µ, σ) while the implementations typically use variance; please clarify whether σ denotes standard deviation or variance.","section":"Section 2, Eq. (2)"},{"comment":"Implementation details are incomplete: latent dimension, MLP size, optimizer, learning rate, batch size, number of training epochs, and whether the SAM image/prompt encoders are frozen or fine-tuned are not reported. These details matter for reproducibility and for judging whether the comparison is fair.","section":"Section 3.2"},{"comment":"The one-tailed paired t-test is mentioned but no test statistic, degrees of freedom, or correction for multiple comparisons are provided. It is also unclear whether the pairing is per slice or per patient; please specify.","section":"Section 3.3"},{"comment":"The choice β=10 is stated without justification or sensitivity analysis. Since the KL weight controls the trade-off between reconstruction fidelity and diversity, a small sensitivity study would strengthen the paper.","section":"Section 2, Eq. (4)"}],"recommendation":"major_revision","confidential_remarks":"The paper is short and the central idea is plausible, but the current evaluation does not establish the stated contribution. The missing deterministic SAM control is a standard but essential experiment, and the GED/diversity issue is fixable with additional analysis. I would not reject at this stage; the scope of a major revision is well defined: add a matched-backbone baseline, report a diversity-only measure, and specify the evaluation protocol. I would also note that the abstract's claim is limited to 'outperforming probabilistic baselines,' which is supported, but the conclusion that the model 'captures annotator variability' is stronger than what the current metrics demonstrate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honestly, this is a straightforward transplant of Probabilistic U-Net's CVAE onto SAM, and the paper would be fine if the experiments actually isolated what the latent variable adds. As submitted, they don't.\n\nThe new bit is real: sampling a latent from an image-conditioned prior and adding it to SAM's sparse prompt embeddings is a neat arrangement, and the LIDC evaluation is new. The authors cite Probabilistic U-Net properly, make code public, and the ELBO is standard. If you want a quick reference for \"SAM with a latent-variable uncertainty module,\" this is a usable template.\n\nThe problem is that Table 1 compares Probabilistic SAM against Probabilistic U-Net, Dropout U-Net, and Dropout SAM, but not against a deterministic SAM trained on the same split. GED rewards closeness to the target distribution in expectation, so a sharp consensus mask can get a low GED without any diversity. The reported gain over Probabilistic U-Net could be mostly SAM's stronger backbone. That may be useful in practice, but it doesn't support the claim that the latent mechanism captures annotator variability.\n\nAlso, the GED/DSC comparisons switch baselines (GED vs. PU-Net, DSC vs. Dropout SAM), there's no per-seed variance, and several implementation details are missing: latent dimension, posterior mask encoding, freeze/fine-tune status, GED sampling protocol. Beta=10 is hand-set with no sensitivity analysis. These are fixable but they matter for replication.\n\nI don't think there's anything dishonest here; the method is clear and the qualitative examples show diversity. But the evidence for the central claim is weaker than the abstract suggests.\n\nThis is for people who want a starting point for probabilistic SAM, not for a deep analysis of uncertainty quantification. I'd send it to peer review, but a serious referee should push for a matched-backbone control and a diversity-only metric. With those, it could be a solid paper. Without them, the conclusion overreaches.","headline":"A clean CVAE-on-SAM integration whose evaluation doesn't isolate the latent mechanism from SAM's backbone, so the diversity claim overreaches.","tokens_in":5297,"tokens_out":2392,"would_cite":false,"duration_ms":26769,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Probabilistic SAM beats U-Net on ambiguous lung CT segmentation","keywords":["probabilistic segmentation","Segment Anything Model","conditional variational autoencoder","segmentation uncertainty","medical image segmentation","LIDC-IDRI","generalized energy distance","annotator variability"],"falsifier":"Run deterministic SAM on the same LIDC-IDRI test set with the same box prompts and evaluation protocol (no latent sampling) and compute GED, DSC, and IoU. If deterministic SAM reaches or beats GED 0.2910, the claim that the latent space is what captures ambiguity is falsified.","tokens_in":4390,"feed_emoji":"🩻","tokens_out":4406,"duration_ms":47505,"temperature":0.7,"pith_summary":"This paper tries to extend the Segment Anything Model (SAM) so that, given one image and one prompt, it produces a distribution of plausible segmentation masks instead of a single deterministic one. The proposed model, Probabilistic SAM, injects a conditional latent variable into SAM's pipeline: during training a posterior network sees the ground-truth mask, while a prior network sees only the image, and the sampled latent vector is added to the prompt embeddings before SAM's frozen mask decoder generates the mask. Trained on LIDC-IDRI lung CT slices with four expert annotators, the model reports a GED of 0.2910, DSC of 0.8255, and IoU of 0.7849, outperforming Probabilistic U-Net, Dropout U-Net, and Dropout SAM on these metrics. If correct, this gives clinicians and downstream models a way to see how much a lesion boundary is in dispute and to sample several plausible outlines from a single prompt.","feed_headline":"Probabilistic SAM beats U-Net on ambiguous lung CT segmentation","feed_subtitle":"Adding a variational latent to SAM lets one prompt produce many plausible masks, matching expert disagreement.","key_machinery":"The central mechanism is the learned latent variable z with two Gaussian encoders: a posterior q(z|x,y) trained with the ground-truth mask and a prior p(z|x) trained with the image alone. A latent sample is projected by an MLP and added to SAM's sparse prompt embeddings; the modified embeddings plus image embeddings go into SAM's frozen mask decoder. At inference, drawing z from the prior is the only source of stochasticity, so repeating the draw yields diverse masks. The training objective combines BCE and Dice reconstruction loss with a KL term (weighted by β=10) that aligns posterior and prior.","core_discovery":"Probabilistic SAM claims that segmentation ambiguity can be modeled by conditioning a latent variable on the image and prompt, then using that latent code to modulate SAM's prompt embeddings before decoding. During training, a posterior network q(z|x,y) takes the image and ground-truth mask to define a Gaussian over z, while a prior network p(z|x) uses only the image; the two are aligned with a KL term, and the mask is reconstructed through SAM's decoder using BCE plus Dice loss. At inference, z is drawn from the prior, projected through an MLP, and added to the sparse prompt embedding, making the decoder's output stochastic. On the LIDC-IDRI test set the authors report GED 0.2910, DSC 0.825","pith_inferences":["Because the experiments include no deterministic SAM arm, part of the reported gain may come from SAM's stronger pretrained encoder rather than the latent mechanism; a deterministic SAM baseline with identical prompts would isolate the latent's contribution.","GED rewards mean accuracy as well as diversity, so a model that simply outputs a sharper consensus mask could lower GED without genuinely capturing annotator variability; checking per-expert likelihoods or the spread of samples would test this.","If the latent space is well structured, latent samples should separate by expert tendency; clustering the inferred z vectors by annotator identity is a testable extension.","The same CVAE modulation could be applied to other promptable segmentation models, and the mechanism suggests a route to controllable ambiguity where the latent dimension corresponds to boundary tightness or style."],"forward_implications":["From one box or point prompt, a user can sample many plausible masks and inspect agreement across samples, making segmentation uncertainty visible.","Because only the prior network, MLP, and latent addition are new at inference, the added cost over deterministic SAM is small.","The lower GED reported means the sampled masks as a set are closer to the four-expert annotation distribution than Probabilistic U-Net's samples on this dataset.","The latent modulation is not tied to a specific prompt type or organ, so the same recipe can attach to any SAM prompt and, in principle, other promptable backbones.","The approach directly targets annotation ambiguity rather than only aleatoric noise, since diversity is trained against multiple expert masks."],"supporting_citations":[{"why":"Supplies the base Segment Anything architecture whose image encoder, prompt encoder, and frozen mask decoder are reused.","marker":"[3]"},{"why":"Supplies the conditional variational autoencoder formulation for learning structured output distributions.","marker":"[11]"},{"why":"Provides the variational inference and ELBO objective underlying the prior and posterior training.","marker":"[12]"},{"why":"Provides the LIDC-IDRI dataset of lung CTs with four expert annotations used for training and evaluation.","marker":"[13]"},{"why":"Defines the Probabilistic U-Net baseline and the data resampling/cropping protocol adopted here.","marker":"[14]"},{"why":"Defines the generalized energy distance metric used to measure how close sample sets are to the expert distribution.","marker":"[15]"}],"fun_headline_variants":["One prompt, many masks: Probabilistic SAM captures ambiguity","Probabilistic SAM: dice with uncertainty in medical segmentation","SAM gets probabilistic to match expert disagreement","Variational latent helps SAM output many plausible masks"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The paper assumes the reported improvements come from its latent mechanism, yet no deterministic SAM is evaluated, so a stronger pretrained backbone alone could be responsible for part or all of the gains.","fun_headline_variants_meta":{"raw":{"variants":["One prompt, many masks: Probabilistic SAM captures ambiguity","Probabilistic SAM: dice with uncertainty in medical segmentation","SAM gets probabilistic to match expert disagreement","Variational latent helps SAM output many plausible masks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000438,"raw_usage":{"total_tokens":2077,"prompt_tokens":772,"completion_tokens":1305,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":1243}},"tokens_in":516,"tokens_out":1305,"duration_ms":11862,"temperature":1.0,"reasoning_tokens":1243,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T04:58:04.781601+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run deterministic SAM on the same LIDC-IDRI test set with the same box prompts and evaluation protocol (no latent sampling) and compute GED, DSC, and IoU. If deterministic SAM reaches or beats GED 0.2910, the claim that the latent space is what captures ambiguity is falsified.","supporting_citations":[{"cited_title":"Segment anything,","cited_arxiv_id":null,"evidence_quote":"Supplies the base Segment Anything architecture whose image encoder, prompt encoder, and frozen mask decoder are reused."},{"cited_title":"Learning structured output representation using deep conditional generative models,","cited_arxiv_id":null,"evidence_quote":"Supplies the conditional variational autoencoder formulation for learning structured output distributions."},{"cited_title":"The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans,","cited_arxiv_id":null,"evidence_quote":"Provides the LIDC-IDRI dataset of lung CTs with four expert annotations used for training and evaluation."},{"cited_title":"A probabilistic U-Net for segmentation of ambiguous im- ages,","cited_arxiv_id":null,"evidence_quote":"Defines the Probabilistic U-Net baseline and the data resampling/cropping protocol adopted here."},{"cited_title":"Energy statistics: A class of statistics based on distances,","cited_arxiv_id":null,"evidence_quote":"Defines the generalized energy distance metric used to measure how close sample sets are to the expert distribution."}],"review_version":1}