{"id":"b4ec19bf-0217-4bc2-80db-4e09e6611346","arxiv_id":"2411.19447","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An image-feature scoring engine plus k-means clustering selects prompt frames, improving SAM2-based segmentation in seven medical modalities.","lead":"This paper describes a system that runs the SAM2 segmentation model on medical images and uses a scoring formula to pick which frames a user should annotate first. It reports higher accuracy than random or evenly spaced frame selection across several medical imaging modalities.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II's random baseline uses 1 prompt frame while AFSE uses 5; the reported gains over random are confounded with prompt budget.","rationale":"The paper proposes a heuristic for selecting prompt frames for SAM2 propagation in medical image sequences. The core empirical claim is Table II. In good faith, the method is plausible and the AFSE-versus-Uniform 5 comparison mostly favors AFSE, so I do not see grounds for outright rejection. The most load-bearing problem is that the random baseline is not matched in annotation budget: one prompt frame versus five. This directly undermines the headline that AFSE outperforms 'random' selection. The same weakness also explains why the textual comparisons of improvements appear inconsistent across rows: percentages are being quoted against different baselines without a controlled protocol. A matched-budget random baseline and variance estimates would settle whether any real selection benefit exists. Since the reader already judged the evidence under-supported and conditional, this concern reinforces that verdict rather than changing it.","tokens_in":8580,"tokens_out":5761,"duration_ms":48512,"concrete_test":"Re-run the Table II protocol with a matched-budget random baseline: for each sequence in each of the 10 datasets, sample 5 random prompt frames and evaluate Dice/IoU, repeated over at least 10 seeds, and report mean±std for Random-5, Uniform-5, and AFSE-5 (plus Random-1 and Uniform-1 for completeness). If AFSE-5 does not beat Random-5 on a clear majority of modalities, or if the gap falls within one standard deviation, the central claim fails. Also report the exact values of α, β, γ, δ, ε used in Eq. (1) to make the selection rule reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-B and Table II claim AFSE consistently outperforms random and uniform strategies, but the comparison is not controlled. The Table II caption defines R as the number of reference frames; 'Random 1' therefore evaluates SAM2 propagation with a single prompt frame, while 'AFSE 5' uses five. The 6-11 point improvements over the random row could result from the larger prompt budget alone, not from the selection engine. The only same-budget baseline, 'Uniform 5', is beaten by AFSE on six of seven modalities, yet the margin on US is 0.15 Dice and Fundus is worse (99.75 vs 99.97), so 'consistently' is already too strong. No error bars, confidence intervals, or repeated random seeds are reported for any Table II row, despite the validation set being only 30% of each dataset; with a single random draw per modality, the random row is a point estimate with unknown variance. The undisclosed weights in Eq. (1) are a secondary reproducibility problem, but the prompt-budget mismatch is the decisive confound: it prevents the experiment from attributing the improvement to the AFSE scoring mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SISeg, a SAM2-based interactive segmentation framework for multimodal medical imaging, centered on an Adaptive Frame Selection Engine (AFSE). AFSE scores each frame with a weighted combination of brightness, contrast, edge density, color histogram similarity, and Hu-moment shape similarity relative to a clinician-chosen reference frame, then applies k-means clustering to select representative prompt frames for SAM2's propagation. Experiments across 10 datasets and 7 modalities report Dice scores comparing AFSE to random and uniform selection strategies, and the authors claim consistent improvements. The central claim is that AFSE's scoring mechanism selects frames that improve downstream segmentation Dice while reducing manual prompts.","tokens_in":8844,"tokens_out":5398,"duration_ms":42440,"significance":"If the core claim were established, the framework could offer a practical way to reduce manual prompting in SAM2-based medical image segmentation, with potential value for modalities where sequence-level interaction is needed. The paper's breadth—10 datasets across 7 modalities—is a strength, as is the integration of a selection mechanism into a widely used foundation model. However, the current evidence does not support the central claim because of a prompt-budget confound in the main comparison, unspecified parameters in the scoring function, and the absence of statistical validation. These issues are load-bearing and require additional experiments rather than simple edits.","major_comments":[{"comment":"The comparison is confounded by prompt budget: the 'Random 1' row uses one reference frame while 'AFSE 5' uses five. The reported gains (e.g., Der 59.84 to 62.55, OCT 31.99 to 33.90) could be due to the larger number of prompt frames alone, not to the AFSE scoring mechanism. A same-budget random baseline (Random 5) is required to support the claim that the selection engine is responsible for the improvement. The only same-budget baseline, Uniform 5, actually beats AFSE on Fundus (99.97 vs 99.75) and essentially ties on US (99.83 vs 99.98), so the statement that AFSE 'consistently outperformed both random and uniform strategies' is not supported by the data as presented.","section":"IV-B, Table II"},{"comment":"The weights α, β, γ, δ, ε in the composite score F are not reported anywhere. Since F is the basis of all frame selection, the method is not reproducible without these values, and the results may depend on constants chosen while looking at the test datasets. The authors should report the exact weights used, explain how they were set, and provide a sensitivity analysis over them (e.g., equal weights versus alternative values). Similarly, the number of clusters k in Eq. (7) is not specified; Table II uses R=5, but the relationship between R and k is never stated.","section":"III-C, Eq. (1)"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any row of Table II. With only a 30% validation split and a single random draw, the random baseline is a single point estimate with unknown variance. The authors should run multiple random seeds or bootstrap resampling and report mean ± std, so that the comparisons between AFSE, Uniform, and Random can be assessed statistically.","section":"IV-A/IV-B"},{"comment":"The evaluation protocol is underdescribed. It is not stated how the bounding-box prompts are generated for each modality (e.g., manual annotation, automatic detector, or ground-truth boxes), nor how the clinician-chosen reference frame is selected and whether the same reference is used for all compared methods. In addition, the SAM2 variant used for Tables II–IV is not identified, although Table I shows substantial variation across variants (e.g., Endo Dice ranges from 85.27 to 93.61). This information is essential for reproducing the experiments.","section":"IV-A/IV-B"},{"comment":"The core assumption that frames similar to the reference in brightness, contrast, edge density, color histogram, and Hu moments are the best prompt frames for SAM2 propagation is not independently validated. The paper does not report any correlation between F and downstream Dice, nor does it compare against a diversity-based selection baseline. The ablation in Table IV shows individual features but still lacks a same-budget random control, so it does not demonstrate that F predicts prompt quality.","section":"III-C"}],"minor_comments":[{"comment":"The index term 'Segmentatino' is a typo for 'Segmentation'.","section":"Abstract/Index Terms"},{"comment":"The 'Four Neg' row for Endo reports '00.78', which appears to be a typo for '0.78' or another value; please correct.","section":"Table III"},{"comment":"The column headers in Table IV are run together (e.g., 'MGBrightness62.31'); the table should be formatted with clear separations between modality names and Dice values.","section":"Table IV"},{"comment":"The text states that AFSE surpasses the second-best method 'by 9.39% in X-ray and 10.97% in Mammography', but Table II shows differences of 4.28 Dice points (60.07 vs 55.79) and 10.97 points (50.82 vs 39.85), respectively. The percentages are inconsistent with the tabulated values.","section":"IV-B"},{"comment":"The 'Medical SAM Adapter' reference appears to be duplicated ([15] and [16]); please check the bibliography and ensure each cited work is distinct and correctly referenced.","section":"References"},{"comment":"The datasets are split into 'training and validation sets', but no training is performed since SAM2 is used zero-shot. Clarify whether 'training' is simply a partition name or whether any fine-tuning occurs.","section":"IV-A"},{"comment":"The definition of S in Eq. (6) is ambiguous: the expression appears to be -log(Σ|MHu(Iref)i − MHu(I)i| + ε), but the placement of parentheses is not clear. Using a clearer notation would avoid confusion.","section":"III-C, Eq. (6)"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is not yet supported because of the prompt-budget confound in Table II and the unreported weights in Eq. (1). These are fixable with additional experiments (a same-budget Random 5 condition, multiple seeds, and either reported weights or a sensitivity analysis). I would not reject the manuscript, but the revision needs to address these load-bearing points before the claim can be taken seriously. The lack of statistical rigor and the overstatement of 'consistent' superiority are also concerning. Please also check the numerical inconsistencies between the text and tables during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful part of this paper is the broad empirical comparison of prompt strategies across seven medical imaging modalities. That is worth having: Table III gives a practical map of which prompt types work for dermoscopy versus ultrasound versus OCT, and the finding that bounding boxes beat point prompts consistently is believable. The AFSE selection engine itself is a simple, plausible idea: score frames by low-level features relative to a clinician-chosen reference, cluster the scores, and use cluster centroids as prompt frames. Nothing about it is deep, and the paper does not claim it is.\n\nThe soft spots are real. The most damaging is Table II. The caption says R is the number of reference frames, and the random baseline uses R=1 while AFSE uses R=5. So the 6–11 point gains over \"Random 1\" are confounded with prompt budget; they do not demonstrate that the selection mechanism works. Against the same-budget \"Uniform 5\" baseline, AFSE wins on six of seven modalities, but the margins are tiny on US (99.98 vs 99.83) and it loses on Fundus. \"Consistently outperforms\" is overstated. There are also no error bars or repeated seeds anywhere, and the validation set is a single 30% split per dataset, so we have no idea how stable any of these numbers are.\n\nThe second real problem is the scoring formula itself. Equation (1) defines F as a weighted sum of five feature scores, and the weights α, β, γ, δ, ε are never given and no fitting procedure is described. That makes the central mechanism irreproducible. You cannot check whether the selection is doing anything beyond, say, picking frames by brightness, and you cannot apply the method to a new dataset without guessing the weights. This is a bigger issue than the absence of code, because even with code the weight values would need to be justified.\n\nMinor issues: no comparison to MedSAM or Medical SAM Adapter, which are the natural baselines for this task; a typo in the index terms; and a suspicious \"00.78\" in Table III that looks like a missing digit. These are not load-bearing but add to the sloppiness.\n\nWho is this for? People building practical annotation workflows on top of SAM2 for medical images. The prompt-strategy comparison is citable, and the AFSE idea is worth a look if you are already using video propagation on 2D image stacks. But the evidence that AFSE beats uniform sampling is weak as presented. The paper deserves a serious referee because the issues are addressable with a controlled budget, multiple seeds, and disclosed weights, and because the empirical scope is genuinely useful. I would not cite it myself until the controlled version exists.","headline":"Plausible SAM2 prompt-selection heuristic with a genuinely useful modality-prompt comparison, but the headline random baseline is confounded by prompt budget and the scoring weights are undisclosed.","tokens_in":9335,"tokens_out":2195,"would_cite":false,"duration_ms":20485,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a hand-designed frame-selection score improves SAM2-based interactive medical segmentation by 6 to 11 Dice points across seven imaging modalities.","keywords":["medical image segmentation","interactive segmentation","Segment Anything Model 2","frame selection","unsupervised scoring","k-means clustering","multimodal imaging","prompt selection"],"falsifier":"Take one modality, such as the chest X-ray dataset, and run the AFSE pipeline while holding out one of the five feature terms from $F$ at a time and while sweeping the five weights and the choice of reference frame; if the reported Dice advantage over uniform selection disappears under any of these variations, the claim that the composite score selects optimal prompt frames is not supported.","tokens_in":8371,"feed_emoji":"🩻","tokens_out":8219,"duration_ms":66428,"temperature":0.7,"pith_summary":"This paper tries to show that the biggest bottleneck in interactive medical image segmentation is not the segmentation model but the choice of which frames to prompt. It introduces SISeg, a SAM2-based system whose Adaptive Frame Selection Engine (AFSE) scores every frame by brightness, contrast, edge density, color-histogram similarity, and Hu-moment shape similarity to a clinician-chosen reference, then uses k-means clustering to pick representative frames as prompts. Across ten datasets covering seven imaging modalities, the paper reports that this selection strategy outperforms random and uniform frame choices, with Dice gains of roughly 6 to 11 points on several modalities while using the same number of prompts. A sympathetic reader would care because the engine is unsupervised, needs no per-modality fine-tuning, and could lower the manual annotation cost of segmenting medical image sequences.","feed_headline":"Frame-selection engine lifts medical segmentation Dice up to 11 points","feed_subtitle":"An unsupervised scorer picks which frames to prompt, beating random and uniform picks across seven imaging modalities.","key_machinery":"The central mechanism is the Adaptive Frame Selection Engine (AFSE), built from a Scorer and a Selector. The Scorer computes the composite score $F = \\alpha B + \\beta C + \\gamma E + \\delta H + \\epsilon S$ from five image features; the Selector applies k-means to the resulting scores and returns the frames closest to the cluster centroids as the prompt frames for SAM2. This turns prompt selection into an unsupervised clustering problem, and it is this automatic ordering and selection that the paper credits for the accuracy improvements and reduced memory load during sequence inference.","core_discovery":"The central discovery claimed is that an unsupervised, hand-designed scoring function can reliably select prompt frames that let SAM2 propagate segmentation through 2D medical image sequences better than naive frame selection. The score is the weighted combination $F = \\alpha B + \\beta C + \\gamma E + \\delta H + \\epsilon S$, where $B$ is mean grayscale brightness, $C$ is the standard deviation of grayscale intensities, $E$ is Canny edge density, $H$ is HSV histogram correlation with the reference frame, and $S$ is a Hu-moment shape similarity term. K-means clustering on this one-dimensional score partitions the sequence, and the frames nearest the centroids become the prompts that drive SAM2's propagation. The paper reports that AFSE with the scorer beats AFSE without the scorer by 9.39 Dice points in X-ray and 10.97 points in mammography, and that bounding-box prompts consistently outperform point prompts across modalities.","pith_inferences":["A natural extension the paper leaves implicit is per-modality weight tuning for the five terms in $F$; if the weights used in the paper are not optimized, the reported Dice gains are a lower bound on what the selection idea could achieve.","The same frame-selection logic applies to any video or volumetric segmentation model, not just SAM2; testing AFSE on a different propagation backbone would show whether the gains come from the selection or from the model.","One testable risk is that the score measures similarity to a single clinician-chosen reference, so the method's success may depend on that reference being representative; swapping the reference frame is an easy robustness check.","A possible future direction is to replace the five hand-picked features with features extracted by the segmentation model itself, which could make the selection engine adapt to the task rather than to generic image statistics."],"forward_implications":["On modalities such as X-ray and mammography, AFSE's scoring mechanism is credited with Dice improvements of about 9 to 11 points relative to a version without the scorer, so clinicians could annotate fewer frames for the same accuracy.","The method works with several SAM2 Hiera model sizes, meaning the selection engine can be paired with whatever capacity the deployment hardware allows.","Bounding-box prompts should be the default prompt type for SAM2-based medical segmentation, since the paper finds point-prompt combinations are brittle across modalities.","Because the scorer is unsupervised and needs no medical prior, the same framework can be dropped onto a new modality with only a reference frame and a set of cluster centers to define."],"supporting_citations":[{"why":"SAM2 is the segmentation backbone whose video-propagation mechanism carries the prompts selected by AFSE; the whole method is built on it.","marker":"[11]"},{"why":"Demonstrates adapting SAM to medical images with bounding-box prompts, the prior adaptation the paper positions its multi-prompt strategy against.","marker":"[9]"},{"why":"Shows single-point segmentation for medical images, the prompt strategy that motivates the need for adaptive frame and prompt selection.","marker":"[26]"},{"why":"Kvasir-SEG supplies the endoscopy images on which the AFSE-versus-uniform comparison is measured.","marker":"[31]"},{"why":"The chest X-ray dataset used to evaluate the X-ray modality, where AFSE shows a large reported gain over uniform selection.","marker":"[36]"}],"fun_headline_variants":["Simple frame scorer lifts SAM2 segmentation by up to 11 Dice","Unsupervised frame selection boosts medical segmentation by 11 Dice","Adaptive prompt picker raises segmentation Dice in 7 imaging modalities","Scorer-driven frame choice improves multimodal segmentation by 11 points","K-means on one score picks frames that lift SAM2 by 11 Dice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a frame's similarity to a clinician-chosen reference, measured by a weighted sum of brightness, contrast, edge density, color-histogram correlation, and Hu-moment shape, predicts how well prompting that frame will let SAM2 segment the rest of the sequence, and the paper never reports the weights or validates that assumption independently.","fun_headline_variants_meta":{"raw":{"variants":["Simple frame scorer lifts SAM2 segmentation by up to 11 Dice","Unsupervised frame selection boosts medical segmentation by 11 Dice","Adaptive prompt picker raises segmentation Dice in 7 imaging modalities","Scorer-driven frame choice improves multimodal segmentation by 11 points","K-means on one score picks frames that lift SAM2 by 11 Dice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2601,"prompt_tokens":926,"completion_tokens":1675,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":1583}},"tokens_in":542,"tokens_out":1675,"duration_ms":11931,"temperature":1.0,"reasoning_tokens":1583,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:10:44.628986+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one modality, such as the chest X-ray dataset, and run the AFSE pipeline while holding out one of the five feature terms from $F$ at a time and while sweeping the five weights and the choice of reference frame; if the reported Dice advantage over uniform selection disappears under any of these variations, the claim that the composite score selects optimal prompt frames is not supported.","supporting_citations":[{"cited_title":"Segment anything in medical images,","cited_arxiv_id":null,"evidence_quote":"Demonstrates adapting SAM to medical images with bounding-box prompts, the prior adaptation the paper positions its multi-prompt strategy against."},{"cited_title":"One-prompt to segment all medical images,","cited_arxiv_id":null,"evidence_quote":"Shows single-point segmentation for medical images, the prompt strategy that motivates the need for adaptive frame and prompt selection."},{"cited_title":"Kvasir-SEG: A Segmented Polyp Dataset,","cited_arxiv_id":null,"evidence_quote":"Kvasir-SEG supplies the endoscopy images on which the AFSE-versus-uniform comparison is measured."},{"cited_title":"CheXmask: a large- scale dataset of anatomical segmentation masks for multi-center chest x-ray images,","cited_arxiv_id":null,"evidence_quote":"The chest X-ray dataset used to evaluate the X-ray modality, where AFSE shows a large reported gain over uniform selection."}],"review_version":1}