{"id":"be32597c-4ae4-4f94-8a26-47612016c4ae","arxiv_id":"2412.13742","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"KnowSAM couples multi-view co-training, learnable dense prompting of SAM, and SAM-to-student knowledge distillation to boost semi-supervised medical image segmentation and reports SOTA Dice scores across five benchmarks.","lead":"KnowSAM is a semi-supervised medical image segmentation framework that combines two student networks with a learnable prompting scheme for the Segment Anything Model (SAM) and knowledge distillation from SAM to improve segmentation with very few labeled images. The authors report state-of-the-art Dice scores on five medical benchmarks, but the gains need independent verification because no error bars are given and the SAM teacher is partly influenced by the students.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that SAM distillation injects independent knowledge is untested: because the mask prompt to SAM (Eq. 7) is the students' own fused prediction, Ys may echo Yf and Lkd (Eq. 10) may be self-distillation; no control with an independent teacher or prompt-free SAM is reported.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the SAM teacher's predictions may not be independent of the student subnets because the mask prompt is built from their fused output. That is exactly the point where the central claim is least secure. My reading of the equations and tables supports this concern: Eq. 7 makes the teacher's output a function of the students' output, and Eq. 10 then distills that output back into the students. Without a control condition that breaks this loop, the reported performance gains cannot be attributed to knowledge from the foundation model. The Table V observation that SAM with LPS alone outperforms the final KnowSAM student on Kvasir and ISIC at 10% labels further indicates that the distillation is not transferring the teacher's full capability. I also note the absence of error bars and multiple seeds as a secondary issue, but the independence problem is more fundamental and is shared with the reader's assessment. Therefore the reader's CONDITIONAL verdict is appropriate; an additional experiment with an independent teacher or self-distillation control would either confirm or resolve the concern.","tokens_in":22309,"tokens_out":3178,"duration_ms":32660,"concrete_test":"Run two control conditions on at least one dataset (e.g., Kvasir with 10% labels): (1) Self-KD: replace Ys in Eq. 10 with the fused student prediction Yf, keeping all other losses and the SAM adapter frozen or unused; (2) Independent-teacher: prompt SAM with an empty/constant mask, or with masks from an independently trained frozen network, so the teacher is not a function of the current students. If condition (1) matches KnowSAM within about 0.5 Dice, the SKD gain is not evidence of external SAM knowledge. Additionally, report the agreement Dice(Ys, Yf) on unlabeled data during training: if it exceeds roughly 0.95 throughout, the teacher is largely copying the prompt, confirming the self-referential loop.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central assertion is that SKD transfers useful knowledge from SAM into the subnets via Eq. 10. This requires SAM's prediction Ys to carry information not already present in the students. But Ys is produced from a mask prompt Yf that is a deterministic fusion of the students' own outputs (Eqs. 1-3 and Eq. 7: Ys = Fs(Pb, Yf; Theta_s)). SAM's decoder is strongly conditioned on mask prompts, so Ys can be a near-copy of Yf; in that regime Lkd = KL(Ya_T, Ys_T) + KL(Yb_T, Ys_T) is essentially a consistency/self-distillation term between each subnet and their own fused prediction, and it cannot correct errors the two subnets share. The paper reports no control with an independent teacher or with the mask prompt removed, so the reported gains (e.g., BCSS 10%: BCP 66.00 vs KnowSAM 72.69) may come from the extra mutual-consistency and entropy regularization that SKD adds rather than from SAM's external knowledge. Supporting evidence that the teacher knowledge is not being captured: in the LPS ablation (Table V), SAM with LPS alone reaches 87.29 Dice on Kvasir-10% and 87.90 on ISIC-10%, while the final KnowSAM student reaches 85.98 and 86.51, so the students do not catch up to the teacher, consistent with Lkd acting as a weak regularizer rather than successful knowledge transfer. The absence of error bars or multi-seed runs makes the small reported gains additionally fragile.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KnowSAM, a semi-supervised medical image segmentation framework that combines a multi-view co-training scheme with two sub-networks, a learnable prompt strategy for SAM that feeds a fused mask prompt and learned dense feature prompts into SAM's decoder, a SAM-induced knowledge distillation loss that pushes the two sub-networks toward SAM's softened predictions, and an uncertainty-guided copy-paste data augmentation. The method is evaluated on five colonoscopy datasets, ISIC-2018, thyroid ultrasound datasets, ACDC, and BCSS at 10% and 30% labeled ratios, reporting Dice/IoU/95HD improvements over several existing semi-supervised methods. The paper also presents ablations for the hybrid aggregation module, the prompt strategy, the data augmentation, and the loss components.","tokens_in":22739,"tokens_out":5405,"duration_ms":48141,"significance":"If the SAM-knowledge-distillation mechanism genuinely transfers external knowledge beyond the sub-networks' own consensus, the framework would be a valuable contribution to semi-supervised medical image segmentation, and the reported gains over strong baselines such as BCP and CAML are potentially interesting. The paper deserves credit for a broad experimental scope, a fairly detailed ablation study, and a planned code release. However, the central claim that SAM provides independent knowledge is not convincingly established, because the mask prompt given to SAM is the students' own fused prediction, making the distillation loop partly self-referential. The absence of error bars or repeated label splits further weakens confidence in the point-estimate improvements. The paper is well structured and clearly written, but the load-bearing experimental evidence needs strengthening.","major_comments":[{"comment":"The knowledge-distillation loop is self-referential: the mask prompt for SAM is the fused prediction Yf of the two subnets (Eq. 3), and Eq. (7) produces Ys = Fs(Pb, Yf; Θs). The distillation loss Lkd in Eq. (10) then trains each subnet to match Ys. Since SAM's decoder is strongly conditioned on the mask prompt, Ys may be a near-copy of Yf, making Lkd essentially a consistency loss between each subnet and their own fused output rather than a transfer of external knowledge. The paper reports no control with an independent teacher (e.g., SAM prompted by fixed coordinates or by ground-truth masks on labeled data) and no comparison with a prompt-free SAM variant. Please add such ablations and quantify the agreement between Ys and Yf on unlabeled data to demonstrate that SAM injects information beyond the subnets' own consensus.","section":"Section III-C, Eqs. (7) and (10)"},{"comment":"The LPS ablation is internally inconsistent with the distillation claim. On Kvasir-10%, SAM with LPS alone reaches 87.29 Dice, while the final KnowSAM student with LPS reaches only 85.98; on ISIC-10% the corresponding numbers are 87.90 for SAM and 86.51 for KnowSAM. Moreover, KnowSAM with LPS is worse than KnowSAM with Box prompting on Kvasir-10% (85.98 vs 86.32). If SKD successfully transferred SAM's knowledge, the students should approach or exceed the teacher's performance; the observed gap instead suggests that Lkd acts as a weak regularizer. Please reconcile these numbers, clarify whether the final inference uses Yf or a single subnet output, and discuss why LPS improves SAM but does not translate into an improvement of the final KnowSAM student on these datasets.","section":"Section IV-D, Table V"},{"comment":"All reported results are single point estimates from one labeled/unlabeled split, without error bars, multiple random splits, or statistical significance tests. This is especially consequential at the 10% labeled ratio, where the ACDC experiment uses only 7 labeled scans and the reported gain over the second-best method is as small as 0.51 Dice points (TN3K-10%). Please provide mean ± std over at least three random splits, or otherwise justify that the reported differences are stable and not artifacts of a particular split.","section":"Section IV-C, Tables I-III"},{"comment":"The BCSS experimental setup is inconsistent. The dataset description states that 3,888 images are divided into training, validation, and test sets in the proportions 70%, 10%, and 20%, which implies roughly 2,721 training images and about 272 labeled images at a 10% labeled ratio. However, Table III reports '7(10%) 63(90%)', which appears to be the ACDC scan counts rather than the BCSS image counts. Please correct the label count or clarify the actual data split used; as written, the table is not interpretable and the '10% labeled' claim for BCSS is ambiguous.","section":"Section IV-A and Table III"},{"comment":"The paper repeatedly claims that the SAM distillation framework 'can be seamlessly integrated into other semi-supervised segmentation methods' and 'easily extended' to other tasks, but no experiment demonstrates such integration. The only comparison is between the proposed MC-seg baseline and the full KnowSAM; no experiment plugs SKD into an existing method such as BCP or UA-MT. Either provide integration experiments that substantiate this claim or remove/weaken it in the abstract and conclusion.","section":"Abstract and Section V"}],"minor_comments":[{"comment":"Equation (13) sums over classes c=0 to C, while Eq. (1) sums over c=0 to C-1; please make the index range consistent and clarify whether the background class is included in the entropy loss.","section":"Eq. (13)"},{"comment":"The text says the comparison includes 12 SOTA methods but then enumerates only nine (MT, UA-MT, DTC, MC-Net, MC-Net+, URPC, MCF, BCP, CAML). CDMA, CauSSL, and BS-Net appear in the tables but are missing from the enumeration; please correct the list.","section":"Section IV-C"},{"comment":"The caption refers to 'two types of prompts (Yf and the learnable feature prompt)', but the learnable feature prompt is denoted Pb in the text; please unify the notation between the figure and the equations.","section":"Fig. 2 caption"},{"comment":"The table rows labeled 'SAM Box', 'SAM LPS', 'KnowSAM Box', and 'KnowSAM LPS' should explicitly state whether the numbers are SAM's own predictions or the final fused inference of the subnets; the accompanying text says 'we report the predicted results of SAM and the fused predictions Yf', but the row labels do not make this distinction clear.","section":"Table V"},{"comment":"The sentence 'This is attributed to a learnable feature prompt provided by the LPS' is vague; it should specify the mechanism by which the learned dense prompt tokens improve SAM's predictions compared with hard coordinate prompts.","section":"Section IV-D.2"},{"comment":"Please clarify whether the SAM teacher is fine-tuned during training and whether the gradient of Lkd is blocked from flowing into SAM; Eq. (8) shows a supervised loss on SAM, but the interaction between Lsam and Lkd is not fully described.","section":"Section III-C"}],"recommendation":"major_revision","confidential_remarks":"The paper is well-written and the empirical scope is broad, but the central mechanism of SAM-induced knowledge distillation needs stronger validation. The self-referential prompt loop (Eqs. 7 and 10) is a genuine concern that can be addressed with additional control experiments. The BCSS '7(10%)' count appears to be a copy-paste error from ACDC and must be corrected. The unsupported claim about seamless integration into other methods should also be either substantiated or removed. I recommend major revision rather than rejection, as the issues appear fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"KnowSAM is a well-assembled combination of known semi-supervised ingredients—co-training with mutual consistency, copy-paste augmentation, adapters for SAM—plus a learnable dense prompt network that replaces hard coordinates. The empirical sweep is genuinely broad: five colonoscopy sets, ultrasound, ISIC, ACDC, BCSS. The MC-seg baseline itself beats strong methods like BCP on several tasks, and the writing is clear.\n\nThe soft spots are real. The load-bearing issue is that the SAM teacher is not independent. Eq. 7 feeds the fused student prediction Yf to SAM as a mask prompt, and Eq. 10 distills SAM's output back into the students. No control with an independent teacher, or with the mask prompt removed, is reported. The stress-test note is right: Ys may be a near-copy of Yf, making Lkd a self-consistency regularizer rather than knowledge transfer. Table V is consistent with that reading—SAM with LPS alone reaches 87.29 Dice on Kvasir-10% while the final KnowSAM student gets 85.98. The students never catch the teacher. That doesn't kill the paper, but it means the 'SAM-induced knowledge' claim is unverified.\n\nThe LPS ablation also has an internal inconsistency: KnowSAM with Box beats KnowSAM with LPS on Kvasir (86.32 vs 85.98), undercutting the claim that LPS helps. The paper reports no error bars or multiple random label splits, which matters at 10% labeled (7 scans on ACDC). And the abstract's claim that the distillation framework can be 'seamlessly integrated' into other SSL methods has no experiments behind it.\n\nThe core architecture is new enough, and the empirical tables are rich enough, that the paper deserves a serious referee. Anyone working on SAM-based semi-supervised segmentation will want to read this, but they should treat the SOTA numbers as conditional. The revision needs three things: error bars, a control that isolates whether SAM contributes independent knowledge (e.g., a frozen prompt-free SAM or a teacher with prompts from ground truth), and a coherent LPS story. As it stands, I'd treat the SOTA claims as plausible but not established.","headline":"A broad, cleanly written SAM-distillation SSL paper; the empirical sweep is impressive but the teacher is fed the students' fused predictions, so the key novelty is unverified.","tokens_in":23251,"tokens_out":4159,"would_cite":false,"duration_ms":33249,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"KnowSAM reports that distilling a SAM teacher's softened predictions into two co-trained subnets beats prior semi-supervised medical segmentation methods on five benchmarks.","keywords":["semi-supervised medical image segmentation","knowledge distillation","Segment Anything Model","learnable prompting","co-training","uncertainty-guided augmentation","pseudo-labeling"],"falsifier":"On a held-out labeled subset, compute SAM's prediction when prompted by the fused student mask and by a mask from an independent source; if SAM's output with the fused prompt is not measurably closer to the ground truth than the students' fused output is, the distillation gain cannot be coming from new SAM knowledge.","tokens_in":22115,"feed_emoji":"🩺","tokens_out":6858,"duration_ms":62515,"temperature":0.7,"pith_summary":"The paper tries to establish that a general-purpose segmentation foundation model can be converted into a reliable teacher for semi-supervised medical image segmentation, rather than being used directly as a segmenter. It proposes KnowSAM, a framework in which two student sub-networks co-train by supervising each other, their fused prediction is offered to SAM as a mask prompt, and SAM's softened predictions are distilled back into both students. The authors claim this loop outperforms twelve existing semi-supervised methods across colonoscopy, dermoscopy, thyroid ultrasound, cardiac MRI, and breast-cancer histology benchmarks, with the largest gains at 10% labeled data. If the claim holds, SAM-induced distillation is a reusable ingredient that can be grafted onto other semi-supervised segmentation pipelines to reduce reliance on pixel-level annotations.","feed_headline":"SAM as teacher lifts Dice in semi-supervised segmentation","feed_subtitle":"Distilling SAM's softened predictions into two co-trained subnets lifts Dice on five medical benchmarks.","key_machinery":"The load-bearing mechanism is a teacher-student loop closed through SAM's decoder. Two subnets produce predictions $\\hat{Y}_a$ and $\\hat{Y}_b$; their entropy maps $\\mathcal{H}_a, \\mathcal{H}_b$ and an XOR dissimilarity map $\\mathcal{M}$ are fused into an aggregated map $\\hat{Y}_f$, which is fed to SAM as a mask prompt. A lightweight network $\\psi(\\cdot)$ maps SAM's image embedding $Z$ to a learnable dense prompt $P_b$, and SAM's decoder computes $\\hat{Y}_s = F_s(P_b, \\hat{Y}_f; \\Theta_s)$. The distillation loss is $L_{kd} = KL(\\hat{Y}^a_T, \\hat{Y}^s_T) + KL(\\hat{Y}^b_T, \\hat{Y}^s_T)$ with temperature-softened probability maps, and gradients flow only into the two subnets, so the students learn to reproduce SAM's refined predictions while SAM itself is supervised only by ground-truth labels.","core_discovery":"The central discovery is that a foundation segmenter can be turned into an effective teacher for semi-supervised medical segmentation if its prompt is the fused prediction of two student networks and its softened outputs are distilled back into those students. The authors call this SAM-induced Knowledge Distillation (SKD). Around that loop, a learnable prompt strategy generates dense feature prompts through a lightweight network instead of hard box coordinates, and an adapter fine-tunes SAM for medical data; a multi-view co-training module fuses the two subnets' predictions with entropy and dissimilarity maps; and an uncertainty-guided copy-paste augmentation creates mixed training samples. On the BCSS pathology benchmark with 10% labeled data, KnowSAM reaches 72.69 Dice compared with 66.00 for BCP, and the paper reports consistent improvements across all five evaluated medical segmentation tasks.","pith_inferences":["The reported gains are all measured with the students' own fused mask as SAM's prompt; an untested consequence is that the loop may be partly self-confirming. Comparing against an independently prompted SAM would separate genuine transfer from echo.","The method's reliance on SAM means its performance on out-of-distribution medical images is tied to how well SAM generalizes there; a domain-shift stress test with unseen scanners or stain types would reveal whether the distillation adds robustness or only in-distribution accuracy.","Because the KL distillation is applied to unlabeled data with no supervised term, the benefit should scale with the amount and diversity of unlabeled data; the paper does not report a scaling curve with unlabeled count varied independently, which would be a natural follow-up."],"forward_implications":["On the BCSS pathology benchmark with 10% labeled data, KnowSAM reaches 72.69 Dice versus 66.00 for BCP, and the pattern of improvement repeats on colonoscopy, ISIC-2018, thyroid ultrasound, and ACDC.","The SAM distillation component alone, without the entropy or mutual-consistency losses, already produces large gains, so the KD loop carries much of the reported improvement.","At inference time only the two lightweight subnets are needed; the SAM teacher can be discarded, so the accuracy gain does not add runtime cost.","Because the design is presented as a general distillation wrapper, other semi-supervised segmentation methods can adopt the same SAM-teacher loop, which the paper states explicitly.","Replacing hard box coordinates with a learnable dense prompt avoids the sharp performance drops that incorrect box prompts cause with SAM; the ablation shows the learnable prompt strategy consistently beats box prompting for SAM on all four tested datasets."],"supporting_citations":[{"why":"Supplies SAM, the frozen foundation model used as the distillation teacher.","marker":"[17]"},{"why":"Provides the temperature-softened KL-divergence recipe used in the SKD loss.","marker":"[41]"},{"why":"Supplies the adapter-tuning approach used to fine-tune SAM's encoder and decoder.","marker":"[19]"},{"why":"The BCP baseline that KnowSAM compares against and whose copy-paste idea UGDA builds on.","marker":"[4]"},{"why":"Supplies the cross pseudo-supervision mutual-consistency mechanism used to align the two subnets.","marker":"[39]"},{"why":"A SAM-based semi-supervised baseline that KnowSAM outperforms in the comparison tables.","marker":"[22]"},{"why":"The CPC-SAM baseline that KnowSAM outperforms in the SAM-based comparison.","marker":"[61]"},{"why":"The MedSAM baseline used to test whether the framework's gains depend on the choice of SAM over a medical-tuned variant.","marker":"[62]"}],"fun_headline_variants":["KnowSAM: SAM teaches students to lift Dice","Learnable prompts make SAM a med-seg teacher","Distilling SAM's dense prompts boosts semi-supervised Dice","Co-trained subnets learn from SAM's softened predictions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes SAM's prediction is better than the students' fused output, so that distilling it back into the students adds real new information instead of amplifying their existing mistakes.","fun_headline_variants_meta":{"raw":{"variants":["KnowSAM: SAM teaches students to lift Dice","Learnable prompts make SAM a med-seg teacher","Distilling SAM's dense prompts boosts semi-supervised Dice","Co-trained subnets learn from SAM's softened predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000159,"raw_usage":{"total_tokens":1244,"prompt_tokens":976,"completion_tokens":268,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":205}},"tokens_in":592,"tokens_out":268,"duration_ms":3114,"temperature":1.0,"reasoning_tokens":205,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:50:07.184537+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out labeled subset, compute SAM's prediction when prompted by the fused student mask and by a mask from an independent source; if SAM's output with the fused prompt is not measurably closer to the ground truth than the students' fused output is, the distillation gain cannot be coming from new SAM knowledge.","supporting_citations":[{"cited_title":"Bidirectional copy- paste for semi-supervised medical image segmentation,","cited_arxiv_id":null,"evidence_quote":"The BCP baseline that KnowSAM compares against and whose copy-paste idea UGDA builds on."},{"cited_title":"Semi-supervised semantic segmentation with cross pseudo supervision,","cited_arxiv_id":null,"evidence_quote":"Supplies the cross pseudo-supervision mutual-consistency mechanism used to align the two subnets."},{"cited_title":"Cross prompting consistency with segment anything model for semi- supervised medical image segmentation,","cited_arxiv_id":null,"evidence_quote":"The CPC-SAM baseline that KnowSAM outperforms in the SAM-based comparison."}],"review_version":1}