{"id":"fa070a88-4c8a-462b-8355-ccfc0e855295","arxiv_id":"2607.23694","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LoRA on SAM3’s prompt encoder, detector, and tracker (0.98% of parameters) raises surgical concept-segmentation mIoU over zero-shot SAM3 and Medical SAM3 while fitting in ~9 GB GPU memory.","lead":"The authors adapt SAM3 to surgical images with LoRA on only the prompt encoder, detector, and tracker, freezing the vision backbone so 0.98% of weights train on one consumer GPU. The adapted model beats zero-shot SAM3 and fully fine-tuned Medical SAM3 on three surgical datasets and feeds masks into reconstruction and physics simulation demos.","discovery_kind":"extension","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The headline win over \"fully fine-tuned Medical SAM3\" rests on a baseline that appears to be run zero-shot and possibly through the wrong prompt interface; near-zero mIoU on classes like liver and Abdominal Wall is the signature of an evaluation mismatch, not of a model that was fully fine-tuned on ","rationale":"The reader identified the same locus (Medical SAM3 baseline integrity in §3.2, Tables 1-2) as the weakest assumption; I agree it is the load-bearing point, and I sharpen the mechanism: the paper's own §3.2 language (\"dilutes its specialized surgical features\") indicates the baseline is the pretrained checkpoint rather than a surgically fine-tuned model, and the unspecified prompt mode for a box/exemplar-prompted model is the most plausible cause of exact 0.00 scores. This is a concrete, checkable correctness risk, not a consensus-disagreement issue. The rest of the paper holds up reasonably: the LoRA recipe is fully specified (r=16, alpha=32, 8.32M params, 9 GB, hyperparameters all given), the zero-shot SAM3 comparison is symmetric and the gains there are dramatic and plausible, the test-split video lists are explicit, and code is promised at a named GitHub repo. The downstream Gaussian-splatting/MPM section is illustrative only, but the paper does not lean any quantitative claim on it. Because the central claim is a conjunction (\"outperforms zero-shot SAM3 AND other mainstream baselines\"), a broken Medical SAM3 leg weakens but does not destroy it — hence CONDITIONAL rather than REJECT, matching the reader's verdict. The fix is cheap: one baseline re-run with correct prompts and/or matched fine-tuning.","tokens_in":8823,"tokens_out":1652,"duration_ms":31780,"concrete_test":"Run Medical SAM3 (2D) on CholecSeg8k test videos using its intended box-prompt interface, with prompts derived from ground-truth masks (one box per instance/class per frame), computing mIoU under the paper's own per-class protocol. Then optionally LoRA- or fully fine-tune Medical SAM3 on the paper's exact training split and re-evaluate. If box-prompted or surgically fine-tuned Medical SAM3 jumps from ~0-9% to >50% mIoU on liver/grasper/fat classes, the Table 1-2 comparison is an interface/training-regime artifact and the \"outperforms fully fine-tuned Medical SAM3\" claim must be dropped; if scores stay near zero, the baseline failure is genuine and the concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim hinges on Tables 1-2, where Medical SAM3 (2D/3D) scores 0.00-8.65 mIoU on major classes (Liver, Abdominal Wall, Grasper, Fat) while zero-shot SAM3 gets 92.57 on Liver and the authors' method gets 96.92. A fully fine-tuned medical foundation model scoring literally 0.00 on Abdominal Wall and ~5 on Liver — the two easiest, largest structures in laparoscopic cholecystectomy — is far more consistent with a prompt/interface or protocol mismatch than with genuine model failure. Two specific ambiguities compound this: (1) Medical SAM3 [7] is a prompt-driven model whose training interface is box/point exemplar prompts; the paper never states which prompt mode was used, and text-prompting a model trained on geometric prompts routinely produces near-empty outputs. (2) §3.2 explains Medical SAM3's failure as \"full fine-tuning across 33 multi-modal datasets dilutes its specialized surgical features,\" which describes the off-the-shelf pretrained checkpoint run zero-shot on surgical data — yet the abstract/§1 claim says the method \"outperforms... fully fine-tuned Medical SAM3 [7] under identical training settings.\" If Medical SAM3 was never fine-tuned on the surgical training split, the comparison is asymmetric: the authors' model saw the surgical training data, the baseline did not. Either way — wrong prompt interface or mismatched training regime — the \"consistently outperforms... mainstream baselines\" claim against the strongest medical competitor is not currently supported, and the qualitative Fig. 3 \"distorted contours\" narrative inherits the same artifact risk. The CAT-Seg and zero-shot SAM3 comparisons do look legitimate and the gains over zero-shot SAM3 (e.g., Grasper 30.26 → 90.52) are large enough to survive even a baseline correction; it is specifically the Medical SAM3 leg of the claim that is load-bearing and unsecured.","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper adapts SAM3 to surgical concept segmentation via Low-Rank Adaptation (LoRA). Low-rank adapters (r=16, α=32) are injected into the prompt encoder, DETR-style detector, and tracker while the vision backbone is frozen, giving 8.32M trainable parameters (0.98% of 849M) and ~9 GB peak training memory on a single RTX 3090. A unified concept vocabulary maps native class labels of CholecSeg8k, EndoVis18, and CaDISv2 to shared text prompts, enabling one universal LoRA weight across datasets. Per-class mIoU results on CholecSeg8k (Table 1) and EndoVis18 (Table 2) show large gains over zero-shot SAM3 and near-zero/single-digit scores for Medical SAM3 (2D/3D); a downstream Gaussian-splatting reconstruction plus MPM physics simulation pipeline is demonstrated qualitatively.","tokens_in":9179,"tokens_out":2448,"duration_ms":48902,"significance":"If the results hold, the contribution is a useful and practical one: a concrete, reproducible recipe (r=16, α=32, AdamW, bf16, 10 epochs, single RTX 3090 at ~9 GB peak memory) for adapting SAM3 to surgical concept segmentation with 0.98% trainable parameters, evaluated on three public surgical benchmarks, with a promised open-source release. The demonstration that a frozen vision backbone plus adapters on the prompt encoder/detector/tracker suffices for large per-class gains over zero-shot SAM3 is a meaningful negative-space result for the surgical PEFT literature, and the single universal LoRA weight across datasets is a practical selling point. The downstream reconstruction/simulation demonstration, while qualitative, illustrates a plausible deployment path. The work is incremental in method terms (standard LoRA applied to SAM3) but timely, and the efficiency claims are directly measurable and falsifiable.","major_comments":[{"comment":"Tables 1–2, §1 and Abstract: the headline claim that the method 'outperforms fully fine-tuned Medical SAM3 under identical training settings' is not supported by the experiments as described. Medical SAM3 scores 0.00 on Abdominal Wall and 4.99/8.65 on Liver (Table 1) — near-empty outputs on the two largest, easiest structures — while zero-shot SAM3 scores 92.57 on Liver. This pattern is the signature of a prompt-interface or protocol mismatch, not of a fine-tuned medical foundation model. Two specific ambiguities must be resolved: (a) §3.2 attributes Medical SAM3's failure to 'full fine-tuning across 33 multi-modal datasets diluting specialized surgical features', which describes the off-the-shelf checkpoint run zero-shot on surgical data — yet §1 claims comparison 'under identical training settings'. Was Medical SAM3 actually fine-tuned on the same surgical training split with the same ","section":"§3.2, Tables 1–2"},{"comment":"CaDISv2 is named as one of three evaluation benchmarks in the Abstract, §1 (contributions), and §3.1 (held-out videos 2, 12, 22), but the paper contains no quantitative table for it. §3.2 reports only a single aggregate number ('overall mIoU of 26.9% and mDice of 37.0%') with no per-class breakdown and no statement of which dataset this aggregate refers to; the only CaDISv2 evidence is the Fig. 4 radar chart, from which exact values cannot be read. A per-class results table for CaDISv2, matching Tables 1–2, is needed for the 'consistent across three benchmarks' claim.","section":"§3.2 / Fig. 4"},{"comment":"§1 states the method 'outperforms dataset-specific SurgTPGS', but SurgTPGS does not appear in Table 1, Table 2, or Fig. 4 — no comparison numbers are reported anywhere. Either include the comparison or remove the claim. Note also that SurgTPGS is a dataset-specific method while the proposed model uses one universal weight, so the comparison conditions should be stated explicitly.","section":"§1, Tables 1–2"},{"comment":"Tables 1–2 report single-run numbers with no error bars, multi-seed runs, or significance tests. Several claimed wins are modest (e.g., EndoVis18 Seq_9 instrument-clasper: 40.51 vs CAT-Seg 38.94; instrument-wrist: 39.18 vs 24.34 is larger, but kidney-parenchyma 73.53 vs CAT-Seg 71.98 is within typical seed noise for LoRA fine-tuning on small surgical splits). Given that the test sets comprise only 4, 2, and 3 videos respectively, at minimum multi-seed means ± std, or a paired per-frame test, is needed to support 'consistent' improvements. Additionally, no ablation on LoRA rank/insertion sites is provided even though §2.3 makes a specific architectural argument for adapting the prompt encoder, detector, and tracker while freezing the vision encoder; a small ablation (e.g., rank {4,16,64}, encoder-only vs decoder-only insertion) would substantiate that design choice.","section":"Tables 1–2, §2.3"}],"minor_comments":[{"comment":"Table 1 and Table 2 captions: 'Quantative' → 'Quantitative'; the captions state 'highlighted the first, second, and third' but do not say what visual convention (bold/underline/color) encodes each rank.","section":"Tables 1–2"},{"comment":"The Evaluation Metrics paragraph (§3.1) promises both mIoU and Dice, but Tables 1–2 report only mIoU; Dice appears only in the §3.2 aggregate sentence and Fig. 4. Report both metrics in the tables or state why mIoU alone is shown.","section":"§3.1, Tables 1–2"},{"comment":"§3.2, final paragraph: 'Zero-shot SAM 3' has a stray space; §2.3 'DETR decoder' is used while §2.2 calls the component a 'DETR-style detector' — unify terminology.","section":"§3.2, §2.3"},{"comment":"The 80 GB figure for full fine-tuning (§1, §2.3) is asserted without citation or measurement; 'over 80% reduction' from 80 GB to 9 GB is arithmetically ~89%. Please give the source of the 80 GB number (own measurement? which checkpoint/batch size?) or qualify it.","section":"§2.3"},{"comment":"The unified vocabulary V is central to the 'universal LoRA weight' contribution, but the actual label→text mapping is never shown. Please include the full mapping table (e.g., as a supplement), since cross-dataset homonym classes (e.g., 'grasper' vs 'instrument-clasper') are exactly where this design could silently degrade.","section":"§2.1"},{"comment":"Baselines CLIP and SurgVLP are not segmentation models per se; a sentence explaining how segmentation outputs were obtained from them (e.g., which segmentation head or decoding procedure) would make Table 1–2 reproducible.","section":"§3.2"},{"comment":"Fig. 4: axes cover 'a curated subset' of categories — state the selection criterion in the caption, and consider including per-dataset aggregate values so readers can cross-check against Tables 1–2.","section":"Fig. 4"},{"comment":"§3.3 (Application): the reconstruction/simulation pipeline is illustrative only. One sentence quantifying or at least describing the validation of the simulation output (beyond the qualitative Fig. 5) would temper the 'directly deployed' language in the Abstract, which currently overstates what is demonstrated.","section":"§3.3, Abstract"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an application-of-existing-techniques paper (LoRA on a newly released SAM3), and its novelty lies mainly in the engineering recipe and the demonstration on three surgical benchmarks rather than in method design. The most consequential evaluation problem — the Medical SAM3 baseline appearing to be run zero-shot rather than fine-tuned under the claimed \"identical training settings\" — is fixable but currently undermines the central comparative claim. I also note that several cited works (SurgTPGS, Endo-4DGS, EndoGSIM) share authors with this manuscript; the SurgTPGS comparison in particular is asserted but never quantified."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful bit here is practical, not theoretical: a concrete LoRA placement on SAM3 (prompt encoder + DETR detector + tracker, vision backbone frozen) that trains at ~9 GB / 0.98% params and lifts hard surgical classes a lot versus zero-shot SAM3. Grasper 30→90 mIoU on CholecSeg8k and the instrument lifts on EndoVis18 are large enough that I believe the adaptation works. The unified concept vocabulary and the single shared adapter across three procedures are sensible engineering. Code link is a plus. Downstream Gaussian-splatting + MPM is illustrative, not a causal mask-quality study—fine if framed that way.\n\nWhat is not new: LoRA on foundation models, medical SAM fine-tunes, and these benchmarks are all established. The contribution is the specific module choice plus the efficiency numbers, not a new problem class.\n\nThe soft spot that actually matters is the Medical SAM3 baseline. Tables 1–2 show near-zero mIoU on liver, abdominal wall, grasper, fat—structures that should not collapse to 0–8% if a medical foundation model is prompted and evaluated correctly. The text also talks like they ran the off-the-shelf Medical SAM3 checkpoint, while the abstract claims a win under “identical training settings.” That mismatch is load-bearing for the strongest claim. CAT-Seg and zero-shot SAM3 comparisons look fairer; strip or re-run Medical SAM3 and the paper still has a clean PEFT story. Minor: no multi-seed error bars, short training schedule, free hyperparameters (r, α, insertion sites) without ablation depth.\n\nWho it’s for: surgical CV / robotic simulation people who need a reproducible single-GPU adapter recipe. Not for someone hunting a conceptual breakthrough. Math is standard LoRA; data splits are stated; citations cover the right prior art. I’d send it to peer review as a methods note with a mandatory baseline cleanup. Engage if you care about deployable surgical SAM adapters; don’t cite the Medical SAM3 win until that’s fixed.","headline":"Solid single-GPU LoRA recipe with real gains over zero-shot SAM3; the “beats fully fine-tuned Medical SAM3” claim is the soft leg and needs fixing before anyone leans on it.","tokens_in":10360,"tokens_out":531,"would_cite":false,"duration_ms":17218,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Adapting under 1% of SAM3 with LoRA beats zero-shot and fully fine-tuned medical SAMs on surgical concept segmentation while fitting on a single consumer GPU.","keywords":["Surgical Concept Segmentation","Parameter-Efficient Fine-Tuning","Low-Rank Adaptation","SAM3","Prompt-Driven Segmentation","Surgical Robotics","Domain Adaptation"],"falsifier":"Re-run Medical SAM3 2D/3D on the same held-out CholecSeg8k and EndoVis18 frames with the paper’s exact text concepts and scoring script; if its mIoU on classes such as abdominal wall, grasper, and liver rises to match or beat the LoRA model, the claim of outperforming full medical fine-tunes fails.","tokens_in":9997,"feed_emoji":"🏥","tokens_out":917,"duration_ms":19372,"temperature":0.7,"pith_summary":"Surgical video has a domain gap from the natural images that foundation segmenters like SAM3 were trained on, so zero-shot accuracy collapses on instruments and tissues. Full fine-tuning of medical SAM variants is too heavy for ordinary GPUs. This paper shows that attaching small low-rank adapters only to SAM3’s prompt encoder, detector, and tracker—while freezing the vision backbone—updates 0.98% of parameters, peaks at about 9 GB of memory, and still produces higher per-class mIoU than zero-shot SAM3 and reported Medical SAM3 baselines across cholecystectomy, nephrectomy, and cataract datasets. One shared adapter weight works across those datasets via a unified text-concept vocabulary. The resulting class masks plug straight into Gaussian-splatting reconstruction and physics simulation for robotic surgery pipelines.","feed_headline":"Under 1% of SAM3 beats full medical fine-tunes on surgery","feed_subtitle":"LoRA on prompt, detector, and tracker hits 9 GB memory and higher mIoU across three surgical datasets.","key_machinery":"Low-Rank Adaptation (LoRA) injected only into the prompt encoder’s cross-modal layers, the detector’s attention projections, and the tracker’s attention layers, with the vision backbone fully frozen and a unified surgical concept vocabulary aligning text prompts across datasets.","core_discovery":"A LoRA-adapted SAM3 that trains only the prompt encoder, DETR-style detector, and tracker (8.32 M parameters, 0.98% of the model) while freezing the vision backbone consistently outperforms zero-shot SAM3 and fully fine-tuned Medical SAM3 variants on prompt-driven surgical concept segmentation across CholecSeg8k, EndoVis18, and CaDISv2, with peak training memory of 9 GB on one consumer GPU and masks usable downstream for reconstruction and physical simulation.","pith_inferences":["If the frozen backbone already carries usable surgical appearance features, similar LoRA-only adaptation may transfer to other narrow clinical domains (endoscopy, interventional radiology) with little extra memory.","The large reported gap versus Medical SAM3 invites a controlled re-evaluation protocol that isolates prompt format and 2D vs video input before treating full fine-tuning as categorically worse.","Extending the same adapters from still frames to full surgical video sequences, as the authors flag for future work, is the natural next stress test of tracker LoRA stability."],"forward_implications":["Surgical concept segmentation can be specialized on a single consumer GPU without full foundation-model fine-tuning.","One shared LoRA weight plus a unified concept vocabulary can serve multiple procedures without per-dataset retraining.","Class-wise masks from the adapted model can drive region-level material assignment in Gaussian-splatting reconstruction and MPM-style physics simulation.","Preserving the frozen vision backbone while adapting prompt, detect, and track paths is presented as enough to close the surgical domain gap for this task."],"fun_headline_variants":["LoRA adapts 0.98% of SAM3 to beat full medical fine-tunes on surgery","Freeze SAM3 backbone, tune prompt detector tracker for surgical concepts","8.32M LoRA params lift SAM3 past zero-shot and full Medical SAM3","Single-GPU LoRA SAM3 tops surgical mIoU on three concept datasets","Prompt-driven surgical masks from 1% SAM3 beat full-parameter medical SAMs"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That the near-floor scores reported for fully fine-tuned Medical SAM3 under the authors’ prompts and protocol are a fair baseline rather than a mismatch in prompting, input mode, or evaluation setup.","fun_headline_variants_meta":{"raw":{"variants":["LoRA adapts 0.98% of SAM3 to beat full medical fine-tunes on surgery","Freeze SAM3 backbone, tune prompt detector tracker for surgical concepts","8.32M LoRA params lift SAM3 past zero-shot and full Medical SAM3","Single-GPU LoRA SAM3 tops surgical mIoU on three concept datasets","Prompt-driven surgical masks from 1% SAM3 beat full-parameter medical SAMs"]},"model":"grok-4.5","effort":"low","cost_usd":0.004212,"raw_usage":{"total_tokens":1245,"prompt_tokens":760,"num_sources_used":0,"completion_tokens":96,"cost_in_usd_ticks":42124000,"prompt_tokens_details":{"text_tokens":760,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":389,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":760,"tokens_out":96,"duration_ms":6691,"temperature":1.0,"reasoning_tokens":389,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T15:40:30.888042+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-run Medical SAM3 2D/3D on the same held-out CholecSeg8k and EndoVis18 frames with the paper’s exact text concepts and scoring script; if its mIoU on classes such as abdominal wall, grasper, and liver rises to match or beat the LoRA model, the claim of outperforming full medical fine-tunes fails.","supporting_citations":[],"review_version":1}