{"id":"945fd39f-f450-4576-9e25-519ce533ecab","arxiv_id":"2507.19592","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"SurgPIS is a surgical part-aware instance segmentation model that predicts instrument instances and their parts together, and can learn from datasets labelled for only one of these tasks.","lead":"SurgPIS segments each surgical instrument as an instance and, at the same time, labels its parts such as shaft, wrist, and clasper in a single model. It can be trained on existing datasets that label only whole instruments or only parts, which lowers the annotation burden for robot-assisted surgery.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Weak-supervision stage is underspecified: L_sup_teach in Eq. (13) is never defined, and the pseudo-label filter in §III-C1 requires ground-truth part masks that do not exist for PSS/IIS-only data, so the central disjoint-dataset training claim cannot be checked as written.","rationale":"The paper does a credible job extending Mask2Former/TAPPS to surgical PIS and provides broad experiments. The ablation study supports the contribution of part-aware bipartite matching and part-specific queries, and the multi-dataset evaluation is extensive. I do not see internal evidence of a fatal design flaw in the fully supervised path. The soft part-map aggregation in Eq. (9) does carry a real risk—argmax over instrument classes could assign part masks to the wrong semantic part class, and this is unvalidated—but that is an accuracy risk that the reported PSS/IIS gains may already partly address. The more fundamental problem is that the weakly-supervised loss in Eq. (13), which is the mechanism for the paper's central claim of learning from disjoint PSS/IIS datasets, is never actually specified. The only related formula, the Dice filter in §III-C1, appears to require ground-truth part masks that do not exist in the weakly labelled sets; if reinterpreted as teacher self-Dice it is vacuous. L_sup_teach is absent entirely. Thus the headline 'first PIS model trained on disjoint IIS/PSS datasets' is not checkable from the paper. This keeps the reader's CONDITIONAL verdict: the work may be valid, but acceptance should be conditioned on defining/releasing the weakly supervised loss and the filter, and ideally on rerunning the key weakly supervised results. My concern overlaps only partly with the reader's weakest assumption: Eq. (9) is a secondary accuracy risk, while my identified concern is a specifiability/reproducibility gap in the same weak-supervision mechanism.","tokens_in":17247,"tokens_out":7550,"duration_ms":88676,"concrete_test":"Obtain the released repository at https://github.com/weimengmeng1999/SurgPIS (or ask the authors) and locate the implementation of L_sup_teach and the pseudo-label filter for PSS/IIS batches. Then implement Eq. (13) exactly as written on a SAR-RARP50 validation batch: if L_sup_teach has no associated code or formula, or the Dice filter is computed against labels that do not exist in the dataset, the weakly-supervised training protocol is underspecified and the reported 77.92 PartPQ cannot be attributed to it. If a working implementation is provided, rerun Table I's last row with that loss removed; a material drop would confirm the loss is load-bearing, while no drop would imply the missing definition is cosmetic.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central weak-supervision stage is not defined in a way that can be executed. In §III-C, the total weakly supervised loss Eq. (13) is written as Lwks = L_sup_teach + {...}; however, the student–teacher consistency loss L_sup_teach is never given a formula or a precise definition anywhere in the text. The only stated mechanism in §III-C1 is an EMA update and a 'Dice-based mask scoring filter' with threshold TreshDice, but the filter is written as Dice(ˆm_teach_{τi,k}, m_i,k)_{i≠0} > TreshDice, where m_i,k denotes ground-truth part masks. For DIIS/DPSS the whole premise is that part-level ground truth is absent; no m_i,k exists to compute this condition. If m_i,k is instead taken to be the teacher's own pseudo-label, then Dice(teacher, itself)=1, making the filter vacuous. Additionally, TreshDice is not reported. Because the claimed ability to train on disjoint PSS/IIS datasets is carried by this undefined term, Eq. (13) is not implementable as written and the central claim cannot be checked from the manuscript.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SurgPIS, a Mask2Former-based model that jointly predicts surgical instrument instances, their instrument classes, and per-instance part masks. The two claimed contributions are (i) a part-specific query transformation that derives part queries from instrument-level queries, and (ii) a two-stage weakly supervised training scheme that first trains on PIS-labelled data and then additionally trains on disjoint datasets labelled only for IIS or PSS, using mask aggregation and a student-teacher consistency mechanism. Results are reported on EndoVis2018, EndoVis2017, SAR-RARP50, and a PIS-annotated GraSP test set, with PartPQ, PQ, and PartIoU gains over a Mask2Former-based baseline (BPSS⊕BIIS) and over TAPPS, together with ablations of the query transformation, bipartite matching, pseudo-label filtering, and augmentation strategies. The paper claims to be the first to address part-aware instance segmentation for surgical instruments, and its main empirical claim is that SurgPIS outperforms prior single-task and part-aware panoptic models across PIS, IIS, ISS, and PSS tasks.","tokens_in":17558,"tokens_out":4427,"duration_ms":53494,"significance":"The problem of learning from datasets with disjoint annotation granularities is important for surgical vision, where no large PIS dataset exists, and the paper is among the first to tackle it in this domain. The empirical evaluation is broad: four datasets, multiple backbones, cross-dataset generalization on GraSP, and ablations of the two main architectural components. If the weak-supervision mechanism were fully specified and reproducible, the contribution would be valuable and the reported PartPQ improvements over the strong Mask2Former-based baseline (11.07–14.49 pp) would be noteworthy. However, the central weak-supervision loss in Eq. (13) is not implementable as written, and the pseudo-label filter appears to require ground-truth part masks that do not exist for the weakly labelled datasets. Because this mechanism carries the paper's main claim of learning from disjoint PSS/IIS data, the current manuscript cannot be independently checked or reproduced despite the otherwise strong empirical scope.","major_comments":[{"comment":"The total weakly supervised loss Lwks is written as Lwks = L_sup_teach + {Lwks_pss or Lwks_iis}, but L_sup_teach is never defined anywhere in the text. Section III-C1 only describes the EMA update in Eq. (7) and a Dice-based filtering criterion. Without a precise formula for L_sup_teach, including how teacher PIS masks are matched to student predictions and which losses are applied, the weak-supervision objective is not executable, and the central claim that SurgPIS can be trained on disjoint PSS/IIS datasets cannot be checked from the manuscript. Please add the missing definition and all associated matching and normalization details.","section":"III-C, Eq. (13)"},{"comment":"The filtering condition is written as Dice(ˆm_teach_{τi,k}, m_{i,k})_{i≠0} > Tresh_Dice, where m_{i,k} in Eq. (1) denotes ground-truth part masks. For DPSS and DIIS, those ground-truth part masks do not exist by construction; if m_{i,k} is instead taken to be the teacher's own pseudo-label, then Dice(teacher, itself)=1, making the filter vacuous. The threshold Tresh_Dice is also not reported. Please clarify what m_{i,k} is in the weakly supervised setting and how the filter is actually computed, and report the threshold value.","section":"III-C1, pseudo-label filtering"},{"comment":"The implementation details state that the EMA decay rate is set to α=0.99, while the sensitivity analysis in Fig. 7 shows that a decay rate around 0.995 consistently yields the best performance on all evaluated metrics. This is an internal contradiction. Either adopt the value suggested by the authors' own sensitivity analysis or explain why α=0.99 is retained; as written, the final model uses a hyperparameter that the paper itself identifies as suboptimal.","section":"IV-B and Fig. 7"},{"comment":"The construction of the soft part-level semantic map ρk relies on γ̂_j = arg max over predicted instrument classes. If the instrument-class prediction is wrong, the predicted part masks are accumulated under the wrong part semantic class, and Lwks_pss in Eq. (11) is then computed against corrupted targets. This error path is load-bearing for the claimed benefit of PSS-only weak supervision, but it is not quantitatively analyzed or ablated. Please provide evidence on the accuracy of γ̂_j on weakly labelled data, or a mechanism that limits the damage caused by instrument-class misclassification.","section":"III-C2, Eq. (9)"}],"minor_comments":[{"comment":"There are two tables labeled 'Table II' in the manuscript: one for ISS comparison and one for the GraSP evaluation. The later table should be renumbered.","section":"IV-E4, Tables"},{"comment":"No error bars or confidence intervals are reported for any of the quantitative results. Given the small test sets and stochastic training, please report variance across multiple runs or at least provide the number of runs.","section":"All tables"},{"comment":"The title refers to 'PSS and ISS datasets,' but the formal notation and Eq. (13) use IIS (instance-level instance segmentation) for the instrument-only labels. Please align the terminology.","section":"Section III-C title and III-C3"},{"comment":"The threshold is written inconsistently as 'TreshDice' and 'Tresh_Dice' across the text and figures. Please use a single notation throughout.","section":"III-C1, notation"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and reports large empirical gains, but the weak-supervision stage, which is the key novelty, is under-specified to the point of being non-implementable as written. The missing definition of L_sup_teach and the ill-defined pseudo-label filter are load-bearing issues that must be resolved before the claims can be verified. The internal inconsistency between the reported EMA decay rate and the sensitivity analysis should also be corrected. I do not see grounds for rejection if these issues are fixable within the manuscript's scope, but the current version is not ready for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First: this is a serious paper with a real core contribution. SurgPIS is the first part-aware instance segmentation model for surgical instruments, and the two novelties — part-specific query transformation and the disjoint-label weak-supervision scheme — are both well-motivated. The experiments are broad: four datasets, multiple backbones, ablations for both architectural changes, cross-dataset generalisation to GraSP, and large gains over the proposed Mask2Former baseline (11–14 pp PartPQ). The ablation shows the part-aware bipartite matching is worth roughly 8 pp and the part-specific queries are worth roughly 35 pp, which is strong evidence the architecture carries the weight. That part of the paper is in good shape.\n\nThe weak-supervision stage is not. Equation (13) defines the total weakly supervised loss as L_sup_teach plus a partial-label loss, but L_sup_teach is never given a formula. Section III-C1 says the teacher generates pseudo-ground-truth PIS labels, and then gives a Dice-based filter, but the filter is written as Dice(ˆm_teach_{τi,k}, m_i,k) where m_i,k is the ground-truth part mask. For DIIS/DPSS there is no ground-truth part mask; if you take m_i,k to be the teacher's own pseudo-label, the Dice score is identically 1 and the filter is vacuous. The threshold TreshDice is not reported. This is not a stylistic lapse: the claimed ability to train on disjoint PSS/IIS datasets depends on this term. The stress-test concern is right, and it is the main reason I cannot call the paper clean.\n\nOther soft spots are minor by comparison. No error bars, and the code availability statement contradicts itself (github link vs 'will be made open access upon publication'). The EMA sensitivity analysis in Fig. 7 peaks around 0.995 while the chosen α is 0.99 — not necessarily wrong, but it should be acknowledged. The argmax assignment in Eq. (9) — assigning each predicted part mask to the part class of the most probable instrument class — is load-bearing for PSS-only training and is not explicitly validated. None of these by themselves sink the paper.\n\nWho is this for? Anyone working in surgical vision or part-aware segmentation. It deserves a serious referee; the fully supervised results are strong and the weak-supervision idea is worth pursuing. But the authors need to define L_sup_teach, rewrite the filter so it is implementable without ground-truth parts, and reconcile the code availability statement. Send it to review, but make it clear the current version is not reproducible.","headline":"Solid architecture and results, but the weak-supervision loss is undefined as written, so the central claim needs a major revision before it can be checked.","tokens_in":18069,"tokens_out":3100,"would_cite":true,"duration_ms":34538,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SurgPIS is the first part-aware instance segmentation model for surgical instruments, predicting each instrument's identity and its parts together.","keywords":["part-aware instance segmentation","surgical instrument segmentation","weakly-supervised learning","teacher-student consistency","mask classification","transformer queries","robot-assisted surgery","EndoVis"],"falsifier":"Train SurgPIS on EndoVis2018 PIS labels plus SAR-RARP50 PSS labels twice: once with Eq. (9) as written, and once with $\\rho_k$ built from ground-truth instrument classes (or from a classifier known to be correct), then compare PartPQ on EndoVis2018. If the oracle-assisted version is materially better, the argmax assumption is load-bearing; if the numbers are equal, the weak-supervision path is robust to instrument-class errors. A simpler observation: on a PSS-only image where the model visibly mislabels an instrument's class, check whether the part-semantic loss $L_{\\mathrm{pss}}^{\\mathrm{wks}}$ rises even though the part masks are correct.","tokens_in":17024,"feed_emoji":"🩺","tokens_out":7450,"duration_ms":75515,"temperature":0.7,"pith_summary":"SurgPIS is proposed as the first part-aware instance segmentation (PIS) model for surgical instruments: it outputs, for every instrument, its class, its instance mask, and the masks of its parts (shaft, wrist, clasper) in a single forward pass. The paper's central claim is that these three levels should be learned together, and that a single transformer-based model can do so without requiring a dataset that has all three annotations. To get around the absence of large PIS-labelled surgical datasets, SurgPIS is trained on disjoint datasets that carry only instance-level labels or only part-level labels, using a teacher-student strategy plus mask aggregation to turn its predictions into the available label type. If the claims hold, one model replaces separate instrument-instance, instrument-semantic, and part-semantic segmenters, and it reaches state-of-the-art PartPQ of 77.92 on EndoVis2018 with a Swin-B backbone.","feed_headline":"One model segments surgical tools, their instances, and their parts","feed_subtitle":"It uses part-only and instance-only labels to reach top part-aware accuracy on surgical benchmarks.","key_machinery":"The load-bearing mechanism is the part-specific query transformation: an MLP maps each of the $N_q$ instrument-level queries $Q$ to $C_{\\mathrm{part}}$ part-specific queries $Q_{\\mathrm{part}} \\in \\mathbb{R}^{(C_{\\mathrm{part}}\\times N_q)\\times C_\\epsilon}$, which are multiplied with the high-resolution feature map and passed through a sigmoid to produce the part masks $\\hat m_{j,k}$. This makes every part mask inherit the instrument instance's identity. Two supporting mechanisms carry the weakly supervised stage: part-aware bipartite matching adds a part-level mask loss $L_{\\mathrm{pm}}$ to the matching cost, and part-semantic mask aggregation builds soft part-level maps $\\rho_k$ by weighting each predicted part mask by the probability of its argmax instrument class (Eq. 9), then renormalizing pixel-wise to a probability distribution. The teacher-student loop with EMA and Dice-based pseudo-label filtering supplies consistent PIS pseudo-labels when only partial annotations exist.","core_discovery":"The paper introduces SurgPIS, built by extending Mask2Former with a part-specific query transformation: after instrument-level object queries are processed by a transformer decoder, an MLP transforms each of them into a fixed set of part-specific queries, so each predicted part mask is explicitly attached to its parent instrument instance. Training with full PIS labels uses a part-aware bipartite matching whose cost matrix includes a part-level mask loss, improving assignment. For weakly labelled data, the student model's PIS predictions are aggregated into part-semantic maps (for PSS-only data) or instrument-instance maps (for IIS-only data), and losses are computed against those partial labels, while an EMA teacher generates pseudo PIS labels filtered by a Dice-confidence threshold. On EndoVis2018 with a Swin-B backbone the model achieves 77.92 PartPQ and outperforms the Mask2Former-based BPSS⊕BIIS baseline by 11.07 percentage points in the fully supervised setting and 14.49 in the weakly supervised combined-dataset setting; similar margins hold on EndoVis2017.","pith_inferences":["Replacing the argmax over predicted instrument classes in Eq. (9) with oracle classes would likely produce cleaner soft part maps and stronger PSS weak supervision; the paper does not ablate this, so the reported gains may understate the aggregation's ceiling.","The part-specific query transformation should transfer to other domains where distinct object categories share identical part types and look alike, such as industrial or endoscopic tools, where the paper shows natural-image PIS models fail.","The paper reports a drop when PSS-only data is removed, but does not separate the effect of extra data from the effect of the teacher-student consistency loss, leaving the marginal contribution of each weakly labelled dataset as an open question."],"forward_implications":["A single SurgPIS model can output PIS, IIS, PSS, and ISS predictions by aggregating its predicted masks, replacing separate task-specific models.","Combining PSS-only and IIS-only datasets during the weakly supervised stage improves PartPQ beyond training on PIS labels alone.","On EndoVis2018 with a Swin-B backbone, SurgPIS reaches 77.92 PartPQ, outperforming the Mask2Former-based BPSS⊕BIIS baseline by 11.07 pp fully supervised and 14.49 pp weakly supervised; similar margins hold on EndoVis2017.","With a ResNet-50 backbone, SurgPIS runs at roughly 30 FPS on 512×512 inputs, making real-time use plausible.","On the unseen GraSP test set, it keeps competitive instrument-level PQ and exceeds the PIS baselines, indicating cross-dataset generalization."],"supporting_citations":[{"why":"Base architecture (Mask2Former) that SurgPIS extends with part-specific query transformation and part-aware matching.","marker":"[21]"},{"why":"TAPPS shared-query joint object-part representation; SurgPIS adopts this design and is compared against it as a natural-image PIS baseline.","marker":"[20]"},{"why":"EndoVis2018 challenge data, the main PIS-labelled training and test set used for the headline PartPQ results.","marker":"[2]"},{"why":"EndoVis2017 data, whose PSS labels and manually created IIS annotations support both the weak-supervision and IIS/PSS evaluations.","marker":"[12]"},{"why":"SAR-RARP50 provides the PSS-only dataset used to test weak supervision from part-level labels.","marker":"[14]"},{"why":"ISINet supplied instance-level annotations for EndoVis2018 and serves as a state-of-the-art IIS baseline.","marker":"[15]"},{"why":"S3Net also supplies instance-level annotations and is a comparison point for IIS and ISS.","marker":"[17]"},{"why":"Source of the Dice-based pseudo-label filtering used to keep only confident teacher masks.","marker":"[48]"},{"why":"Source of the EMA teacher-student consistency mechanism used in weakly supervised training.","marker":"[49]"}],"fun_headline_variants":["SurgPIS unifies surgical instrument instance and part segmentation","Weakly supervised part-aware segmentation for surgery","Unified part-aware segmentation for surgical instruments","SurgPIS: weak supervision links parts to surgical instruments","Part-aware instance segmentation for surgical tools from partial labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"When training on part-level-only data, the method assumes that the argmax of the predicted instrument-class probability for each predicted part mask is the correct instrument class; if that class is wrong, the part masks are accumulated under the wrong part label and the weakly supervised part loss is computed against corrupted targets.","fun_headline_variants_meta":{"raw":{"variants":["SurgPIS unifies surgical instrument instance and part segmentation","Weakly supervised part-aware segmentation for surgery","Unified part-aware segmentation for surgical instruments","SurgPIS: weak supervision links parts to surgical instruments","Part-aware instance segmentation for surgical tools from partial labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000663,"raw_usage":{"total_tokens":3052,"prompt_tokens":993,"completion_tokens":2059,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":609,"completion_tokens_details":{"reasoning_tokens":1983}},"tokens_in":609,"tokens_out":2059,"duration_ms":16687,"temperature":1.0,"reasoning_tokens":1983,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:14:42.910288+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SurgPIS on EndoVis2018 PIS labels plus SAR-RARP50 PSS labels twice: once with Eq. (9) as written, and once with $\\rho_k$ built from ground-truth instrument classes (or from a classifier known to be correct), then compare PartPQ on EndoVis2018. If the oracle-assisted version is materially better, the argmax assumption is load-bearing; if the numbers are equal, the weak-supervision path is robust to instrument-class errors. A simpler observation: on a PSS-only image where the model visibly mislabels an instrument's class, check whether the part-semantic loss $L_{\\mathrm{pss}}^{\\mathrm{wks}}$ rises even though the part masks are correct.","supporting_citations":[{"cited_title":"Task-aligned part-aware panoptic seg- mentation through joint object-part representations,","cited_arxiv_id":null,"evidence_quote":"TAPPS shared-query joint object-part representation; SurgPIS adopts this design and is compared against it as a natural-image PIS baseline."},{"cited_title":"2018 robotic scene segmentation challenge,","cited_arxiv_id":null,"evidence_quote":"EndoVis2018 challenge data, the main PIS-labelled training and test set used for the headline PartPQ results."},{"cited_title":"2017 robotic instrument segmentation challenge,","cited_arxiv_id":null,"evidence_quote":"EndoVis2017 data, whose PSS labels and manually created IIS annotations support both the weak-supervision and IIS/PSS evaluations."},{"cited_title":"ISINet: an instance- based approach for surgical instrument segmentation,","cited_arxiv_id":null,"evidence_quote":"ISINet supplied instance-level annotations for EndoVis2018 and serves as a state-of-the-art IIS baseline."},{"cited_title":"From forks to forceps: A new framework for instance segmentation of surgical instruments,","cited_arxiv_id":null,"evidence_quote":"S3Net also supplies instance-level annotations and is a comparison point for IIS and ISS."},{"cited_title":"Polite teacher: Semi-supervised instance segmentation with mutual learning and pseudo-label thresholding,","cited_arxiv_id":null,"evidence_quote":"Source of the Dice-based pseudo-label filtering used to keep only confident teacher masks."}],"review_version":1}