{"id":"8bff8240-b69a-4af0-a8c7-ee2b20346778","arxiv_id":"2507.04304","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A dual SegFormer pipeline with confidence-based fusion achieves 0.80 mIoU on EndoVis2018 holistic segmentation but lags prompt-based models on EndoVis2017.","lead":"Surg-SegFormer uses two SegFormer models, one for anatomy and one for tools, and fuses their outputs for surgical scene segmentation. It reports a mean IoU of 0.80 on EndoVis2018 and 0.54 on EndoVis2017, but its claim of beating all state-of-the-art is only partially supported.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The EndoVis2018 Task 1 SOTA claim rests on unmatched evaluation: class sets differ between the seven-label definition in §IV and the ten columns in Table IV, and baselines from [5] are used without evidence of an identical split; the 0.80 vs 0.65 gap may be protocol artifact.","rationale":"The reader's weakest assumption correctly identified that data splits and evaluation protocols may differ between Surg-SegFormer and the baselines. My stress-test sharpens this into a specific, checkable mismatch: the paper defines a seven-label protocol in Section IV but reports ten class-wise mIoUs in Table IV, so the 0.80 headline mIoU may not be computed over the same classes as the AdaptiveSAM and MedT numbers taken from [5]. This is not a matter of outside consensus; it is an internal inconsistency in the paper's own protocol description. The architecture and loss design are reasonable, and the reported gains could be real, but the evidence as presented does not rule out the protocol-artifact explanation. The paper also overstates the general SOTA claim: in Table II, Surg-SegFormer trails AdaptiveSAM and S3Net on EndoVis2017 (0.54 vs 0.72) and SurgicalSAM on EndoVis2018 Type 2018 (0.64 vs 0.80). However, the precise EndoVis2018 Task 1 claim is the load-bearing result, and the class-set mismatch is the most direct threat to it. I did not find a deeper flaw in the model construction itself, and the proposed matched-protocol check would settle the issue. Since the reader's conditional verdict already accounts for this risk, no verdict change is needed.","tokens_in":9247,"tokens_out":4654,"duration_ms":53216,"concrete_test":"Obtain the exact EndoVis2018 Task 1 split and label mapping used by AdaptiveSAM/MedT in [5]; evaluate Surg-SegFormer and AdaptiveSAM on the same frames under the same class protocol, then recompute Table II. If the mIoU gap (0.80 vs 0.65) does not survive the matched protocol, the SOTA claim is a comparison artifact. Additionally, recompute Surg-SegFormer's mIoU using only the seven labels defined in Section IV; if that value differs materially from 0.80, Table IV averaged a different class set than the baselines.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Surg-SegFormer sets a new state of the art on EndoVis2018 Task 1 (0.80 mIoU / 0.89 Dice vs AdaptiveSAM 0.65/0.69, Table II). That comparison is valid only if both models are evaluated on the same label set and the same data split. The paper never reports the train/test split, and its class definitions are self-contradictory: Section IV defines seven merged labels (Background Tissue, RI, KP, CK, SI, SN, UP), but Table IV reports ten class-wise columns (adding ST, Clamps, Suction). Thus the 0.80 mIoU is apparently averaged over a different class set than the numbers taken from reference [5]. Because mIoU is a per-class average, including or excluding classes materially changes the score; the reported 15-point margin over AdaptiveSAM could be an artifact of protocol mismatch. The EndoVis2017 comparison has the same weakness: baseline values from [16] and re-trained baselines are mixed without specifying identical training frames or prompt settings. This is not a disagreement with consensus; it is an internal inconsistency in the evaluation protocol that directly affects the headline SOTA claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Surg-SegFormer, a prompt-free surgical scene segmentation model consisting of two SegFormer branches: SegAnatomy (B2 encoder) for anatomical structures and SegTool (B5 encoder with a custom skip-connected decoder) for articulated instruments. The branch outputs are combined by a confidence-based conditional fusion rule, and training uses a combined Tversky plus cross-entropy loss with α=0.7, β=0.3. The model is evaluated on EndoVis2017 and EndoVis2018, with reported overall mIoU/Dice of 0.54/0.56 on EndoVis2017 and 0.80/0.89 on EndoVis2018 Task 1. The paper claims that this sets a new state of the art on EndoVis2018 Task 1, surpassing MedT and AdaptiveSAM, while remaining competitive on instrument-type segmentation. The contributions claimed are the dual-model framework, the priority-weighted conditional fusion strategy, and the benchmark evaluation.","tokens_in":9555,"tokens_out":4794,"duration_ms":52069,"significance":"If the reported results are reproducible under a consistent evaluation protocol, Surg-SegFormer would be a useful prompt-free alternative to interactive SAM-based methods for holistic surgical scene segmentation, addressing a real clinical need in intraoperative and post-operative analysis. The architectural choices are reasonable: specializing separate encoders for anatomy and tools, adding skip connections to retain fine spatial detail, and combining Tversky and cross-entropy losses to handle class imbalance are all sensible design decisions. The paper evaluates on two public benchmarks, reports class-wise tables, and includes a loss-function ablation, which is commendable. However, the significance is currently undermined by load-bearing evaluation inconsistencies: the abstract's unqualified SOTA claim is contradicted by the paper's own EndoVis2017 numbers, the EndoVis2018 class list is inconsistent (seven labels defined in Section IV versus ten columns in Table IV), the train/test split is never specified, and the ablation in Section IV-D reports an mIoU of 85.70% that cannot be reconciled with the 0.80 mIoU reported in Table II.","major_comments":[{"comment":"The abstract's unqualified statement that Surg-SegFormer 'outperforms current state-of-the-art techniques' is contradicted by Table II: on EndoVis2017, Surg-SegFormer achieves 0.54 mIoU, below S3Net (0.72), MATIS (0.63), and AdaptiveSAM (0.72). The paper itself acknowledges this gap in Section IV-C. Since both the abstract and the conclusion claim consistent superiority over prompt-based and other SOTA methods, the claim must be restricted to the specific task where it holds (e.g., EndoVis2018 Task 1) or replaced with a carefully qualified comparison.","section":"Abstract; Section IV-A; Table II"},{"comment":"Table IV reports ten class-wise columns (BT, RI, KP, CK, SI, ST, Clamps, Suction, UP), while Section IV defines seven merged labels: Background Tissue, RI, Kidney Parenchyma, Covered Kidney, SI, SN, and UP. The discrepancy is not explained: are ST, Clamps, and Suction additional classes, and is the 0.80 mIoU in Table II computed over seven or ten classes? Because mIoU is a per-class average, including or excluding classes materially changes the score; without a single consistent label set, the comparison with MedT and AdaptiveSAM in Table II is not a like-for-like comparison.","section":"Section IV; Table IV"},{"comment":"The paper never specifies the train/validation/test split, the number of training frames, or the evaluation protocol for EndoVis2017 and EndoVis2018. It also mixes re-trained baselines (marked with *) with numbers taken from the literature ([5] and [16]) without establishing that all methods used identical splits, class merges, resolutions, and post-processing. Since the central SOTA claim on EndoVis2018 Task 1 rests on a 15-point mIoU margin over AdaptiveSAM, the reader cannot exclude protocol mismatch as the source of the gain. The authors should report the exact split and per-class protocol and provide results for all baselines under the identical protocol.","section":"Section IV-A; Section IV-B; Tables II-V"},{"comment":"The ablation study reports that the combined loss achieves mIoU 85.70% and Dice 89.21%, which cannot be reconciled with the EndoVis2018 Task 1 results of 0.80 mIoU and 0.89 Dice in Table II, nor with any number elsewhere in the paper. The ablation does not state which dataset, split, or class set it uses, and the percentage-scale values suggest a different evaluation protocol. This internal inconsistency must be resolved before the ablation can support the loss-function claim.","section":"Section IV-D; Table VI"},{"comment":"The fusion strategy is described as an OR operation, but Equation (1) is a conditional selection: it chooses the instrument prediction when Pinst > Panat or when the anatomy mask is zero, and the anatomy prediction otherwise. This is not an OR operation, and no priority weights appear in the equation despite the phrase 'priority-weighted conditional fusion.' Moreover, Section IV-D does not include an ablation of the fusion rule, so the contribution of this central design element to the reported performance is not demonstrated.","section":"Section III-B1; Equation (1)"}],"minor_comments":[{"comment":"The equations are misnumbered: the fusion rule in Section III-B1 is labeled Eq. (1), and then the Tversky index, cross-entropy, and combined loss are labeled 'Equation 1', 'Equation 2', and 'Equation 3', respectively. Please renumber them consistently.","section":"Section III-C"},{"comment":"The symbol α is used both as the Tversky false-positive weight (with β=0.3) and as the mixing coefficient in the combined loss in Eq. (4), which creates ambiguity. Please rename one of these parameters.","section":"Section III-C"},{"comment":"Table III reports SegFormer* mIoU values of 0.00 for Bipolar Forceps and Prograsp Forceps but 0.003 for Large Needle Driver; the inconsistent precision and the exact zeros should be checked for numerical or reporting errors.","section":"Section IV-A; Table III"},{"comment":"The post-processing morphological operations mentioned in the fusion description are not specified; kernel shapes and sizes should be given for reproducibility.","section":"Section III-B1"},{"comment":"The text 'Figure shows how the final output is fused' lacks a figure number and should refer to a specific figure.","section":"Section III-B"},{"comment":"The claim that Surg-SegFormer is 'lightweight' is not supported by any parameter count or computational cost measurement. Please add parameter counts or FLOPs for the two branches and the fused model.","section":"Section III-A; Section V"}],"recommendation":"major_revision","confidential_remarks":"The evaluation inconsistencies are serious enough that the SOTA claim cannot currently be accepted. In particular, the 7-vs-10 class discrepancy, the missing split details, and the 85.70% vs 0.80 mIoU mismatch between the ablation and the main table are likely to be flagged by any careful reviewer. The authors should be asked to provide a revised manuscript with a single consistent label set, full protocol details, and aligned metric scales. If the EndoVis2018 Task 1 result survives a matched-protocol comparison, the paper may make a useful contribution; otherwise, the claims should be downgraded accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the engineering is sensible, the evaluation is not. The idea of running two SegFormer branches—one for anatomy, one for tools—and fusing with a priority rule is a legitimate way to get prompt-free holistic segmentation, and it is genuinely useful for post-op video analysis. No circularity, no invented entities; the components are standard but the combination seems new. Credit where due: they report a real ablation on losses and architecture variants, and they are upfront that promptable models still beat them on EndoVis2017.\n\nThat last point is also the first crack. The abstract says 'outperforms current state-of-the-art techniques,' but the paper's own Table II has AdaptiveSAM at 0.72 mIoU on EndoVis2017 versus their 0.54. So the SOTA claim is at best dataset-specific, and the abstract overstates it.\n\nMore serious: the EndoVis2018 Task 1 comparison may not be apples-to-apples. Section IV defines seven merged labels, but Table IV reports ten class columns. mIoU is a per-class average; adding or removing classes changes the number. If the baselines from [5] were evaluated on a different label set, the 0.80 vs 0.65 gap could be protocol artifact. The paper never specifies the train/test split, and it mixes re-trained baselines with numbers pulled from the original papers. That is a load-bearing gap, not cosmetic.\n\nThe ablation table also bothers me. Table VI reports combined loss at 85.70 mIoU, which is far above the 0.80 in Table II, with no explanation of what dataset/split produced it. That kind of unexplained number suggests the metric is not consistently defined across the paper.\n\nBottom line: the architecture is worth a look, and the prompt-free direction has practical value, but the evidence as presented does not substantiate the headline result. I'd send it to peer review rather than desk reject—the idea is salvageable and the community would benefit from a corrected comparison—but I'd require the authors to align class sets, state the exact evaluation protocol, release code/data, and soften the SOTA language. As is, I wouldn't cite the numbers.","headline":"Sensible dual-SegFormer fusion for surgical scene segmentation, but the evaluation protocol is too inconsistent to support the SOTA claim.","tokens_in":10075,"tokens_out":2692,"would_cite":false,"duration_ms":29672,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper reports a prompt-free dual-transformer model, Surg-SegFormer, that reaches 0.80 mIoU and 0.89 Dice on EndoVis2018 holistic scene segmentation, surpassing MedT and AdaptiveSAM.","keywords":["surgical scene segmentation","semantic segmentation","transformer","SegFormer","robot-assisted surgery","EndoVis","prompt-free segmentation","model fusion"],"falsifier":"Re-run Surg-SegFormer and the reference baselines on identical EndoVis2018 Task 1 folds, using the same merged seven labels and the same number of training frames, and compare per-class mIoU; if the reported 16-point gap over MedT shrinks to noise under a shared protocol, the state-of-the-art claim would not be supported. A second check is to count how many test frames actually contain a suturing needle and verify whether the 0.98 needle IoU depends on class prevalence.","tokens_in":9077,"feed_emoji":"🩺","tokens_out":4056,"duration_ms":38358,"temperature":0.7,"pith_summary":"Surg-SegFormer is a prompt-free dual-transformer model for segmenting whole robotic surgical scenes, covering anatomical tissues, articulated tools, and fine structures like suturing needles in one pass. The paper claims it achieves 0.80 mIoU and 0.89 Dice on the EndoVis2018 holistic scene segmentation task, outperforming MedT and AdaptiveSAM, and 0.54 mIoU on the EndoVis2017 instrument segmentation benchmark, beating classical U-Net and a single SegFormer backbone. The motivation is practical: prompt-driven models such as SAM require per-frame human input, which is infeasible for surgical videos that often exceed an hour. If the reported results hold, Surg-SegFormer could give residents and postoperative reviewers an automated, prompt-free map of go- and no-go zones in long procedures.","feed_headline":"Dual SegFormer hits 0.80 mIoU on surgical scenes","feed_subtitle":"Prompt-free twin transformer beats MedT and AdaptiveSAM on EndoVis2018 without user prompts.","key_machinery":"The central object is the dual-instance pipeline built from SegFormer. SegAnatomy is the SegFormer-B2 variant fine-tuned for anatomical structures, the best of six variants on anatomy per the paper's variant table. SegTool uses the SegFormer-B5 encoder paired with a custom lightweight decoder that projects each encoder stage to a uniform channel count, upsamples all features to full resolution, concatenates them for multi-scale fusion, and applies dense skip connections to retain spatial detail for small objects like tool tips. The two outputs are merged by a priority-weighted conditional fusion rule: at each pixel, the tool prediction wins if its confidence exceeds the anatomy prediction or if the anatomy prediction is empty, otherwise the anatomy prediction wins. Training uses a combined Tversky plus cross-entropy loss with alpha 0.7 and beta 0.3 to suppress false negatives on small, delicate classes.","core_discovery":"The paper's central claim is that a dual-branch extension of SegFormer, with one branch fine-tuned for anatomy and another for tools, plus a confidence-weighted fusion of their outputs, yields state-of-the-art holistic surgical scene segmentation. On EndoVis2018 Task 1 (merged seven-label anatomy-plus-instrument-parts), the model reports 0.80 mIoU and 0.89 Dice, sixteen points of mIoU above MedT, and it is best in eight of ten classes, including near-perfect 0.98 mIoU on the extremely thin suturing needle class. The paper also reports 0.54 mIoU and 0.56 Dice on the seven-instrument EndoVis2017 task, which beats U-Net and the retrained SegFormer backbone but trails prompt-tuned leaders such as S3Net and AdaptiveSAM at 0.72; on EndoVis2018 instrument-type segmentation the model is competitive at 0.64 mIoU but below SurgicalSAM's 0.80. Thus the paper's state-of-the-art claim is specifically for holistic anatomy-and-tool segmentation, not for instrument-only tasks.","pith_inferences":["The dual-branch recipe, with one expert for tiny metallic objects and another for deformable tissue, could likely transfer to other video segmentation problems where object classes differ sharply in scale and texture, but the paper does not test such transfer.","Because the comparison mixes the paper's own retrained baselines with numbers taken from other papers, and the paper does not disclose its train/validation split, the true margin over MedT and AdaptiveSAM could be smaller, or larger, under a fully shared protocol.","The paper reports only per-frame metrics; a natural extension would be temporal consistency, since segmentation flicker across consecutive frames is exactly what would matter for the long-video tutoring scenario the paper motivates."],"forward_implications":["If the EndoVis2018 Task 1 numbers hold, holistic anatomy-plus-tool segmentation can be done in a single prompt-free pass, making automated post-hoc analysis of hour-long surgical videos feasible.","Thin, rare structures such as suturing needles can be segmented reliably (0.98 mIoU) without any user prompt, a class on which several baselines score near zero.","The confidence-weighted OR fusion is a computationally cheap mechanism for resolving overlaps between tool and anatomy masks in crowded frames.","The same architecture is claimed to generalize across anatomy-rich and instrument-only benchmarks, though its margin is largest on the anatomy-rich task."],"supporting_citations":[{"why":"Supplies the SegFormer encoder-decoder architecture that both detection branches are built from.","marker":"[9]"},{"why":"Provides the EndoVis2018 benchmark with anatomy and instrument-part labels where the main 0.80 mIoU result is measured.","marker":"[18]"},{"why":"Provides the EndoVis2017 instrument-only benchmark for the second evaluation.","marker":"[17]"},{"why":"Supplies the AdaptiveSAM, MedT, TransUNet, and SAM-ZS baselines and their EndoVis2018 and EndoVis2017 scores that Surg-SegFormer is claimed to surpass.","marker":"[5]"},{"why":"Supplies the SurgicalSAM, S3Net, MATIS, ISI-Net, and TernausNet comparison numbers used in the instrument-type tables.","marker":"[16]"},{"why":"Defines the Tversky index used in the combined loss to handle class imbalance.","marker":"[14]"}],"fun_headline_variants":["Prompt-free twin transformer scores 0.80 mIoU in surgery","Dual-branch SegFormer beats SOTA on holistic surgical scenes","Surg-SegFormer: automatic scene parsing hits 0.80 mIoU","No user prompts needed: SegFormer twin tops surgical mIoU","Holistic surgical scene segmentation: Surg-SegFormer excels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The state-of-the-art comparison assumes that the reported mIoU values for Surg-SegFormer and for the baselines were computed on the same training and validation frames of EndoVis2017 and EndoVis2018, but the paper never states its data split and mixes its own retrained baselines with numbers taken from other papers.","fun_headline_variants_meta":{"raw":{"variants":["Prompt-free twin transformer scores 0.80 mIoU in surgery","Dual-branch SegFormer beats SOTA on holistic surgical scenes","Surg-SegFormer: automatic scene parsing hits 0.80 mIoU","No user prompts needed: SegFormer twin tops surgical mIoU","Holistic surgical scene segmentation: Surg-SegFormer excels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000493,"raw_usage":{"total_tokens":2444,"prompt_tokens":992,"completion_tokens":1452,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":1358}},"tokens_in":608,"tokens_out":1452,"duration_ms":11211,"temperature":1.0,"reasoning_tokens":1358,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:50:10.992293+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run Surg-SegFormer and the reference baselines on identical EndoVis2018 Task 1 folds, using the same merged seven labels and the same number of training frames, and compare per-class mIoU; if the reported 16-point gap over MedT shrinks to noise under a shared protocol, the state-of-the-art claim would not be supported. A second check is to count how many test frames actually contain a suturing needle and verify whether the 0.98 needle IoU depends on class prevalence.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SegFormer encoder-decoder architecture that both detection branches are built from."}],"review_version":1}