Pith. sign in

REVIEW 4 major objections 5 minor 20 references

PDZSeg: Adapting the Foundation Model for Dissection Zone Segmentation with Visual Prompts in Robot-assisted Endoscopic Submucosal Dissection

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read PDZSeg adapts a vision foundation model to turn surgeon scribbles and bounding boxes into precise dissection-zone contours for robot-assisted ESD.

desk verdict A useful new benchmark and a sensible prompting recipe, but the central claim is false on the paper's own best setting: long-scribble Point-Rend beats PDZSeg on every metric in Table 1. read the letter →

arxiv 2411.18169 v1 pith:COK6AIS4 submitted 2024-11-27 cs.CV cs.AI

classification cs.CVcs.AI
keywords dissectionzonesegmentationvisualpromptingscribblepromptsboundingboxrobot-assistedESDDINOv2LoRAendoscopicsubmucosal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces PDZSeg, a segmentation model that takes a visual prompt—a point, a scribble, or a bounding box drawn by the surgeon—overlays it directly on the RGB endoscopic image, and predicts the dissection zone during robot-assisted endoscopic submucosal dissection (ESD). To make this work, the authors fine-tune the DINOv2 foundation model with LoRA adapters on a new benchmark dataset, ESD-DZSeg, built from 21 ex-vivo porcine robotic ESD procedures and 1,849 expert-annotated images. They report that prompted versions outperform their unprompted baseline and, on average across prompt types, outperform the comparison segmentation models, with the long-scribble prompt giving the best dissection-zone IoU and robust behavior under image corruption. If accepted, the work offers a practical way for an experienced surgeon to guide a less experienced one by simply sketching the safe dissection region, and it provides the first benchmark for this task.

What carries the argument

The central mechanism is visual referral by image overlay: instead of a learned prompt embedding, the prompt is drawn into the RGB input itself, so the model sees the user's mark as part of the scene. The encoder is DINOv2, a self-supervised vision transformer, kept largely frozen and adapted through LoRA, a low-rank adaptation that injects small trainable matrices into the query and value projections of each attention block. The decoder is an all-MLP network that fuses multi-level features from transformer blocks 3, 6, 9, and 12 and maps them to a two-class mask (dissection zone and no-go zone), so the prompt acts as a location and boundary cue.

What would settle it

Take the trained PDZSeg and evaluate it on the same test set with prompts systematically displaced outside the dissection zone, partially erased scribbles, or scribbles that cross the true boundary; if IoU remains high and the predicted mask does not move with the prompt, the overlay is acting as a localization hint rather than as boundary information, which would falsify the claimed mechanism.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that visual prompts can be folded into segmentation without a separate prompt-encoding branch: painting the user's marker onto the image and fine-tuning a foundation model is enough to steer the model toward the dissection zone. The evidence is the ESD-DZSeg results: with a long scribble, the model reaches 74.06 IoU and 84.30 Dice on the dissection zone, versus 47.55 IoU and 63.01 Dice for the same architecture without a prompt, and the model retains higher IoU than a strong baseline under Gaussian noise, smoke, contrast, brightness, and motion blur when the same prompt is provided. The paper also positions the mixed-prompt experiments as showing that partial or heterogeneous prompt availability during training still yields strong segmentation performance.

Load-bearing premise

The load-bearing premise is that overlaying a prompt teaches the model to refine tissue boundaries rather than to segment only tissue near the prompt, and that an expert's placed prompt reliably indicates the dissection zone; neither is tested, and the dataset is ex-vivo porcine from one center.

Editorial extensions

If this is right

  • With a long-scribble prompt, PDZSeg's dissection-zone IoU rises from 47.55 (no prompt) to 74.06 on ESD-DZSeg, showing that the type of visual prompt strongly modulates performance.
  • The visual-prompt overlay also improves standard segmentation baselines such as DeepLabv3, whose dissection-zone IoU climbs from 45.95 to 69.24 with a bounding box, so the prompting scheme is not tied to the proposed architecture alone.
  • Training with mixed prompt availability (prompted-to-unprompted ratios of 6:4, 5:5, and 4:6) keeps performance well above unprompted-only training, supporting deployment where the surgeon does not prompt every frame.
  • Under five corruption types, the long-scribble-prompted model keeps dissection-zone IoU between 64.91 and 73.52, while the Point-Rend baseline drops to 2.89 under Gaussian noise, indicating prompt-conditioned robustness.
  • The paper introduces the ESD-DZSeg dataset, 1,849 images from 21 ex-vivo porcine robotic ESD videos, as a benchmark for dissection zone segmentation and visual prompt interpretation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The overlay design suggests the prompt may function as a soft region prior rather than a strict boundary constraint; a direct ablation that removes or erodes the scribble inside the target region would clarify whether the model is localizing or contour-refining.
  • Because all images come from ex-vivo porcine tissue with a fixed endoscope and robotic platform, transfer to live human ESD is untested; a cross-domain evaluation with different centers, in-vivo tissue, and varying scope optics would be the most direct next experiment.
  • The paper evaluates only correct, expert-style prompts; robustness to misplaced, partial, or adversarial prompts is not measured, so a natural safety extension is to pair the model with a prompt-quality check or allow the surgeon to correct the suggested contour.
  • If prompt-conditioned robustness holds in vivo, the approach could be extended from a suggestion tool to an intraoperative alarm that detects when the electric knife approaches the predicted boundary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes PDZSeg, a dissection-zone segmentation model for robot-assisted endoscopic submucosal dissection. A DINOv2 ViT backbone is fine-tuned with LoRA and connected to an all-MLP decoder; user-provided visual prompts (point, short/long scribble, bounding box) are overlaid directly on the RGB image. The authors introduce the ESD-DZSeg benchmark built from 1,849 ex-vivo porcine images from 21 procedures and report in-domain, prompt-availability, and robustness experiments. They conclude that PDZSeg outperforms state-of-the-art segmentation methods and effectively uses visual prompts for dissection-zone suggestion.

Significance. The dataset and code release are useful community assets, and the prompt-overlay design is simple enough to be applied to existing baselines. The intended application—real-time contour guidance from an experienced surgeon's prompt—is clinically meaningful. The visual prompt is a legitimate test-time input, so I do not view the performance gain from adding it as circular; the concern is whether the model is using the prompt as intended. However, the headline performance claim is contradicted by Table 1 in the long-scribble condition, and the experimental design lacks statistical validation. The paper is best viewed as a preliminary benchmark and method proposal rather than a demonstrated SOTA result.

major comments (4)
  1. [Abstract and Table 1] Table 1, long-scribble rows: in the condition the authors themselves identify as the best prompt type, Point-Rend outperforms PDZSeg on every reported metric (dissection-zone IoU 74.73 vs 74.06, Dice 85.54 vs 84.30, no-go-zone IoU 99.17 vs 99.15, mean IoU 86.95 vs 86.60, mean Dice 92.56 vs 91.93). Consequently, the abstract and Section 5 statement that PDZSeg 'outperforms state-of-the-art segmentation approaches' is not supported by the data and should be qualified or corrected.
  2. [Section 4.2, Tables 1-4] All quantitative claims rest on a single train/test split of four videos (369 test images) with no error bars, confidence intervals, or significance tests, and many favorable differences are small (e.g., 65.45 vs 64.15 dissection-zone IoU with short-scribble prompts). The paper should add repeated runs with variance estimates and paired tests, or explicitly present the results as preliminary observations.
  3. [Sections 3.2.1 and 4.2] The experiments never vary prompt quality or location, so the paper does not test whether the model refines tissue boundaries from the prompt or simply segments in the vicinity of the overlaid cue. Adding evaluations with noisy, misplaced, or contradictory prompts would support the claimed mechanism and the clinical-safety motivation.
  4. [Section 4.2, Table 4] The robustness experiment applies five image corruptions at a single fixed severity level (3) and reports no severity-response analysis or uncertainty quantification, so the conclusion that PDZSeg is robust is not established beyond this specific setting.
minor comments (5)
  1. [Section 4.1] The implementation states that Vit-Base has a 'feature dimension of 784', but DINOv2 ViT-Base uses an embedding dimension of 768; please correct or clarify the intended dimension.
  2. [Various section headers] Several table captions contain typos: 'T able 1', 'T able 2', and 'T able 3' appear with an extra space, and Section 3.1 writes 'Dissecon Zone Segmentation' instead of 'Dissection Zone Segmentation'.
  3. [Table 1] The Medical-SAM-Adapter baseline reports only dissection-zone IoU and Dice and only under the point-prompt condition; all other cells are '/'. Please complete the table or state explicitly why those results are unavailable.
  4. [Section 3.2.2] The attention equations write Wqx, Wkx, and Wvx without subscript formatting; these should be W_q x, W_k x, and W_v x for readability.
  5. [Figure 3] The caption says 'The first two columns display the ground truth and the segmentation masks predicted by our model', but the layout appears to show one ground-truth column followed by multiple prediction columns; please re-check the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the training/evaluation chain is self-contained; the strong SOTA claim is contradicted by the paper's own Table 1, but that is a correctness issue, not circularity.

full rationale

The paper's derivation chain is not circular. PDZSeg is constructed from a pretrained DINOv2 encoder, LoRA adapters, and an MLP decoder; the input is the RGB image with a visual prompt overlaid, and the training target is the expert-annotated dissection zone mask. Training uses the ESD-DZSeg training videos (1,480 images), and evaluation is on held-out videos (369 images), so no parameter is fitted to the test outputs. The visual prompt is an input at test time, not a target used for fitting, and the reported improvement from prompts is a property of the task rather than a circular construction. The paper does not define the predicted dissection zone in terms of the prompt, nor does it rename a fitted constant as a prediction. Self-citations are present but not load-bearing: reference [3] is cited only as the source of the robotic data-collection platform and the ex-vivo porcine recordings, not as evidence for the segmentation result, while DINOv2 [4] and LoRA [5] are external methods. No uniqueness theorem or prior same-author result is invoked to forbid alternative designs. In contrast, the paper's central claim that PDZSeg 'outperforms state-of-the-art segmentation approaches' is contradicted by its own Table 1: under the long-scribble prompt, Point-Rend achieves higher dissection-zone IoU (74.73 vs 74.06), Dice (85.54 vs 84.30), mean IoU (86.95 vs 86.60), and mean Dice (92.56 vs 91.93). This is an empirical inconsistency, not circular reasoning, and therefore does not raise the circularity score. Honest finding: no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities or forces. Its assumptions are domain-transfer assumptions common in medical imaging: pretrained features transfer, expert annotations are reliable, and animal models represent human conditions. The free parameters are standard hyperparameters and experimental design choices, none fitted to the test set.

free parameters (4)
  • LoRA rank = 4
    Chosen by hand for the LoRA adaptation matrices; not fitted to the test set, but it affects capacity and is not justified by an ablation.
  • Input image size = 532 x 532
    Resized from the original 1310 x 1010; a design choice that may discard spatial detail relevant to fine tissue boundaries.
  • Corruption severity level = 3 of 5
    Used for the robustness evaluation; only one severity level is tested and sensitivity to severity is not analyzed.
  • Prompt to non-prompt training ratio = 6:4, 5:5, 4:6 in Table 2; 25% each in Table 3
    These mixes are chosen by the authors to simulate clinical variability; there is no analysis of how the ratio affects performance.
assumptions (5)
  • domain assumption DINOv2's self-supervised features transfer to endoscopic submucosal images.
    Invoked in Section 3.2.2 (image encoder), where the pretrained DINOv2 is used without domain-specific pretraining.
  • domain assumption LoRA fine-tuning preserves the general visual features of DINOv2 while adapting to the target domain.
    Invoked in Section 3.2.2 (LoRA module); no ablation verifies that LoRA is better than full fine-tuning or feature freezing.
  • domain assumption Expert endoscopist annotations of dissection zones are accurate and consistent.
    Section 3.1 states expert endoscopists annotated the zones; no inter-observer agreement is reported.
  • domain assumption Ex-vivo porcine tissue is a valid proxy for human ESD procedures.
    Section 3.1 describes data collection from ex-vivo porcine models; the paper does not validate on human data.
  • standard math Standard transformer equations (patch embedding, attention softmax) are correct.
    Used in Section 3.2.2 to define the model; no novel mathematical claims are made.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PDZSeg: Adapting the Foundation Model for Dissection Zone Segmentation with Visual Prompts in Robot-assisted Endoscopic Submucosal Dissection." pith.science (2026). https://pith.science/paper/COK6AIS4

@misc{pith2026241118169,
  author       = {Pith},
  title        = {Pith review of: PDZSeg: Adapting the Foundation Model for Dissection Zone Segmentation with Visual Prompts in Robot-assisted Endoscopic Submucosal Dissection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/COK6AIS4}},
  note         = {Machine review of arXiv:2411.18169}
}
read the original abstract

Purpose: Endoscopic surgical environments present challenges for dissection zone segmentation due to unclear boundaries between tissue types, leading to segmentation errors where models misidentify or overlook edges. This study aims to provide precise dissection zone suggestions during endoscopic submucosal dissection (ESD) procedures, enhancing ESD safety. Methods: We propose the Prompted-based Dissection Zone Segmentation (PDZSeg) model, designed to leverage diverse visual prompts such as scribbles and bounding boxes. By overlaying these prompts onto images and fine-tuning a foundational model on a specialized dataset, our approach improves segmentation performance and user experience through flexible input methods. Results: The PDZSeg model was validated using three experimental setups: in-domain evaluation, variability in visual prompt availability, and robustness assessment. Using the ESD-DZSeg dataset, results show that our method outperforms state-of-the-art segmentation approaches. This is the first study to integrate visual prompt design into dissection zone segmentation. Conclusion: The PDZSeg model effectively utilizes visual prompts to enhance segmentation performance and user experience, supported by the novel ESD-DZSeg dataset as a benchmark for dissection zone segmentation in ESD. Our work establishes a foundation for future research.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 6 canonical work pages

  1. [1]

    Surgical endoscopy 26, 3584–3591 (2012)

    Chiu, P.W.Y., Teoh, A.Y.B., To, K.F., Wong, S.K.H., Liu, S.Y.W., Lam, C.C., Yung, M.Y., Chan, F.K.L., Lau, J.Y.W., Ng, E.K.W.: Endoscopic submucosal dissection (esd) compared with gastrectomy for treatment of early gastric neoplasia: a retrospective cohort study. Surgical endoscopy 26, 3584–3591 (2012)

  2. [2]

    In: Med- ical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, pp

    Zhang, J., Nie, Y., Lyu, Y., Li, H., Chang, J., Yang, X., Zhang, J.J.: Symmetric dilated convolution for surgical gesture recognition. In: Med- ical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, pp. 409–418 (2020). Springer

  3. [3]

    The International Journal of Robotics Research 43(3), 281–304 (2024)

    Gao, H., Yang, X., Xiao, X., Zhu, X., Zhang, T., Hou, C., Liu, H., Meng, M.Q.-H., Sun, L., Zuo, X., et al.: Transendoscopic flexible parallel contin- uum robotic mechanism for bimanual endoscopic submucosal dissection. The International Journal of Robotics Research 43(3), 281–304 (2024)

  4. [4]

    arXiv preprint arXiv:2304.07193 (2023)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

  5. [5]

    arXiv preprint arXiv:2106.09685 (2021)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P.H., et al.: Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6881–6890 (2021)

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Fan, M., Lai, S., Huang, J., Wei, X., Chai, Z., Luo, J., Wei, X.: Rethink- ing bisenet for real-time semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9716–9725 (2021)

  8. [8]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Kirillov, A., Wu, Y., He, K., Girshick, R.: Pointrend: Image segmentation as rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9799–9808 (2020)

Show all 20 references
  1. [9]

    BMC Medical Imaging 22(1), 109 (2022) Springer Nature 2021 LATEX template 12 PDZSeg: Dissection Zone Segmentation

    Ni, J., Wu, J., Elazab, A., Tong, J., Chen, Z.: Dnl-net: deformed non- local neural network for blood vessel segmentation. BMC Medical Imaging 22(1), 109 (2022) Springer Nature 2021 LATEX template 12 PDZSeg: Dissection Zone Segmentation

  2. [10]

    Chen, L.-C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arxiv. arXiv preprint arXiv:1706.05587 5 (2017)

  3. [11]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)

  4. [12]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.-Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015–4026 (2023)

  5. [13]

    arXiv preprint arXiv:2304.12620 (2023)

    Wu, J., Ji, W., Liu, Y., Fu, H., Xu, M., Xu, Y., Jin, Y.: Medi- cal sam adapter: Adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304.12620 (2023)

  6. [14]

    arXiv preprint arXiv:2307.03601 (2023)

    Zhang, S., Sun, P., Chen, S., Xiao, M., Shao, W., Zhang, W., Liu, Y., Chen, K., Luo, P.: Gpt4roi: Instruction tuning large language model on region-of-interest. arXiv preprint arXiv:2307.03601 (2023)

  7. [15]

    arxiv 2023

    Chen, K., Zhang, Z., Zeng, W., Zhang, R., Zhu, F., Zhao, R.: Shikra: Unleashing multimodal llm’s referential dialogue magic. arxiv 2023. arXiv preprint arXiv:2306.15195

  8. [16]

    arXiv preprint arXiv:2309.17421 9(1), 1 (2023)

    Yang, Z., Li, L., Lin, K., Wang, J., Lin, C.-C., Liu, Z., Wang, L.: The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421 9(1), 1 (2023)

  9. [17]

    ICLR (2021)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. ICLR (2021)

  10. [18]

    In: International Conference on Learning Representations (ICLR), San Diega, CA, USA (2015)

    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR), San Diega, CA, USA (2015)

  11. [19]

    arXiv preprint arXiv:1902.04502 (2019)

    Poudel, R.P., Liwicki, S., Cipolla, R.: Fast-scnn: Fast semantic segmenta- tion network. arXiv preprint arXiv:1902.04502 (2019)

  12. [20]

    arXiv preprint arXiv:1903.12261 (2019)

    Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 (2019)

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.