Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction

Atsushi Kouno; Hiroki Matsuzaki; Shunsuke Kikuchi

REVIEW 4 major objections 4 minor 36 references

Stitch-Inferencer stitches valid endoscopic frames into an instrument-free panorama, then runs segmentation and tracking on that panorama and reprojects results, granting long-range context without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:33 UTC pith:NUFCFGJT

load-bearing objection Useful inference wrapper with a real but fixable ablation gap: gains might come from masking, not stitching. the 4 major comments →

arxiv 2607.14968 v1 pith:NUFCFGJT submitted 2026-07-16 cs.CV

Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction

Shunsuke Kikuchi , Atsushi Kouno , Hiroki Matsuzaki This is my paper

classification cs.CV

keywords endoscopic videoimage stitchingpanoramic reconstructionsemantic segmentationobject trackinghomographyinstrument occlusioninference-time enhancement

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that endoscopic video understanding can be improved by keeping an explicit image-space memory — a stitched panorama — instead of relying on learned temporal features. The method masks out instruments, borders, and ports, warps each valid frame into a global canvas, runs a downstream model on a compact region of interest, and reprojects predictions to the current frame. The authors show that this inference-time wrapper improves Dice for most segmentation baselines and improves tracking accuracy across several trackers, without fine-tuning. The claim matters because it offers a training-free way to give existing single-frame models temporal context in the operating room.

Core claim

The central claim is that an online, instrument-free panoramic canvas can serve as an explicit spatial memory for endoscopic video, replacing the implicit feature memory used by video models. By accumulating valid observations under a cumulative homography and masking tools, ports, and black borders, the method preserves pixels that are temporarily occluded or outside the field of view. Downstream segmentation or tracking models applied to a compact ROI on this canvas, with predictions warped back to the current frame, outperform the same models applied to the raw frame on CholecSeg8k anatomy segmentation and on STIR24/SurgT tracking benchmarks.

What carries the argument

The mechanism is an online panoramic canvas built by estimating the homography between consecutive frames, masking invalid regions (instruments, ports, dark borders), and alpha-blending each valid frame into a 3x canvas. A cumulative warp tracks the current frame's location on the canvas; a blur gate skips poor frames; inference runs on a compact region of interest and is inverse-warped back to the frame. The ROI sizing is load-bearing: too large a view dilutes local detail, too small loses context.

Load-bearing premise

The whole pipeline depends on the assumption that the laparoscopic view is approximately planar and perpendicular to the tissue, so consecutive frames can be aligned by a single homography; if deformation or parallax breaks that alignment, the panorama accumulates errors and the context becomes stale or warped.

What would settle it

Run the same inference without and with Stitch-Inferencer on a long clinical video with strong tissue deformation (e.g., organs being retracted or bowel peristalsis), and look for a sequence where the stitched panorama visibly misaligns and the downstream Dice on the reprojected frame falls below the per-frame baseline.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Single-frame segmentation and tracking models get temporal context at inference time with no retraining.
Regions occluded by instruments or outside the field of view remain accessible through previously stitched pixels.
The stitching module runs at over 60 FPS, so the pipeline stays real-time for intraoperative use.
Consistent gains across CNN-based segmentation baselines and multiple trackers suggest the method is model-agnostic.
ROI construction matters: a narrow ROI around the current frame outperforms the full stitched canvas.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the panorama is treated as a persistent observation buffer, the same mechanism could support higher-level surgical understanding tasks such as phase recognition or surgical VQA, since the explicit canvas is interpretable and maskable.
The method's main weakness suggests a natural extension: confidence-aware warping or adaptive reset policies could make it robust to deformable tissue and parallax, which the paper lists as future work.
The core idea may transfer to other near-planar endoscopic or industrial scenes, where camera motion is moderate and a homography is a good local model, enabling similar training-free context for other models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful inference wrapper with a real but fixable ablation gap: gains might come from masking, not stitching.

read the letter

Colleague,

Know this paper for one thing: it offers a genuinely simple, model-agnostic way to give single-frame segmentation/trackers temporal context without retraining — stitch masked current-frame observations into a panorama, run the model on a compact ROI, reproject. That is useful, and the empirical package, though not huge, is honest in the numbers: most baselines improve, SegFormer and Litetracker don't on primary metrics, and the authors say so in the text even if the abstract overstates it.

The real soft spot is internal validity. The baseline is raw-frame inference. The Stitch pipeline masks instruments, ports, and borders before accumulating pixels. Tools and ports are known distractors, and they are excluded from the evaluation. So the reported Dice gains may be from input sanitization rather than from panoramic context. Table 3 compares ROI sizes but never runs "masked current frame, no accumulation." That single condition would isolate the stitching contribution, and it is absent. That is a concrete gap; the paper's central claim is not fully identified.

There is also a data leak to flag. The port mask is trained on Cholec80-port (a self-citation — first author's own dataset), and the segmentation evaluation is on CholecSeg8k, which is itself derived from Cholec80. If the port mask model saw any of the same Cholec80 frames or subjects, the port-mask accuracy in this evaluation could be optimistic. The paper doesn't disclose the overlap. It may be minor — the mask is auxiliary, and they align on m2caiSeg/GynSurg partly — but the relationship needs stating.

The stitched-panorama idea itself is not new — homography stitching and invalid region masking exist — but the packaging as an inference-time drop-in for arbitrary models is not in the cited prior work, and the real-time claim (60+ FPS for the module) is concrete. The method also rests on a planar-homography assumption the authors admit is vulnerable to deformation and parallax; that's a real deployment ceiling, not a hidden flaw.

Where does that leave the paper? It deserves a serious referee. The central idea is plausible and practically relevant, but the current draft's main effect is not causally attributed. A revision that adds a masked-frame-no-accumulation control — and clears up the Cholec80 overlap — would make the claim land. As it stands, I'd treat the reported gains as an upper bound on what the stitching actually contributes.

Referee Report

4 major / 4 minor

Summary. The paper proposes Stitch-Inferencer, an inference-time, model-agnostic framework for endoscopic video. The method maintains an online panoramic canvas by masking invalid regions (image borders, surgical tools, trocar ports), estimating inter-frame homographies from ALIKED/LightGlue matches, and compositing masked valid observations. Downstream segmentation or tracking models are then applied to a compact ROI on the panorama, and their predictions are reprojected to the current frame, requiring no retraining. On CholecSeg8k, the method reports Dice gains for four of five single-frame segmentation baselines (e.g., UNet++ 0.6884→0.7436), while SegFormer's Dice drops from 0.7128 to 0.6995. On STIR24/SurgT, most trackers improve, though Litetracker shows a small STIR24 δ-avg decrease (0.76298→0.76215) and slightly worse SurgT Err2D/Err3D. A three-way ROI ablation (internal, external, entire canvas) is reported. The paper also states important limitations regarding planar homography and deformable tissue.

Significance. If the central causal claim is confirmed—that explicit panoramic accumulation of previously observed pixels improves downstream inference—Stitch-Inferencer would be a practically valuable, model-agnostic plug-in: it requires no retraining, runs in real time (45–52 FPS full pipeline), and provides an interpretable image-space memory alternative to learned video models. The manuscript is also transparent about its planar-homography limitations and evaluates across a broad set of baselines and two tasks. However, the current evidence does not isolate the contribution of panoramic stitching from the contribution of input sanitization (tool/port/border masking). A simple control condition is needed before the paper's central claim can be accepted.

major comments (4)

[§3.3, Table 3; §2 (Invalid-region mask)] The central causal claim—that stitching/accumulation of panoramic context improves downstream inference—is not identified. The stitched variant differs from the baseline in two ways: it removes tools, ports, and borders via M_t, and it adds contextual pixels through the canvas. Since tools and ports are excluded from the evaluation metric, removing them could alone explain the reported Dice/tracking gains. Table 3 ablates the ROI definition (internal, external, entire canvas) but never includes a condition where only the current masked frame, without any accumulation, is passed through the same ROI crop/resize pipeline. This control is necessary to distinguish the effect of panoramic memory from the effect of input sanitization. Please add a 'masked current frame only' condition for both CholecSeg8k and STIR24.
[Abstract; Table 1; Table 2] The abstract claims 'consistent improvements across diverse baselines,' but the paper's own tables contradict this. Table 1 shows SegFormer Dice dropping from 0.7128 to 0.6995. Table 2 shows Litetracker δ-avg on STIR24 decreasing from 0.76298 to 0.76215, with slightly worse Err2D (5.1→5.2) and Err3D (1.55→1.83) on SurgT. The body text in §3.1 correctly limits the segmentation claim to 'all evaluated single-frame baselines except SegFormer,' but the abstract and discussion state the stronger claim. This should be qualified, or the exceptions should be explicitly addressed (e.g., with variance estimates).
[§2 (Inference ROI and reprojection); §3.3 Table 3] A second confound in the comparison is resolution/scale. The baseline is evaluated on the full frame at its native resolution, while the stitched variant resizes an ROI that contains the current frame plus additional context to the model input size. This changes the effective scale of the current-frame content and can independently influence segmentation and tracking performance. The internal-ROI condition in Table 3 should be compared against an internal-ROI applied to the current masked frame alone (no canvas accumulation). Without this control, the ROI size effect and the panoramic-context effect are entangled.
[§2 (Matching and inter-frame homography); §3.2] For tracking experiments, the paper applies optical-flow-based keypoint filtering using NeuFlowv2, but it does not state explicitly between which frames the flow is computed. If the flow is computed between I_t and I_{t-1}, the text should say so. If the flow uses future frames, the real-time/online claim in Table 2 would be compromised. Please specify the temporal direction and the latency contribution of this filtering step, since the paper's practical contribution depends on online operation.

minor comments (4)

[Throughout] The Tables contain formatting typos: 'T able 1', 'T able 2', 'T able 3'. Also, the p95 latency column does not specify units in the caption; should be milliseconds.
[§2, Eq. (2)] The notation for the seam-aware blending weight is inconsistent. The text introduces 'α_t^C = W(α_t^I; H_{t→C})', but H_{t→C} is not defined; the paper earlier defines W_{It→C} using S^T G_t. Please unify the notation.
[§3.3, Table 3] The ablation table does not report standard deviations or the number of sequences/frames used. Given the small mean differences for STIR24 (0.7778→0.7796), a variance estimate is important to assess whether the internal ROI gain is meaningful.
[§4, Limitations] The limitations paragraph appropriately acknowledges the planar-homography vulnerability and the lack of long-duration dynamic-camera benchmarks. This is a strength of the paper's framing, but the authors might also note that their re-anchoring heuristic (shear>15°, rotation>15°, scale>2.0) is not an adaptive deformation correction, so the stated limitation is accurate.

Circularity Check

0 steps flagged

No significant circularity; central claim is an empirical wrapper, with only a minor non-load-bearing self-citation for the auxiliary port mask.

full rationale

The core derivation chain is not self-definitional: Stitch-Inferencer is an inference-time wrapper that feeds pre-trained single-frame models a stitched, masked ROI and reprojects outputs. The reported gains (Table 1: UNet++ 0.6884→0.7436; Table 2 STIR/SurgT) are measured against external baselines and datasets (CholecSeg8k, STIR24, SurgT), not derived from the method's equations; no fitted parameter is relabeled as a prediction. The improvement is not forced by construction—SegFormer degrades (0.7128→0.6995), showing the result is contingent. The only same-author citation is Cholec80-port [13], used to train the auxiliary port mask (Section 2, 'Invalid-region mask'); this mask is a component of input sanitization, is validated separately (Dice 0.77), and could in principle be replaced by an external port segmenter, so it is not load-bearing for the stitching claim. The paper's own limitation (Section 4: planar homography vulnerable to deformation/parallax, no correction for stale context) honestly scopes the method. The absence of an ablation isolating stitching from masking is an internal-validity concern, not circularity, since the gain is empirical rather than a mathematical consequence of the pipeline.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The method does not introduce new physical or algorithmic entities beyond the panoramic canvas, which is constructed from observed pixels. It rests mainly on domain assumptions about homography validity, mask accuracy, and distribution compatibility. Several fixed hyperparameters (τ_blur, r, re-anchor thresholds, 2σ flow filter, canvas size) are hand-set and could affect results, but they are not fitted to the target metric.

free parameters (5)

Blur threshold τ_blur = 30
Frames with Laplacian variance below 30 are skipped for canvas storage; this threshold is hand-set and no sensitivity analysis is provided.
Blending decay radius r = 201 px
Spatial alpha-blending weight decays linearly up to 201 pixels from the invalid-region boundary, computed at 1/8 scale; the value is chosen by hand.
Re-anchor thresholds = shear > 15°, rotation > 15°, scale > 2.0
If the cumulative warp exceeds these extremes, the panorama re-initializes; the specific thresholds are not ablated.
Optical-flow outlier threshold = 2σ
Tracking experiments discard keypoints whose flow magnitude is a 2σ outlier; this is a fixed heuristic without ablation.
Canvas size = 3h × 3w
The canvas is three times the frame size in each dimension; the paper does not report experiments varying this choice.

axioms (4)

domain assumption Planar homography accurately models inter-frame alignment in laparoscopic video.
The stitching pipeline estimates H(t→t-1) and assumes the view is approximately perpendicular to the anatomical surface; the paper's own limitations acknowledge vulnerability to non-planar deformation and parallax.
domain assumption Tool, port, and border masks are accurate and transferable to the evaluation videos.
Invalid-region masking removes instruments, ports, and borders; if these masks fail, instruments or dark regions are stitched into the canvas and can corrupt downstream inference. Mask accuracy is reported only on EndoVis18 and Cholec80-port, not on the evaluation folds.
domain assumption The panorama ROI preserves enough local detail and input distribution for off-the-shelf models trained on original frames.
Downstream models are applied to a stitched ROI rather than natural frames. The ablation shows ROI size matters and SegFormer degrades, indicating a distribution shift that is not fully controlled.
domain assumption Accumulated pixels remain geometrically consistent over time without a drift-correction mechanism.
The canvas is only re-anchored on extreme warp; long sequences with tissue drift may produce stale or incorrectly aligned context, as the authors note in the limitations.

pith-pipeline@v1.3.0-alltime-deepseek · 7173 in / 12812 out tokens · 135813 ms · 2026-08-02T00:33:21.623837+00:00 · methodology

0 comments

read the original abstract

Surgical video understanding is fundamental to navigation systems. Endoscopic perception is often hindered by a limited field-of-view and frequent instrument occlusions, making spatio-temporal context essential for robust inference. These challenges have motivated video models that aggregate information across frames. However, existing video models typically store past observations implicitly in learned feature representations, often requiring task-specific video training, substantial annotated data, and increased computational cost. We propose Stitch-Inferencer, a real-time, model-agnostic inference framework that replaces implicit feature memory with an explicit image-space panoramic canvas. By stitching valid observations across frames, Stitch-Inferencer preserves previously observed pixels in an online, instrument-free view, expanding the effective field-of-view and providing direct access to regions that are temporarily occluded or absent from the current frame. Downstream segmentation or tracking models are applied to a compact region of interest on the panorama, and their predictions are reprojected to the current frame, enabling existing models to exploit long-range context without retraining. Experiments on anatomy segmentation and point/box tracking demonstrate consistent improvements across diverse baselines while preserving real-time throughput. The stitching module alone runs at over 60 FPS, providing a practical inference-time solution to enhance endoscopic perception in computationally constrained intraoperative environments. Source code will be made publicly available.

Figures

Figures reproduced from arXiv: 2607.14968 by Atsushi Kouno, Hiroki Matsuzaki, Shunsuke Kikuchi.

**Figure 1.** Figure 1: Graphical abstract. Stitch-Inferencer creates an instrument-free panoramic view and improves downstream segmentation and tracking at inference time without additional training; green and red boxes indicate the panorama context and the currentframe region with reprojection, respectively. rapidly, relying on the current frame alone remains brittle in realistic procedures, motivating the need for long-range… view at source ↗

**Figure 2.** Figure 2: Overview of the Stitch-Inferencer pipeline. Green: keypoint matching and homography estimation. Orange: warping, canvas update, and reprojection. standardization: letting m(xt) denote the flow-magnitude map, we compute its mean µt and standard deviation σt over valid pixels and discard keypoints at locations xt,i satisfying |m(xt,i) − µt| > 2σt. From the remaining correspondences {(xt,i, xt−1,i, wi)} wher… view at source ↗

**Figure 3.** Figure 3: Qualitative anatomy segmentation example on CholecSeg8k with ConvNeXt-B UNet++. From left to right: input frame, original inference, Stitch-Inferencer inference, and ground truth. Red boxes highlight a peripheral region where Stitch-Inferencer better recovers the ground-truth segmentation by leveraging panoramic spatial context [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

36 extracted references · 7 linked inside Pith

[1]

Scientific Data12(1), 825 (2025)

Alabi, O., et al.: Cholecinstanceseg: A tool instance segmentation dataset for la- paroscopic surgery. Scientific Data12(1), 825 (2025)

2025
[2]

Allan, M., et al.: 2018 robotic scene segmentation challenge (2020), https://arxiv.org/abs/2001.11190

Pith/arXiv arXiv 2018
[3]

Carion, N., et al.: Sam 3: Segment anything with concepts (2025), https://arxiv.org/abs/2511.16719

Pith/arXiv arXiv 2025
[4]

Medical Image Analysis91, 102985 (2024)

Cartucho, J., et al.: Surgt challenge: Benchmark of soft-tissue trackers for robotic surgery. Medical Image Analysis91, 102985 (2024)

2024
[5]

In: ECCV

Chen, L.C., et al.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: ECCV. pp. 833–851 (2018)

2018
[6]

IJCARS19(2), 375–382 (2024)

Grammatikopoulou, M., et al.: A spatio-temporal network for video semantic seg- mentation in surgical videos. IJCARS19(2), 375–382 (2024)

2024
[7]

Hong, W.Y., et al.: Cholecseg8k: A semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80 (2020), https://arxiv.org/abs/2012.12453

Pith/arXiv arXiv 2020
[8]

In: MICCAI (2024)

Hu, Q., et al.: Sali: Short-term alignment and long-term interaction network for colonoscopy video polyp segmentation. In: MICCAI (2024)

2024
[9]

Jiang, C., et al.: Medical sam3: A foundation model for universal prompt-driven medical image segmentation (2026), https://arxiv.org/abs/2601.10880

arXiv 2026
[10]

Ju, X., et al.: Segcol challenge: Semantic segmentation for tools and fold edges in colonoscopy data (2024), https://arxiv.org/abs/2412.16078

Pith/arXiv arXiv 2024
[11]

In: ICCV

Karaev, N., et al.: Cotracker3: Simpler and better point tracking by pseudo- labelling real videos. In: ICCV. pp. 6013–6022 (2025)

2025
[12]

In: MICCAI (2025)

Karaoglu, M.A., Ji, W., Abbas, A., Navab, N., et al.: Litetracker: Leveraging tem- poral causality for accurate low-latency tissue tracking. In: MICCAI (2025)

2025
[13]

Kikuchi, S., et al.: Cholec80-port: A geometrically consistent trocar port segmentation dataset for robust surgical scene understanding (2026), https://arxiv.org/abs/2602.17060

arXiv 2026
[14]

In: ICCV (2023)

Lindenberger, P., et al.: LightGlue: Local Feature Matching at Light Speed. In: ICCV (2023)

2023
[15]

CVPR (2022)

Liu, Z., et al.: A convnet for the 2020s. CVPR (2022)

2022
[16]

In: ICLR (2019)

Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019)

2019
[17]

International Journal of Computer Vision (2018)

Lukežič, A., et al.: Discriminative correlation filter tracker with channel and spatial reliability. International Journal of Computer Vision (2018)

2018
[18]

CoRR (2020)

Maqbool, S., et al.: m2caiseg: Semantic segmentation of laparoscopic images using convolutional neural networks. CoRR (2020)

2020
[19]

Mascagni, P., et al.: Endoscapes, a critical view of safety and surgical scene seg- mentation dataset for laparoscopic cholecystectomy. Sci. Data12(2025)

2025
[20]

Nasirihaghighi, S., et al.: Gynsurg: A comprehensive gynecology laparoscopic surgery dataset (2025), https://arxiv.org/abs/2506.11356

Pith/arXiv arXiv 2025
[21]

In: WACV

Neoral, M., et al.: MFT: Long-term tracking of every pixel. In: WACV. pp. 6837– 6847 (2024)

2024
[22]

In: ECCV

Park, J., et al.: Videomamba: Spatio-temporal selective state space model. In: ECCV. pp. 1–18 (2024)

2024
[23]

Psychogyios, D., et al.: Sar-rarp50: Segmentation of surgical instrumentation and action recognition on robot-assisted radical prostatectomy challenge (2024), https://arxiv.org/abs/2401.00496

Pith/arXiv arXiv 2024
[24]

In: MICCAI

Ronneberger, O., et al.: U-net: Convolutional networks for biomedical image seg- mentation. In: MICCAI. pp. 234–241 (2015) 10 S. Kikuchi et al

2015
[25]

arXiv:2511.06549 (2025), https://arxiv.org/abs/2511.06549

Rueckert, T., et al.: Video dataset for surgical phase, keypoint, and instru- ment recognition in laparoscopic surgery (phakir). arXiv:2511.06549 (2025), https://arxiv.org/abs/2511.06549

arXiv 2025
[26]

IEEE TMI43, 2634–2645 (2024)

Schmidt, A., et al.: Surgical tattoos in infrared: A dataset for quantifying tissue tracking and mapping. IEEE TMI43, 2634–2645 (2024)

2024
[27]

In: ICIP

Wang, H., et al.: Temporal memory attention for video semantic segmentation. In: ICIP. pp. 2254–2258. IEEE (2021)

2021
[28]

TPAMI (2019)

Wang, J., et al.: Deep high-resolution representation learning for visual recognition. TPAMI (2019)

2019
[29]

Computational Visual Media8(3), 415–424 (2022)

Wang, W., et al.: Pvt v2: Improved baselines with pyramid vision transformer. Computational Visual Media8(3), 415–424 (2022)

2022
[30]

In: ECCV

Xiao, T., et al.: Unified perceptual parsing for scene understanding. In: ECCV. pp. 432–448 (2018)

2018
[31]

In: NeurIPS (2021)

Xie, E., et al.: Segformer: Simple and efficient design for semantic segmentation with transformers. In: NeurIPS (2021)

2021
[32]

In: ICCV

Xu, X., et al.: Dual-temporal exemplar representation network for video semantic segmentation. In: ICCV. pp. 10775–10785 (2025)

2025
[33]

Zhang, Z., et al.: Neuflow v2: Push high-efficiency optical flow to the limit (2025)

2025
[34]

IEEE Transactions on Instrumentation & Measure- ment72, 1–16 (2023)

Zhao, X., et al.: Aliked: A lighter keypoint and descriptor extraction network via deformable transformation. IEEE Transactions on Instrumentation & Measure- ment72, 1–16 (2023)

2023
[35]

IEEE Transactions on Medical Imaging (2019)

Zhou, Z., et al.: Unet++: Redesigning skip connections to exploit multiscale fea- tures in image segmentation. IEEE Transactions on Medical Imaging (2019)

2019
[36]

Zia, A., et al.: Intuitive surgical surgtoolloc challenge results: 2022-2023 (2025), https://arxiv.org/abs/2305.07152

Pith/arXiv arXiv 2022

[1] [1]

Scientific Data12(1), 825 (2025)

Alabi, O., et al.: Cholecinstanceseg: A tool instance segmentation dataset for la- paroscopic surgery. Scientific Data12(1), 825 (2025)

2025

[2] [2]

Allan, M., et al.: 2018 robotic scene segmentation challenge (2020), https://arxiv.org/abs/2001.11190

Pith/arXiv arXiv 2018

[3] [3]

Carion, N., et al.: Sam 3: Segment anything with concepts (2025), https://arxiv.org/abs/2511.16719

Pith/arXiv arXiv 2025

[4] [4]

Medical Image Analysis91, 102985 (2024)

Cartucho, J., et al.: Surgt challenge: Benchmark of soft-tissue trackers for robotic surgery. Medical Image Analysis91, 102985 (2024)

2024

[5] [5]

In: ECCV

Chen, L.C., et al.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: ECCV. pp. 833–851 (2018)

2018

[6] [6]

IJCARS19(2), 375–382 (2024)

Grammatikopoulou, M., et al.: A spatio-temporal network for video semantic seg- mentation in surgical videos. IJCARS19(2), 375–382 (2024)

2024

[7] [7]

Hong, W.Y., et al.: Cholecseg8k: A semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80 (2020), https://arxiv.org/abs/2012.12453

Pith/arXiv arXiv 2020

[8] [8]

In: MICCAI (2024)

Hu, Q., et al.: Sali: Short-term alignment and long-term interaction network for colonoscopy video polyp segmentation. In: MICCAI (2024)

2024

[9] [9]

Jiang, C., et al.: Medical sam3: A foundation model for universal prompt-driven medical image segmentation (2026), https://arxiv.org/abs/2601.10880

arXiv 2026

[10] [10]

Ju, X., et al.: Segcol challenge: Semantic segmentation for tools and fold edges in colonoscopy data (2024), https://arxiv.org/abs/2412.16078

Pith/arXiv arXiv 2024

[11] [11]

In: ICCV

Karaev, N., et al.: Cotracker3: Simpler and better point tracking by pseudo- labelling real videos. In: ICCV. pp. 6013–6022 (2025)

2025

[12] [12]

In: MICCAI (2025)

Karaoglu, M.A., Ji, W., Abbas, A., Navab, N., et al.: Litetracker: Leveraging tem- poral causality for accurate low-latency tissue tracking. In: MICCAI (2025)

2025

[13] [13]

Kikuchi, S., et al.: Cholec80-port: A geometrically consistent trocar port segmentation dataset for robust surgical scene understanding (2026), https://arxiv.org/abs/2602.17060

arXiv 2026

[14] [14]

In: ICCV (2023)

Lindenberger, P., et al.: LightGlue: Local Feature Matching at Light Speed. In: ICCV (2023)

2023

[15] [15]

CVPR (2022)

Liu, Z., et al.: A convnet for the 2020s. CVPR (2022)

2022

[16] [16]

In: ICLR (2019)

Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019)

2019

[17] [17]

International Journal of Computer Vision (2018)

Lukežič, A., et al.: Discriminative correlation filter tracker with channel and spatial reliability. International Journal of Computer Vision (2018)

2018

[18] [18]

CoRR (2020)

Maqbool, S., et al.: m2caiseg: Semantic segmentation of laparoscopic images using convolutional neural networks. CoRR (2020)

2020

[19] [19]

Mascagni, P., et al.: Endoscapes, a critical view of safety and surgical scene seg- mentation dataset for laparoscopic cholecystectomy. Sci. Data12(2025)

2025

[20] [20]

Nasirihaghighi, S., et al.: Gynsurg: A comprehensive gynecology laparoscopic surgery dataset (2025), https://arxiv.org/abs/2506.11356

Pith/arXiv arXiv 2025

[21] [21]

In: WACV

Neoral, M., et al.: MFT: Long-term tracking of every pixel. In: WACV. pp. 6837– 6847 (2024)

2024

[22] [22]

In: ECCV

Park, J., et al.: Videomamba: Spatio-temporal selective state space model. In: ECCV. pp. 1–18 (2024)

2024

[23] [23]

Psychogyios, D., et al.: Sar-rarp50: Segmentation of surgical instrumentation and action recognition on robot-assisted radical prostatectomy challenge (2024), https://arxiv.org/abs/2401.00496

Pith/arXiv arXiv 2024

[24] [24]

In: MICCAI

Ronneberger, O., et al.: U-net: Convolutional networks for biomedical image seg- mentation. In: MICCAI. pp. 234–241 (2015) 10 S. Kikuchi et al

2015

[25] [25]

arXiv:2511.06549 (2025), https://arxiv.org/abs/2511.06549

Rueckert, T., et al.: Video dataset for surgical phase, keypoint, and instru- ment recognition in laparoscopic surgery (phakir). arXiv:2511.06549 (2025), https://arxiv.org/abs/2511.06549

arXiv 2025

[26] [26]

IEEE TMI43, 2634–2645 (2024)

Schmidt, A., et al.: Surgical tattoos in infrared: A dataset for quantifying tissue tracking and mapping. IEEE TMI43, 2634–2645 (2024)

2024

[27] [27]

In: ICIP

Wang, H., et al.: Temporal memory attention for video semantic segmentation. In: ICIP. pp. 2254–2258. IEEE (2021)

2021

[28] [28]

TPAMI (2019)

Wang, J., et al.: Deep high-resolution representation learning for visual recognition. TPAMI (2019)

2019

[29] [29]

Computational Visual Media8(3), 415–424 (2022)

Wang, W., et al.: Pvt v2: Improved baselines with pyramid vision transformer. Computational Visual Media8(3), 415–424 (2022)

2022

[30] [30]

In: ECCV

Xiao, T., et al.: Unified perceptual parsing for scene understanding. In: ECCV. pp. 432–448 (2018)

2018

[31] [31]

In: NeurIPS (2021)

Xie, E., et al.: Segformer: Simple and efficient design for semantic segmentation with transformers. In: NeurIPS (2021)

2021

[32] [32]

In: ICCV

Xu, X., et al.: Dual-temporal exemplar representation network for video semantic segmentation. In: ICCV. pp. 10775–10785 (2025)

2025

[33] [33]

Zhang, Z., et al.: Neuflow v2: Push high-efficiency optical flow to the limit (2025)

2025

[34] [34]

IEEE Transactions on Instrumentation & Measure- ment72, 1–16 (2023)

Zhao, X., et al.: Aliked: A lighter keypoint and descriptor extraction network via deformable transformation. IEEE Transactions on Instrumentation & Measure- ment72, 1–16 (2023)

2023

[35] [35]

IEEE Transactions on Medical Imaging (2019)

Zhou, Z., et al.: Unet++: Redesigning skip connections to exploit multiscale fea- tures in image segmentation. IEEE Transactions on Medical Imaging (2019)

2019

[36] [36]

Zia, A., et al.: Intuitive surgical surgtoolloc challenge results: 2022-2023 (2025), https://arxiv.org/abs/2305.07152

Pith/arXiv arXiv 2022