REVIEW 4 major objections 6 minor 29 references
Reviewed by Pith at T0; open to challenge.
T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →
T0 review · grok-4.5
Dense 2D tracks let surgical video rebuild deformable 3D anatomy online even when camera poses are missing or noisy.
2026-07-10 07:54 UTC pith:HS5SVZQT
load-bearing objection Solid systems paper: track-anchored deformation + motion-gated pose lets online deformable GS-SLAM work without clean kinematics, with clear StereoMIS gains; single-dataset and unvalidated gate are the real limits. the 4 major comments →
Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Track2Map shows that dense 2D point tracks can both initialize sparse deformation anchors and gate camera-pose updates, yielding an online deformable 3D Gaussian SLAM system that produces higher-quality reconstructions and more accurate trajectories on StereoMIS than prior endoscopic SLAM and than pose-prior-dependent online Gaussian pipelines, under clean, noisy, or missing pose initialization.
What carries the argument
Motion-aware pose gating from track statistics: the circular standard deviation of 2D track flow directions decides whether the camera is moving; only then is pose refined jointly with the scene, while static periods freeze pose and rely on track-lifted 3D anchors to drive deformation.
Load-bearing premise
The system assumes that coherent, low-dispersion flow directions mean the camera is moving and that high dispersion means only tissue or tools are moving, so it freezes pose whenever the flow directions are too scattered.
What would settle it
Sequences in which large coherent tissue motion or pure optical-axis camera motion produces low flow-direction dispersion while the endoscope is actually static (or the reverse), causing the gate to open or close incorrectly and measurably raising trajectory error or reconstruction distortion relative to the reported StereoMIS numbers.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. Track2Map is an online stereo 3D Gaussian Splatting pipeline for deformable surgical scenes that jointly refines camera pose and a sparse-anchor deformation field from video. Dense 2D tracks (CoTracker3) are lifted with stereo depth to (i) initialize anchor deformations and (ii) gate pose updates via the circular standard deviation of track flow directions, freezing pose when motion is judged local rather than camera-driven. The system is designed to run with clean, noisy, or absent pose priors. On StereoMIS it reports higher reconstruction metrics than endoscopic SLAM baselines in the no-prior regime and than Online-endo-track under clean/noisy priors, plus lower ATE/RPE than EndoGSLAM-H and Endo3R, with ablations of pose optimization, gating, and deformation.
Significance. If the StereoMIS gains transfer, the work is practically useful: many RAMIS settings lack reliable kinematics, and a single pipeline that degrades gracefully from clean priors to pure vision-only SLAM is valuable for AR overlays and instrument–tissue analysis. Strengths include multi-regime evaluation (none / light / heavy / clean pose), comparison to both SLAM and prior-dependent 3DGS methods, pose ATE/RPE, a component ablation (Table 3), public code, and an explicit limitations section. The track-anchored deformation init and motion gate are concrete engineering contributions relative to unconstrained joint pose–deformation optimization.
major comments (4)
- The central claim of robust reconstruction and pose under clean, noisy, and absent priors is supported only on StereoMIS (Tables 1–3, Figs. 3–4). No second dataset, no per-sequence variance/error bars, and no cross-procedure split are reported. For a journal claim of regime-agnostic robustness this is load-bearing: gains may reflect StereoMIS motion statistics (intermittent lateral camera motion, trackable texture) rather than general RAMIS behavior. At minimum, results on another public stereo/deformable set (or a held-out StereoMIS procedure with different motion) and sequence-level statistics are needed.
- Sec. 2.1 defines the motion gate gt = I[σθ,t ≤ τ] with τ = 0.38 from circular std of track flow directions, and freezes pose when gt = 0. This mechanism is central to drift reduction and to the no-prior / noisy-prior claims, yet the paper reports no gate accuracy, no static/moving frame labels, no sensitivity of PSNR/ATE to τ, and no breakdown of how often the gate fires per sequence. The Limitations section lists plausible failure modes (optical-axis motion, coherent tissue motion, occlusion, weak texture) without quantifying them. A short validation of gate decisions against kinematics-derived motion labels (or a τ sweep) is required to show the gate is correct, not merely helpful on this data (Table 3).
- Table 1 monocular Track2Map (Depth Anything V2) already beats prior SLAM methods by a large margin (PSNR 26.70 vs EndoFlow-SLAM 21.96), and stereo adds a further gain. The manuscript does not isolate how much of the no-prior improvement comes from modern stereo/monocular depth and CoTracker3 versus the proposed gating and track-anchored deformation. A controlled comparison that holds depth and tracker fixed while swapping only the mapping/pose modules (or reports EndoGSLAM/EndoFlow with the same depth) would make the contribution attribution clearer.
- The system is described as online SLAM but runs at ~6 s/frame (Limitations), with keyframe processing recommended. For journal positioning as an online SLAM method, either (i) report wall-clock breakdown and a keyframe-only accuracy/latency trade-off, or (ii) soften the real-time/online framing to “incremental / sequential” so claims match the implementation.
minor comments (6)
- Fig. 2 caption and body: STIR is cited as both [17] and [18]; unify the STIR2024 / STIRC2024 naming and reference keys.
- Eq. (7) writes δξ ∈ SE(3) while Eq. (3) and the text treat δξ as a 6D se(3) increment; fix the group vs algebra notation.
- Loss weights in Sec. 3.1 list wdepth, wcolor, wpose_prior but Sec. 2.3 uses wphoto, wgeo, wdef, wtrk, wreg; provide a single consistent weight table and values used for all regimes.
- Table 1: EndoFlow-SLAM SSIM is printed as “0.590.27” (missing separator between SSIM and LPIPS).
- Fig. 3 compares trajectories on three sequences but does not state units, alignment method (e.g., Sim(3)/SE(3)), or which frames are gated static; a short caption note would aid reproducibility.
- Clarify whether tool masks (Fig. 1) are required at test time or optional, and how masked pixels enter Lphoto/Lgeo when masks are absent.
Circularity Check
No circularity: empirical SLAM pipeline evaluated with independent external metrics on StereoMIS; design choices are not success criteria by construction.
full rationale
Track2Map is an engineering/systems paper: it combines off-the-shelf CoTracker3 tracks, stereo depth, a sparse-anchor deformable 3DGS representation (following the loss family of Hayoz et al. [8], non-overlapping authors), a motion gate on circular flow-direction dispersion, and joint photometric/geometric optimization. Claimed improvements (PSNR/SSIM/LPIPS reconstruction; ATE/RPE pose) are measured against external baselines and StereoMIS ground truth, not quantities defined by the gate threshold τ, the track-lifted anchors, or the fitted loss weights. Ablations (Tab. 3) remove components and re-measure the same independent metrics; they do not redefine success as the presence of those components. There is no self-definitional loop (X defined via Y then used to derive Y), no fitted parameter re-labeled as a prediction of a closely related target, no load-bearing uniqueness theorem imported from the same authors, and no renaming of a known empirical pattern as a first-principles result. Building on [8]'s deformation losses and citing external trackers/depth models is ordinary prior-art use, not circular derivation. Limitations of the motion-gate heuristic affect generalization risk, not circularity of the reported claims.
Axiom & Free-Parameter Ledger
free parameters (3)
- motion-gate threshold τ =
0.38
- loss weights (w_depth, w_color, w_pose_prior, w_trk, w_reg, etc.) =
wdepth=10, wcolor=5, wpose_prior=0.25
- deformation interpolation scale α and nearest-anchor count N
axioms (4)
- domain assumption Endoscope motion in RAMIS is typically intermittent; long static-camera intervals are common while tools deform tissue.
- ad hoc to paper Low circular dispersion of 2D track flow directions indicates camera-dominant motion; high dispersion indicates local tissue/tool motion.
- domain assumption Stereo depth and camera intrinsics provide metric 3D lifts of 2D tracks suitable for SE(3) alignment and deformation init.
- domain assumption Sparse anchor-based deformation plus local rigidity/isometry regularizers (as in [8]) adequately model soft-tissue motion for online GS mapping.
invented entities (2)
-
Track2Map motion gate (circular std of track flow directions)
no independent evidence
-
Track-anchored deformation initialization (3D-lifted CoTracker displacements as sparse control offsets)
no independent evidence
read the original abstract
Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.
Figures
Reference graph
Works this paper leans on
-
[1]
In: 2020 25th international conference on pattern recogni- tion (ICPR)
Agudo, A.: Total estimation from rgb video: on-line camera self-calibration, non- rigid shape and motion. In: 2020 25th international conference on pattern recogni- tion (ICPR). pp. 8140–8147. IEEE (2021)
work page 2020
-
[2]
IEEE transactions on pattern analysis and machine intelligence38(5), 979–994 (2015)
Agudo, A., Moreno-Noguer, F., Calvo, B., Montiel, J.M.M.: Sequential non-rigid structure from motion using physical priors. IEEE transactions on pattern analysis and machine intelligence38(5), 979–994 (2015)
work page 2015
-
[3]
Medical image analysis90, 102956 (2023)
Bobrow, T.L., Golhar, M., Vijayan, R., Akshintala, V.S., Garcia, J.R., Durr, N.J.: Colonoscopy 3d video dataset with paired depth from 2d-3d registration. Medical image analysis90, 102956 (2023)
work page 2023
-
[4]
Bonilla, S., Zhang, S., Psychogyios, D., Stoyanov, D., Vasconcelos, F., Bano, S.: Gaussian pancakes: geometrically-regularized 3d gaussian splatting for realistic en- doscopicreconstruction.In:InternationalConferenceonMedicalImageComputing and Computer-Assisted Intervention. pp. 274–283. Springer (2024)
work page 2024
-
[5]
Medical image analysis91, 102985 (2024)
Cartucho, J., Weld, A., Tukra, S., Xu, H., Matsuzaki, H., Ishikawa, T., Kwon, M., Jang, Y.E., Kim, K.J., Lee, G., et al.: Surgt challenge: Benchmark of soft-tissue trackers for robotic surgery. Medical image analysis91, 102985 (2024)
work page 2024
-
[6]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Guo, J., Dong, W., Huang, T., Ding, H., Wang, Z., Kuang, H., Dou, Q., Liu, Y.H.: Endo3r: Unified online reconstruction from dynamic monocular endoscopic video. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 170–180. Springer (2025)
work page 2025
-
[7]
International journal of computer assisted radiology and surgery (2023)
Hayoz, M., Hahne, C., Gallardo, M., Candinas, D., Kurmann, T., Allan, M., Sznit- man, R.: Learning how to robustly estimate camera pose in endoscopic videos. International journal of computer assisted radiology and surgery (2023)
work page 2023
-
[8]
In: International Conference on Medical Image Computing and Computer- Assisted Intervention
Hayoz, M., Hahne, C., Kurmann, T., Allan, M., Beldi, G., Candinas, D., Márquez- Neila, P., Sznitman, R.: Online 3d reconstruction and dense tracking in endoscopic videos. In: International Conference on Medical Image Computing and Computer- Assisted Intervention. pp. 444–454. Springer (2024)
work page 2024
-
[9]
In: 2025 IEEE International Conference on Robotics and Automation (ICRA)
Huang, Y., Cui, B., Bai, L., Chen, Z., Wu, J., Li, Z., Liu, H., Ren, H.: Advancing dense endoscopic reconstruction with gaussian splatting-driven surface normal- aware tracking and mapping. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 4084–4091. IEEE (2025)
work page 2025
-
[10]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Huang, Y., Cui, B., Bai, L., Guo, Z., Xu, M., Islam, M., Ren, H.: Endo-4dgs: Endo- scopic monocular scene reconstruction with 4d gaussian splatting. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 197–207. Springer (2024) 10 T. Song et al
work page 2024
-
[11]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Johari, M.M., Carta, C., Fleuret, F.: Eslam: Efficient dense slam system based on hybrid representation of signed distance fields. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 17408–17419 (2023)
work page 2023
-
[12]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Karaev, N., Makarov, Y., Wang, J., Neverova, N., Vedaldi, A., Rupprecht, C.: Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6013–6022 (2025)
work page 2025
-
[13]
In: Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention
Karaoglu, M.A., Ji, W., Abbas, A., Navab, N., Busam, B., Ladikos, A.: Litetracker: Leveraging temporal causality for accurate low-latency tissue tracking. In: Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention. pp. 308–317. Springer (2025)
work page 2025
- [14]
-
[15]
Commu- nications of the ACM65(1), 99–106 (2021)
Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. Commu- nications of the ACM65(1), 99–106 (2021)
work page 2021
-
[16]
In: Pro- ceedingsoftheIEEE/CVFWinterConferenceonApplicationsofComputerVision
Neoral, M., Šer` ych, J., Matas, J.: Mft: Long-term tracking of every pixel. In: Pro- ceedingsoftheIEEE/CVFWinterConferenceonApplicationsofComputerVision. pp. 6837–6847 (2024)
work page 2024
-
[17]
Point Tracking in Surgery--The 2024 Surgical Tattoos in Infrared (STIR) Challenge
Schmidt, A., Karaoglu, M.A., Sinha, S., Jang, M., Ha, H.G., Jung, K., Gu, K., Ullah, I., Lee, H., Šer` ych, J., et al.: Point tracking in surgery–the 2024 surgical tattoos in infrared (stir) challenge. arXiv preprint arXiv:2503.24306 (2025)
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[18]
Schmidt, A., Mohareri, O., DiMaio, S., Yip, M.C., Salcudean, S.E.: Tracking and mapping in medical computer vision: A review. Medical Image Analysis (2024)
work page 2024
-
[19]
In: IEEE confer- ence on computer vision and pattern recognition
Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: IEEE confer- ence on computer vision and pattern recognition. pp. 4104–4113 (2016)
work page 2016
-
[20]
In: International Workshop on Foundation Models for General Medical AI
Seenivasan, L., Cheng, J., Fang, J., Bai, J., Soberanis-Mukul, R.D., Mangulabnan, J.E., Vedula, S.S., Ishii, M., Hager, G., Taylor, R.H., et al.: Endotracker: Robustly tracking any point in endoscopic surgical scene. In: International Workshop on Foundation Models for General Medical AI. pp. 140–149. Springer (2025)
work page 2025
-
[21]
arXiv preprint arXiv:2602.18314 (2026)
Song, T., Stoyanov, D., Mazomenos, E., Vasconcelos, F.: Diff2dgs: Reliable re- construction of occluded surgical scenes via 2d gaussian splatting. arXiv preprint arXiv:2602.18314 (2026)
-
[22]
In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Taylor, J., Jepson, A.D., Kutulakos, K.N.: Non-rigid structure from locally-rigid motion. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 2761–2768. IEEE (2010)
work page 2010
-
[23]
In: European Conference on Computer Vision (ECCV)
Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: European Conference on Computer Vision (ECCV). pp. 402–419. Springer (2020)
work page 2020
-
[24]
In: 2025 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
Wang, K., Yang, C., Lin, R., Yang, X., Shen, W.: Endosd-slam: Real-time de- formable endoscopic slam via sparse-dense hybrid representation. In: 2025 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 4214–
work page 2025
-
[25]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Wang, K., Yang, C., Wang, Y., Li, S., Wang, Y., Dou, Q., Yang, X., Shen, W.: En- dogslam: Real-time dense reconstruction and tracking in endoscopic surgeries using gaussian splatting. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 219–229. Springer (2024)
work page 2024
-
[26]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Wen, B., Trepte, M., Aribido, J., Kautz, J., Gallo, O., Birchfield, S.: Foundation- stereo: Zero-shot stereo matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 5249–5260 (2025) Track2Map 11
work page 2025
-
[27]
In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion
Wu, T., Miao, Y., Li, Z., Zhao, H., Dang, K., Su, J., Yu, L., Li, H.: Endoflow-slam: Real-time endoscopic slam with flow-constrained gaussian splatting. In: Interna- tional Conference on Medical Image Computing and Computer-Assisted Interven- tion. pp. 202–212. Springer (2025)
work page 2025
-
[28]
In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Xu, M., Zhao, L., Huang, S., Hao, Q.: Active slam in 3d deformable environments. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 7952–7958. IEEE (2022)
work page 2022
-
[29]
Advances in Neural Information Processing Systems37, 21875–21911 (2024)
Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., Zhao, H.: Depth anything v2. Advances in Neural Information Processing Systems37, 21875–21911 (2024)
work page 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.