{"id":"1e158a5b-0eb0-40ba-9b87-70a147611117","arxiv_id":"2607.17810","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Vis2Reg uses mask-gated visible-domain self-supervision with differentiable rasterization to achieve landmark-free 3D-2D liver registration, improving Dice by 13.7 points over prior state-of-the-art.","lead":"Vis2Reg is a new registration method that aligns a 3D liver model to partial laparoscopic video frames using masks and visibility-aware self-supervision, with no 3D ground truth required. It reports 92.6% Dice and 1.43 mm Chamfer distance on the P2I-LReg benchmark, a large improvement over prior work.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Potential confound: Vis2Reg's Q_v is DepthAnything-derived while P2I-LReg provides sparse clouds, and Dice/CD are also the training losses; the 13.71pp gain over Self-P2IR may reflect input representation or metric-supervision overlap rather than the visibility-aware mechanism.","rationale":"The reader's weakest assumption focuses on mask/depth noise inside the visibility-aware supervision. My concern is related but shifted: the more load-bearing issue for the comparative claim is whether the input Q is actually the same for Vis2Reg and the baselines, and whether the reported metrics are independent of the training losses. The paper's own statement that each sample provides an 'intraoperative sparse point cloud' while the method reconstructs Q_v via DepthAnything makes input parity genuinely uncertain. Also, because Dice is explicitly optimized through L_sil and CD is essentially L3D, Table 1 may reflect fitting to the same supervision signal rather than improved geometric registration. This does not invalidate the method: the synthetic rigid-init results and the Table 2 ablations give some internal support for the design, and the loss terms are reasonable for self-supervised registration. But the central real-data claim is comparative and externally oriented, so it needs parity and an independent error measure. The reader's CONDITIONAL verdict already captures the need for such validation, so I do not recommend changing it; the stress-test sharpens the specific condition to check.","tokens_in":9251,"tokens_out":8906,"duration_ms":90961,"concrete_test":"Reproduce Table 1 under two controlled conditions: (i) all baselines and Vis2Reg consume the same merged Q, first the DepthAnything-derived cloud and then the dataset-provided sparse point cloud, with identical masks, intrinsics, and patient folds; (ii) on the synthetic GT split, compute a true surface-to-surface error (warped P vs. GT deformed mesh) alongside Dice/CD. If Vis2Reg's >13.71pp Dice and 1.54mm CD advantages over Self-P2IR persist under both Q representations and on the GT error, the concern is resolved; if the advantage shrinks or reverses, the reported real-data gain is an artifact of input representation or metric-supervision overlap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparative claim (Sec. 4) is that visibility-aware non-rigid supervision, not initialization, produces the 13.71pp Dice and 1.54mm CD gains over Self-P2IR on real data. Two conditions must hold: (A) all methods consume the same input observation Q; (B) the reported metrics measure registration quality rather than the training objectives. Condition A is not established. Sec. 2.1 says Q_v is reconstructed from laparoscopic frames with monocular DepthAnything, while Sec. 3.1 says P2I-LReg provides an 'intraoperative sparse point cloud.' If Vis2Reg's dense DepthAnything Q differs from the sparse cloud used by baselines, the comparison is confounded; Table 2's ablations keep Q fixed and cannot expose this. Condition B is also doubtful: Dice is the same BCE+Dice silhouette loss L_sil in Eq. 5, and the reported CD is the one-way Chamfer L3D minimized during training (Sec. 2.3); Sec. 3.1 itself states CD is 'not a target registration error.' Consequently, Table 1 numbers may measure consistency with the same masks and depth-derived cloud used for training, not anatomical accuracy. The Sec. 4 limitation (no internal ground truth) confirms real-data accuracy is unvalidated. The concern is not that the method is internally wrong but that the headline gain may reflect a different input representation or metric-supervision overlap rather than the proposed visibility-aware supervision.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Vis2Reg, a self-supervised, landmark-free framework for 3D–2D registration of a preoperative liver model to partial intraoperative laparoscopic observations. The method combines a GeoTransformer-based rigid-initialization module with a SIREN implicit deformation field, and introduces a visibility-aware self-supervision signal obtained by differentiable point rasterization and mask-gated back-projection. Training uses a one-way observation-to-model Chamfer loss, a symmetric visible-domain loss on mask-gated back-projected points, silhouette BCE+Dice, and deformation regularizers. On the P2I-LReg benchmark the authors report Dice 92.60±7.24% and Chamfer Distance 1.43±1.26 mm, outperforming Self-P2IR by 13.71 pp and 1.54 mm, with 111 ms per-frame inference, and they provide ablations supporting the visibility-gating and one-way-loss design choices.","tokens_in":9589,"tokens_out":5239,"duration_ms":46392,"significance":"If the reported improvements are genuine, Vis2Reg would be a useful contribution to AR-guided laparoscopic liver surgery: the visibility-aware self-supervision is well motivated, the pipeline is geometry-only at inference, and the two-stage synthetic-to-real training with a public benchmark and patient-level 5-fold protocol is a strength. The ablation study is internally consistent and shows that each proposed component contributes to the reported performance. However, the central comparative claim is currently weakened by two load-bearing issues: the real-data metrics overlap with the training objectives, and it is not established that all methods consume the same intraoperative input representation. Independent validation on a metric not used in training, and a precise statement of the input clouds used by each baseline, are needed before the headline gains can be attributed to the visibility-aware mechanism.","major_comments":[{"comment":"The central comparison in Table 1 may not hold the input observation fixed. §2.1 states that each intraoperative view Q_v is reconstructed from laparoscopic frames with monocular depth (DepthAnything), whereas §3.1 states that P2I-LReg provides \"an intraoperative sparse point cloud.\" If Vis2Reg consumes dense DepthAnything-derived clouds while the baselines consume the P2I-LReg sparse clouds, the reported 13.71 pp Dice gain and 1.54 mm CD reduction could reflect a change in input representation rather than the visibility-aware supervision. The paper should state exactly which clouds each method uses and, if the inputs differ, rerun all baselines on the same Q used by Vis2Reg, or provide an ablation with baselines on both representations.","section":"§2.1 and §3.1"},{"comment":"The two headline metrics are training objectives: Dice is the BCE+Dice silhouette term L_sil in Eq. (5), and the reported CD is the one-way Chamfer L_3D minimized during training. §3.1 itself states that CD is \"not a target registration error\" and §4 states that no internal ground truth is available. As a result, the real-data numbers in Table 1 measure how well the method optimizes its own losses, not anatomical registration accuracy. In addition, Eq. (5) defines CD with squared L2 distances, so the \"mm\" unit in Table 1 is inconsistent; reporting sqrt(CD) or a distance-based variant would be needed. I request an evaluation on an objective not used in training (e.g., projected landmark/vessel error, or synthetic deformations with real-style noise) before the headline improvement can be attributed to registration quality.","section":"§3.1, Eq. (5), Tables 1–2"},{"comment":"The synthetic rigid-initialization numbers are reported as single means (RRE 0.08°, RTE 0.26 mm) without standard deviations over the 5 folds or random seeds, which is insufficient to support the \"more reliable rigid seed\" claim. The GeoTransformer baseline is also run as a standalone matcher with closed-form pose estimation and no RANSAC or ICP, while Vis2Reg's rigid module uses mutual filtering, PROSAC, and trimmed ICP; the comparison therefore conflates the matcher with the robust pose-estimation pipeline. Reporting GeoTransformer with standard robust estimation (as in ref. [26]) and giving fold-wise statistics would make the rigid-init claim testable.","section":"§3.2, Table 1"},{"comment":"The visible-domain supervision is constructed entirely from the intraoperative mask M_v and the monocular-depth-derived cloud Q_v. The paper applies \"statistical denoising\" to Q_v but does not quantify depth noise or validate mask quality, and it assumes a near-static liver over the F=3-frame window. If the mask over- or under-segments the liver or DepthAnything is systematically biased on laparoscopic images, those errors propagate directly into L_vis and L_sil and thence into the reported Dice and CD. A sensitivity analysis (e.g., perturbing masks and depth by realistic corruptions, or reporting depth error against available structure) would establish robustness of the supervision signal.","section":"§2.3, §3.1"}],"minor_comments":[{"comment":"The symbol for the visible-domain supervision set is typeset inconsistently (V^M_v in the text vs. a differently formatted form in Fig. 1); please unify the notation.","section":"Fig. 1 and §2.3"},{"comment":"The abstract's \"111 ms per-frame inference time\" excludes depth and mask reconstruction, as noted in §3.2; the runtime including those steps would be informative for the near-real-time claim.","section":"§3.2"},{"comment":"The statement that Dice is the \"primary AR-overlay metric\" should clarify that it is a silhouette overlap between the rendered registered surface and the liver mask, not a volumetric or anatomical-target overlap.","section":"§3.1"},{"comment":"The choice of F=3 as a \"short local, non-temporal window corresponding to a near-static liver\" would benefit from a brief justification or citation, since camera and tissue motion are non-negligible in laparoscopy.","section":"§2.1"}],"recommendation":"major_revision","confidential_remarks":"The input-representation confound is the most serious issue; if the authors can demonstrate that all methods use identical Q, the paper may become publishable after adding an independent metric. The paper also does not state whether code or trained models will be released, which would materially strengthen reproducibility for a self-supervised registration method. I would not recommend reject at this stage, because the ablation design and benchmark protocol are sound and the missing experiments are within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take. The mask-gated, one-way visible-domain supervision is a real addition to the Self-P2IR line, and the ablations mostly back it. But don't trust the 13.7 pp Dice headline yet: the reported metric is the training loss, and the comparison may not share the same input cloud.\n\nWhat is genuinely new: Vis2Reg constructs a visible-domain 3D target by rasterizing its own warped model, gating with the intraoperative mask, back-projecting, and aligning that set to the observed cloud Q. That is different from Self-P2IR's symmetric rendered-mask consistency, and it is the right kind of idea for partial visibility. The paper checks components: removing L_vis or the gate costs 13-18 pp Dice, symmetric L_3D hurts, and a weak rigid seed hurts most. They also disclose that CD is not a target registration error and restrict claims to P2I-LReg. The 111 ms runtime is a real practical point.\n\nSoft spots, in order. First, the main real-data metrics are the same objectives minimized during training: Dice is the silhouette BCE+Dice loss, and CD is the one-way Chamfer in L_3D. On a held-out patient split that still measures generalization, but it measures fit to masks and to a depth-derived cloud, not anatomical accuracy. The paper says as much in Sec. 4; that should be front and center rather than a closing caveat. Second, and more important, the comparison may be unfair. Sec. 2.1 says Q_v is reconstructed with DepthAnything; Sec. 3.1 says the dataset provides an intraoperative sparse point cloud. If Vis2Reg uses the dense depth cloud while baselines use the provided sparse cloud, the 13.7 pp gain could come from input representation, not visibility-aware supervision. The claim that Self-P2IR already has a near-perfect rigid seed removes initialization as an explanation only if both methods see the same Q. That is not established. Mask quality and depth bias are also not quantified, and the gate can reinforce those errors. Third, the GeoTransformer baseline is run without RANSAC/ICP, so its poor numbers are expected; they disclose it, but comparing against a deliberately weak version of a method is not a strong competitor result. Fourth, no code or data release; minor.\n\nBottom line: the method is plausible, the writing is honest, and the ablation logic is sound. In its current form, the headline clinical-accuracy claim is unvalidated. This deserves a serious referee; I would send it out with a request for same-input experiments and an independent evaluation target, ideally landmark- or ground-truth-based, or at least a metric not used in training.","headline":"The mask-gated one-way visible-domain supervision is a genuine contribution and the ablations support it, but the 13.7 pp Dice headline is not yet trustworthy because the reported metric is also the training loss and the baselines may not share the same input cloud.","tokens_in":10135,"tokens_out":3666,"would_cite":true,"duration_ms":34424,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that visibility-aware, mask-gated self-supervision attains 92.60% Dice and 1.43 mm CD, beating the prior best by 13.71 points.","keywords":["3D-2D registration","laparoscopic liver surgery","visibility-aware self-supervision","point cloud registration","non-rigid deformation","differentiable rasterization","augmented reality surgery","partial-to-complete registration"],"falsifier":"Retrain and test on P2I-LReg with masks dilated or eroded by several pixels, or with monocular depth corrupted by calibrated noise, and check whether the 92.60% Dice and 1.43 mm CD survive; if the accuracy tracks mask or depth quality rather than the visibility-aware supervision design, the central claim would be refuted.","tokens_in":9049,"feed_emoji":"🏥","tokens_out":9660,"duration_ms":82225,"temperature":0.7,"pith_summary":"The paper tries to establish that deformable 3D-to-2D registration of a preoperative liver model to laparoscopic video can be trained without any 3D ground truth, provided the self-supervision is restricted to the geometry actually visible through the intraoperative mask. Prior landmark-free methods supervise against the whole preoperative surface, so they pull occluded regions toward observations that cannot exist; Vis2Reg instead rasterizes the warped model, keeps only pixels that fall inside the liver mask, and back-projects those pixels to form a visible-domain 3D target. On the public P2I-LReg benchmark it reports a 92.60% silhouette Dice and 1.43 mm Chamfer distance, improving the strongest prior method by 13.71 Dice points while running at 111 ms per frame. The clinical point is that accurate AR overlays for liver resection depend on this kind of visibility-correct supervision, because intraoperative liver views are mostly occluded and no paired 3D ground truth is available.","feed_headline":"Mask-gated supervision lifts liver-laparoscopy alignment to 92.6% Dice","feed_subtitle":"Trains on visible liver surface only, no 3D ground truth, and runs at 111 ms per frame.","key_machinery":"The load-bearing object is the mask-consistent visible-domain supervision set $V^M_v$, produced by differentiable point rasterization and mask-guided back-projection. For each of $F$ views, the warped preoperative cloud $\\hat{W}$ is rasterized into a depth map and a silhouette; the pixels where the rendered depth is positive and the intraoperative mask equals one are back-projected with the camera intrinsics to obtain explicit 3D points on the observable liver surface. These points enter a symmetric visible-domain Chamfer loss against the observed partial cloud, while a one-way observation-to-model Chamfer and silhouette BCE/Dice losses ensure unobserved model regions are never dragged toward missing data. The deformation itself is carried by an implicit displacement field $g_\\phi$, a multilayer perceptron with sinusoidal activations conditioned on pair features and a positional encoding, applied after a robust rigid initialization built from soft correspondences and hypothesis refinement.","core_discovery":"The paper's central claim is that mask-gated, one-way visible-domain supervision, not rigid initialization, is what resolves the partial-visibility failure mode in deformable liver registration. Starting from a near-perfect rigid seed, the strongest prior method still trails Vis2Reg by 13.71 Dice points, so the remaining gap must come from the non-rigid supervision; conversely, weakening the rigid seed inside Vis2Reg drops Dice to 69.32%, showing that the visibility-aware deformation field also needs a reliable starting pose. The specific mechanism is the mask-consistent visible set $V^M_v$ obtained by rendering the warped cloud into each view, intersecting rendered depth with the intraoperative mask $\\{u: \\hat{D}_v(u)>0 \\wedge M_v(u)=1\\}$, and back-projecting those pixels into 3D. Training then aligns this visible set to the observed points with a symmetric Chamfer term, penalizes unobserved model regions through a one-way observation-to-model Chamfer, and regularizes the implicit displacement field for magnitude, smoothness, and topology.","pith_inferences":["The same mask-gated visible-domain back-projection should transfer to other deformable partial-to-complete registration tasks where a segmentation mask and monocular depth are available, such as kidney or lung laparoscopy; the paper does not test this.","The reported 111 ms excludes depth and mask reconstruction, so a full intraoperative pipeline would add those costs; a deployment-ready timing benchmark would need to include them.","Because only surface overlap is measured, the paper does not establish that internal tumour or vessel geometry is registered; evaluating that would require internal anatomical ground truth, which the paper explicitly leaves to future work.","The $F=3$ near-static window sidesteps respiratory motion; extending the formulation to a temporal stream with explicit motion modeling could preserve accuracy during breathing, but that extension is not claimed here."],"forward_implications":["On the P2I-LReg benchmark, Vis2Reg reaches 92.60% Dice and 1.43 mm Chamfer distance, a 13.71-point Dice improvement over the strongest prior method, at 111 ms per frame.","At inference the model is geometry-only: masks and camera intrinsics are used to build the training signal but are not needed to warp a preoperative model to a new intraoperative view.","The ablation results imply that a correct rigid seed and the mask-gated visible-domain loss are both necessary; removing the visibility-aware loss or its mask gating costs 13 to 18 Dice points, while weakening the rigid seed is the largest single degradation.","A strong rigid seed by itself is insufficient, because the prior method already starts from a near-perfect seed and still trails by 13.71 Dice points.","Because the training supervision needs only masks, intrinsics, and monocular depth, the framework applies to settings where paired 3D ground truth for the liver is unavailable."],"supporting_citations":[{"why":"Defines the P2I-LReg benchmark and the Self-P2IR baseline whose 78.89% Dice and 2.97 mm CD serve as the improvement target for Vis2Reg's visibility-aware supervision.","marker":"[26]"},{"why":"Supplies the differentiable point rasterization used to render depth and silhouette from the warped cloud and to enable mask-guided back-projection.","marker":"[9]"},{"why":"Provides the monocular depth estimator whose back-projected output forms the intraoperative observation cloud $Q_v$.","marker":"[24]"},{"why":"Supplies the geometric transformer matching and soft assignment that produce the correspondence set feeding the rigid initialization.","marker":"[19]"},{"why":"Provides the dynamic-graph feature encoder that extracts the multi-scale geometric features used for matching and for conditioning the deformation field.","marker":"[23]"}],"fun_headline_variants":["Visibility-aware self-supervision lifts liver registration to 92.6% Dice","Mask-gated supervision enables liver alignment without ground truth: 92.6% Dice","Liver registration from partial views: 92.6% Dice via mask-gated supervision","Self-supervised liver registration: 92.6% Dice at 111 ms","Mask-consistent visible sets drive liver registration to 92.6% Dice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire training signal rests on the intraoperative liver mask and the monocular depth estimates being accurate enough that the mask-gated back-projection really captures the visible liver surface; if either is systematically biased on laparoscopic images, every loss term inherits that bias.","fun_headline_variants_meta":{"raw":{"variants":["Visibility-aware self-supervision lifts liver registration to 92.6% Dice","Mask-gated supervision enables liver alignment without ground truth: 92.6% Dice","Liver registration from partial views: 92.6% Dice via mask-gated supervision","Self-supervised liver registration: 92.6% Dice at 111 ms","Mask-consistent visible sets drive liver registration to 92.6% Dice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001992,"raw_usage":{"total_tokens":7791,"prompt_tokens":976,"completion_tokens":6815,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":6708}},"tokens_in":592,"tokens_out":6815,"duration_ms":42727,"temperature":1.0,"reasoning_tokens":6708,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:34:09.877448+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain and test on P2I-LReg with masks dilated or eroded by several pixels, or with monocular depth corrupted by calibrated noise, and check whether the 92.60% Dice and 1.43 mm CD survive; if the accuracy tracks mask or depth quality rather than the visibility-aware supervision design, the central claim would be refuted.","supporting_citations":[{"cited_title":"In: SIGGRAPH Asia 2020 Courses","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable point rasterization used to render depth and silhouette from the warped cloud and to enable mask-guided back-projection."}],"review_version":2}