{"id":"a01b662e-92a0-4f73-91fc-362f35b99804","arxiv_id":"2510.13310","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A fully GPU-native, PyTorch-based global Structure-from-Motion pipeline using sparse-aware Levenberg-Marquardt with optional metric depth priors reports ~8-40× speedups over COLMAP at comparable accuracy on several benchmarks.","lead":"InstantSfM is a Structure-from-Motion system that runs entirely on GPUs inside PyTorch, estimating camera poses and 3D points from images up to 40× faster than the standard COLMAP pipeline while keeping similar accuracy. It targets deep-learning 3D pipelines that need fast, scale-aware camera calibration for NeRF and 3D Gaussian Splatting.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tanks-and-Temples results contradict the 'comparable accuracy' half of the central claim; the deficit may stem from initialization, not the proposed optimization, but must be resolved.","rationale":"The reader's weakest-assumption analysis identifies the 'comparable accuracy' claim as the central risk. Table III shows InstantSfM below COLMAP/GLOMAP on every Tanks-and-Temples scene and threshold, directly contradicting the abstract. The paper's own explanation points to implementation differences in initialization (PyCeres vs. Ceres, OpenCV vs. PoseLib), not to the proposed sparse-LM optimization. This is precisely the kind of load-bearing concern that must be resolved: if the initialization fix closes the gap, the accuracy claim may hold; if not, the core optimization is at fault. The concrete test isolates the two stages. I agree with the reader's CONDITIONAL verdict, as the efficiency evidence is credible, the accuracy evidence is mixed, and the depth-prior contribution remains under-specified (no Jacobian equations; unclear depth definition). No verdict change is warranted beyond the already-conditional acceptance with the stated conditions.","tokens_in":13029,"tokens_out":6977,"duration_ms":60929,"concrete_test":"On the Tanks-and-Temples benchmark, keep InstantSfM's sparse LM (GP and BA) unchanged, but replace its Python/PyCeres ViewGraphCalibration and OpenCV relative-pose estimation with COLMAP's C++ Ceres/PoseLib initializations (i.e., feed COLMAP's initial model into InstantSfM). Recompute AUC@3/5/10/20. If the gap to COLMAP/GLOMAP shrinks to roughly 1-2 points, the deficit is initialization and potentially fixable; if the gap persists, the proposed optimization itself is less accurate, invalidating the 'comparable accuracy' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract; contribution bullet 2) asserts reconstruction accuracy 'comparable' to COLMAP/GLOMAP. Table III (Tanks and Temples) directly contradicts this: InstantSfM is below COLMAP and GLOMAP on every scene and threshold, with average AUC@5 75.63 vs. 78.48/78.61, and Playground collapses to AUC@3=14.00 vs. 26.76. The authors' own explanation (Sec. V.A) attributes this to using PyCeres for ViewGraphCalibration and OpenCV for relative pose estimation, rather than C++ Ceres/PoseLib. If that attribution is correct, the deficit is an implementation artifact that could be fixed, and the 'comparable accuracy' claim might be salvageable for the optimization core. However, as presented, the actual end-to-end system does not deliver comparable accuracy on a standard SfM benchmark. A secondary but related concern: the depth-prior mechanism (Sec. IV.C, Eq. 4) is under-specified; no Jacobian equations are given, and it is unclear whether depth_ij is Euclidean ray distance or z-depth, which affects metric-scale correctness. This does not weaken the T&T concern but compounds the lack of evidence for the metric-scale contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents InstantSfM, a GPU-native Structure-from-Motion pipeline implemented in PyTorch. It applies sparse-aware Levenberg-Marquardt optimization to both global positioning (GP) and bundle adjustment (BA), and optionally injects metric depth priors into the optimization to recover metric-scale reconstructions. The authors claim state-of-the-art efficiency—up to ~40x speedup over COLMAP on large scenes—and reconstruction accuracy comparable to established classical pipelines (COLMAP, GLOMAP) and learning-based methods (VGGSfM). Experiments span MipNeRF360, DTU, Tanks and Temples, ScanNet, ScanNet++, plus runtime benchmarks on BAL and 1DSfM. The efficiency results are strong, but the accuracy comparison is mixed: on Tanks and Temples the proposed method is consistently below COLMAP and GLOMAP on every scene and threshold, while on MipNeRF360 and DTU downstream rendering metrics are more favorable. The paper also introduces a depth-prior mechanism (Eq. 4) whose definition and Jacobian structure are under-specified.","tokens_in":13245,"tokens_out":4601,"duration_ms":40346,"significance":"If the efficiency results hold, the paper makes a practically important contribution: a sparse, GPU-native, PyTorch-compatible SfM pipeline that handles thousands of images on a single GPU, removing the CPU bottleneck of COLMAP/GLOMAP. The runtime comparisons in Tables VI and VII and Fig. 3 show order-of-magnitude speedups over COLMAP on large scenes, and the ScanNet experiments demonstrate robustness in cases where COLMAP and GLOMAP fail. The depth-prior ablation is a useful step toward metric-scale SfM. However, the central 'comparable accuracy' claim is not uniformly supported: Table III shows clear end-to-end accuracy deficits on Tanks and Temples, and the depth-prior formulation is not fully specified. The paper's value is therefore currently concentrated in the efficiency/robustness results rather than the accuracy-equivalence claim.","major_comments":[{"comment":"The abstract claims reconstruction accuracy 'comparable' to COLMAP/GLOMAP. Table III contradicts this on Tanks and Temples: InstantSfM is below COLMAP and GLOMAP on every scene and every threshold (e.g., average AUC@5 = 75.63 vs. 78.48 for COLMAP and 78.61 for GLOMAP; Playground AUC@3 = 14.00 vs. 26.76). The authors attribute the deficit to using PyCeres for ViewGraphCalibration and OpenCV in Python for relative pose estimation. This attribution must be validated: please run the same pipeline with reference C++ Ceres/PoseLib front-ends (or with identical initializations) to show that the proposed GP/BA optimization core is not the cause, or revise the accuracy claim to describe the actual end-to-end system. As presented, the system does not deliver the claimed accuracy equivalence.","section":"§V.A, Table III; Abstract"},{"comment":"The depth-prior formulation is under-specified. Eq. (4) writes u_ij = v_ij - (X_j - t_i)/depth_ij, but the manuscript does not define depth_ij (Euclidean ray distance vs. z-depth from the depth map) nor the meaning of v_ij (unit ray direction, normalized pixel direction, or direction including intrinsics). If depth_ij is z-depth, the relationship (X_j - t_i)/depth_ij is not the same as the ray direction v_ij, and the residual in metric units is not well posed. The claim that the depth prior 'resolve[s] scale ambiguity' depends on this definition. Please give precise definitions and the depth-constrained Jacobian blocks for both the GP and BA stages.","section":"§IV.C, Eq. (4)"},{"comment":"On MipNeRF360 and DTU the accuracy evaluation is indirect: the paper reports only 3DGS rendering metrics (PSNR/SSIM/LPIPS), not camera pose or 3D point accuracy. Rendering quality can mask systematic SfM errors, and the unusual averages in Table I (e.g., Ours average PSNR 28.43 vs. COLMAP 26.92, driven by large drops in several COLMAP scenes) suggest evaluation protocol differences that are not explained. Please report direct SfM metrics (pose AUC or rotation/translation errors, triangulation reprojection errors) for these datasets, or at minimum explain why COLMAP/GLOMAP scores differ from their published behavior on the same data.","section":"§V.A, Tables I–II"},{"comment":"The ScanNet and ScanNet++ depth-prior experiments use ground-truth depth from RGB-D sensors. This is an oracle external prior, so comparing 'Ours (w/ depth)' against COLMAP/GLOMAP (which receive no such prior) is not a fair comparison of SfM accuracy. The 'Ours (w/o depth)' ablation is useful, but the paper's statement that depth prior enables metric-scale reconstruction 'significantly outperform[ing] compared methods' should be clearly framed as a controlled study with an external oracle, not as a property of the SfM system alone. Please also report the metric-scale error with and without the depth prior explicitly, rather than only Chamfer distance.","section":"§V.A, Tables IV–V"}],"minor_comments":[{"comment":"The notation v_ij is described as 'pixel ray direction vectors' but the equation uses 2D feature points inconsistently. Please clarify the exact definition and the dimension of v_ij (unit vector vs. pixel coordinate).","section":"§III.A, Eq. (1)"},{"comment":"The sentence 'as shown in .' has an incomplete cross-reference; the algorithm should be cited as 'in Algorithm 1'.","section":"§IV, Algorithm 1"},{"comment":"The runtime plots are in log space, which obscures the actual speedup factors. Please include numeric labels or a table with exact timings for the key comparisons.","section":"§V.B, Fig. 3 and Fig. 4"},{"comment":"The acronym 'PBA' is not defined. Please expand it on first use, and clarify the hardware/software versions for Ceres-GPU and PBA.","section":"§V.B, Table VI"},{"comment":"Reference [26] (GLOMAP) is a key baseline but appears as a 2024 ECCV paper; the text uses GLOMAP in multiple places without a companion citation to the original arXiv version. Also, the related-work discussion of [15] (the sparse BA framework the method builds on) could more clearly state which components are inherited from [15] and which are new to this paper.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's central optimization machinery is explicitly built on the authors' own prior work [15] (arXiv:2409.12190, 'Bundle adjustment in the eager mode'). The incremental content here is the extension to global positioning, the depth-prior mechanism, and the full system integration. The overlap is substantial enough that the editor should verify that [15] is either published or that its status does not create a dual-submission concern. The efficiency and robustness results are strong and likely sufficient for the paper's core value, but the 'comparable accuracy' claim needs either a more careful, benchmark-specific qualification or a fix to the front-end components before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The efficiency story is real and the paper deserves serious refereeing, but the 'comparable accuracy' claim needs to be walked back or fixed. The genuinely new piece is shipping a complete global SfM pipeline in PyTorch by extending sparse-aware LM from BA (their own [15]) to global positioning, plus the depth-constrained Jacobian for metric-scale recovery. That integration is useful and, if the code ships, it will be a practical drop-in for 3DGS/NeRF pipelines. The runtime tables (VI, VII) are credible: consistent large speedups over COLMAP and GLOMAP, and scaling to thousands of images on one GPU where learned methods OOM. Credit where due: the sparse matrix machinery is standard but the engineering is nontrivial, and they are transparent about borrowing the GP objective from GLOMAP and the sparse ops from [15].\n\nNow the soft spots. The Tanks and Temples results (Table III) directly contradict the abstract's 'comparable accuracy': InstantSfM is below COLMAP and GLOMAP on every scene and threshold, and Playground collapses to AUC@3=14.0 vs 26.8. The authors' explanation—PyCeres and OpenCV in Python for initial calibration—is plausible and means the deficit may be fixable, but as presented the end-to-end system does not deliver comparable accuracy on that benchmark. On MipNeRF360 and DTU the averages look fine, but those are NVS metrics after 3DGS training, which can wash out pose errors, and the averages mask competitor collapses (e.g., GLOMAP's kitchen at PSNR 16.1). No error bars anywhere, and some table footnotes (COLMAP untested on one ScanNet scene) make comparisons uneven. The depth prior section is thin: no Jacobian equations, and it is ambiguous whether depth is Euclidean ray distance or z-depth, which matters for metric scale. That is a minor omission but should be clarified.\n\nOverall: the efficiency half of the central claim is solid; the accuracy half is overstated. The paper is a strong systems contribution, not a theoretical one. It needs code release, a fairer accuracy comparison, and a toned-down abstract. I would send it to peer review and let the reviewers pressure them on the details. It will be useful to practitioners even if the accuracy gap never fully closes, because the speedup is real and many pipelines just need a good enough initialization.","headline":"A credible GPU-native SfM efficiency win with an accuracy claim that overshoots on Tanks and Temples; worth refereeing with conditions.","tokens_in":13907,"tokens_out":1171,"would_cite":true,"duration_ms":11455,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InstantSfM claims a fully GPU-native, PyTorch-compatible global Structure-from-Motion pipeline that runs up to 40x faster than COLMAP while keeping comparable accuracy.","keywords":["Structure-from-Motion","GPU","Bundle Adjustment","Global Positioning","Sparse Optimization","PyTorch","Depth Prior","Levenberg-Marquardt"],"falsifier":"Run InstantSfM and COLMAP on the same 300-frame Tanks-and-Temples scenes and compare relative-pose AUC; if InstantSfM's Playground AUC@5 stays below COLMAP's by more than a few points, the 'comparable accuracy' claim, as measured on that benchmark, fails even if the speedup holds.","tokens_in":12812,"feed_emoji":"⚡","tokens_out":3798,"duration_ms":29481,"temperature":0.7,"pith_summary":"InstantSfM proposes to rebuild the entire Structure-from-Motion pipeline on the GPU, using sparse-aware Levenberg-Marquardt optimization natively in PyTorch for both bundle adjustment and global positioning. The paper claims this removes the CPU bottleneck that limits classical SfM systems and allows thousands of images to be processed on a single GPU, with up to ~40x speedup over COLMAP. It also injects metric depth priors directly into the optimization via the Jacobian, resolving scale ambiguity so that cameras and point clouds come out at metric scale. If correct, this would make SfM a fast, flexible, differentiable component for deep learning and 3D reconstruction workflows.","feed_headline":"GPU-native SfM runs up to 40x faster than COLMAP","feed_subtitle":"A PyTorch pipeline with metric depth priors claims to scale to thousands of images.","key_machinery":"The machinery is a sparse Jacobian representation combined with Levenberg-Marquardt steps whose expensive operations — JTJ, JT r, diagonal fetching, and the linear solve — are implemented with GPU sparse-matrix primitives. Depth priors enter by replacing the per-observation scale variable with 1/depth, so the Jacobian blocks gain depth-constraint terms. Explicit masking of under-constrained variables keeps the linear system nonsingular.","core_discovery":"The central claim is that a complete global SfM system can be implemented entirely on the GPU and in PyTorch without sacrificing reconstruction quality. The authors build on sparse-aware bundle adjustment, extend it to global positioning, and add a depth-constrained Jacobian structure that lets metric depth priors resolve scale ambiguity within the optimization itself. They report up to ~40x speedup over COLMAP on large-scale scenes, comparable accuracy on several benchmarks, and robustness on indoor RGB-D scenes where COLMAP and GLOMAP often fail.","pith_inferences":["If the approach is adopted, SfM could become differentiable end-to-end, enabling joint training of feature extraction and geometric optimization.","The depth-prior trick suggests a general way to inject metric information into any scale-ambiguous geometric optimization, potentially generalizing to SLAM or multi-view stereo.","The reported accuracy gap on Tanks-and-Temples indicates initialization quality is the bottleneck; a drop-in replacement of PyCeres/OpenCV with C++-equivalent components would test that hypothesis."],"forward_implications":["SfM can scale to thousands of images on a single GPU, where learning-based pipelines like VGGSfM run out of memory.","Learning-based feature descriptors can be used directly in Python, removing the C++ conversion step.","Supplying monocular or sensor depth gives metric-scale camera poses and point clouds, useful for robotics and simulation.","The sparse-aware optimization approach may accelerate other large-scale non-linear least squares problems beyond SfM."],"fun_headline_variants":["PyTorch SfM hits 40x GPU speedup","GPU-native SfM: 40x faster than COLMAP","SfM on GPU: 40x speedup, no quality loss","InstantSfM: full SfM on GPU, 40x faster","Depth-prior GPU SfM beats COLMAP 40x"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The accuracy comparison assumes that Python-based view-graph calibration and OpenCV relative-pose estimation produce camera initializations as good as COLMAP/GLOMAP's C++ Ceres/PoseLib pipeline; the paper's own Tanks-and-Temples results show it sometimes does not.","fun_headline_variants_meta":{"raw":{"variants":["PyTorch SfM hits 40x GPU speedup","GPU-native SfM: 40x faster than COLMAP","SfM on GPU: 40x speedup, no quality loss","InstantSfM: full SfM on GPU, 40x faster","Depth-prior GPU SfM beats COLMAP 40x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1101,"prompt_tokens":761,"completion_tokens":340,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":256}},"tokens_in":505,"tokens_out":340,"duration_ms":3083,"temperature":1.0,"reasoning_tokens":256,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T09:46:31.419156+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run InstantSfM and COLMAP on the same 300-frame Tanks-and-Temples scenes and compare relative-pose AUC; if InstantSfM's Playground AUC@5 stays below COLMAP's by more than a few points, the 'comparable accuracy' claim, as measured on that benchmark, fails even if the speedup holds.","supporting_citations":[],"review_version":1}