Pith. sign in

REVIEW 3 major objections 5 minor 33 references

The paper claims that restricting each Schur-LM camera update to a low-dimensional subspace selected by geometry-aware gating stabilizes bundle adjustment under low parallax, lifting relative-pose accuracy from ~13% to ~80% on near-spherica

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:40 UTC pith:3VKATNAA

load-bearing objection Plausible solver-side adaptation of CSS to Schur-reduced BA with large PhoneSweep gains, but the unspecified hybrid fallback and missing artifacts keep the attribution unverified. the 3 major comments →

arxiv 2607.15652 v1 pith:3VKATNAA submitted 2026-07-17 cs.CV

CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment

Ayano Kaneda , Takafumi Taketomi , Shugo Yamaguchi , Shigeo Morishima This is my paper
classification cs.CV
keywords bundle adjustmentSchur complementLevenberg-Marquardtcolumn space searchlow parallaxtrust regioncamera pose estimationfocal calibration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Bundle adjustment in low-parallax or near-rotational scenes is unreliable: the Schur-reduced camera system becomes ill-conditioned, so standard Levenberg–Marquardt steps move along poorly supported directions and can produce accurate reprojection but wrong poses. The paper's central claim is that this can be fixed without touching the optimization objective, the variables, or the trust-region rule—only by restricting each camera-update direction to a low-dimensional subspace built from camera blocks that pass geometric reliability gates. A gate filters cameras by parallax, view connectivity, and rotation agreement; Column Space Search then scores the eligible blocks by their local predicted decrease in the damped Schur system, selects a top-k set, and constructs a compact basis for the projected LM step. On a near-spherical phone dataset, this raised a joint relative-pose accuracy metric (AUC@30) from about 13 to 80 on one device and from 12 to 84 on another, while keeping objective reduction comparable on large-scale Internet-photo benchmarks. A sympathetic reader would care because the method is a drop-in solver replacement: it promises substantially better pose reliability in degenerate geometry without sacrificing reprojection quality or changing the estimation problem.

Core claim

The paper's claim is that the instability of Schur-based Levenberg–Marquardt under weak geometry is a property of the full-dimensional camera update, not the objective. The method keeps the standard damped Schur system and trust-region rule but parameterizes each camera increment as Δc = V_t y_t with a low-rank basis built per iteration: a geometry gate (parallax, rotation agreement, view connectivity) filters camera blocks, Column Space Search scores the eligible blocks by single-block predicted decrease in the damped Schur system, selects top-k, and runs Lanczos on the block-localized operator to get Ritz vectors plus one complement direction. Solving the projected system and lifting the i

What carries the argument

The central mechanism is a gate-then-score subspace selector inside the Schur-LM trust-region loop. A precomputed geometry-aware support S keeps cameras that satisfy parallax, view-connectivity, and rotation-agreement thresholds. At each LM iteration, Column Space Search scores each block i in S by the local Schur predicted reduction s_i = ½ g_i^T H_ii^{-1} g_i, picks the top-k blocks K_t, and forms a low-dimensional basis: m Lanczos steps on the block-localized operator P_K H_λ P_K, taking the top-k Ritz vectors, plus one normalized complement direction from the residual outside the support. The projected damped Schur system is solved in this basis and the update is lifted to the full camer

Load-bearing premise

The central claim depends on the unstated guarantee that the prediction safeguards and hybrid fallback never alter ordinary successful iterations—no trigger details are given, so the improvements cannot be isolated from the fallback path.

What would settle it

Rerun the PhoneSweep experiments with the prediction safeguards and hybrid fallback disabled, and instrument the solver to count every fallback activation. If the accuracy gains persist with safeguards off, the subspace restriction is the active ingredient; if they disappear, the gains come from the fallback rather than the gated basis.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Existing Schur-LM BA implementations can adopt the method as a drop-in replacement: only the computation of Δc changes; the objective, residuals, Jacobians, damping, and trust-region rule stay exactly as in the baseline.
  • On the near-spherical phone dataset (PhoneSweep), the authors report AUC@30 of 80–84 for CSS-BA versus 12–13 for standard Schur-LM and 33–43 for a power-series-based solver, on two devices.
  • On the large-scale BAL benchmark, CSS-BA achieves objective reduction comparable to the baselines, indicating the subspace restriction does not severely hurt well-conditioned problems, though gains there are modest.
  • In the hardest tested sequences, the method also stabilizes focal-length estimation, reducing absolute focal error from hundreds of percent to under 1% on one device.
  • Because it restricts only the update direction, the method is compatible with robust losses and existing outlier-handling stages in BA pipelines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Disabling or logging the prediction safeguards and hybrid fallback would directly test whether the reported gains come from the subspace restriction or from an unstated fallback path; the paper gives no such diagnostic.
  • The gate thresholds are fixed heuristics; a learned or adaptive gate tuning parallax and rotation thresholds per scene could broaden applicability beyond the tested regimes.
  • Because the subspace restriction is solver-side and problem-agnostic, the same gate-and-search scheme could transfer to other block-structured nonlinear least-squares problems with weak observability, such as SLAM loop closure or multiview extrinsic calibration.
  • The reported runtime overhead (about 2x standard LM and ~6.8x the power-series baseline) suggests using the method selectively—after a coarse global initialization or as a conditional fallback when conditioning diagnostics flag low parallax.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Gate-Guided CSS-BA, a solver-side modification of Schur-reduced Levenberg–Marquardt bundle adjustment. Rather than changing the BA objective, residuals, or variable set, it restricts each camera update to a low-dimensional subspace built from a gated, CSS-scored top-k set of camera blocks. The subspace is formed by a localized Lanczos/Ritz basis plus one complement direction, and the projected LM step is then lifted to the full camera space. Experiments on PhoneSweep report large relative-pose accuracy gains over Normal-LM and PoBA under low-parallax, near-rotational motion, with AFE improvements on Nexus5X but not uniformly on iPhone13Mini; BAL results show similar objective reduction to baselines. The paper claims the method is a drop-in replacement that stabilizes the Schur-LM update in weak-geometry regimes.

Significance. If the reported gains are real and attributable to the subspace restriction, the paper makes a useful conceptual contribution: it separates the solver's update-space design from the estimation objective and shows that restricting update directions can stabilize weak-geometry BA without altering the problem. The method is clearly positioned relative to keyframe/state-selection approaches, and the main pipeline description (Algorithm 1, up to the fallback note) is coherent. The claim that the CSS scoring is not circular is reasonable, since it uses the same Schur model as the solve but only as a heuristic ranking criterion. However, the central experimental claim is currently supported only by aggregate numbers from an unreleased implementation, with a key algorithmic component (prediction safeguards and hybrid fallback) left unspecified. The absence of code/data and variance information, together with the vague fallback description, prevents independent verification of the attribution.

major comments (3)
  1. [Algorithm 1, line 25; Sec. E] The central attribution of the PhoneSweep gains to the CSS subspace restriction is not verifiable. Algorithm 1 line 25 defers 'prediction safeguards and hybrid fallback' to Sec. E, and Sec. E only states that these mechanisms are 'triggered only when the projected CSS step fails to provide a sufficiently reliable predicted decrease' without defining the reliability criterion, the fallback update, or the trigger frequency. Since the text says 'Unless otherwise noted, the runtime includes prediction safeguards and a hybrid fallback,' every reported result may include fallback behavior. If the fallback fires preferentially in low-parallax iterations, Table 1 compares a hybrid solver to vanilla LM, not the proposed subspace restriction. Please specify the trigger condition, the fallback step, per-dataset trigger counts, and provide an ablation with the fallback disabled.
  2. [Sec. 4, Tables 1, 2, 5] The empirical evidence is reported only as aggregate means without variance or per-sequence breakdowns, and no code or data are released. Table 1 shows a single AUC@30 per device, Table 2 family-wise means, and Table 5 reports two unnamed 'Seq. A/B' cases. The large PhoneSweep margins (AUC@30 about 80 versus 13 and 43) cannot be checked for per-sequence variability or for possible selection bias. Please include per-sequence results or standard deviations, identify the runtime sequences, and release the solver code or a complete pseudo-code including the fallback. Without this, the claimed reproducibility as a 'drop-in replacement' is not established.
  3. [Sec. 4.3 and Sec. B, Table A.1] The method depends on several gate thresholds that are adapted per dataset or per device. The main comparison uses τ_ra=8° for PhoneSweep and τ_ra=20° for BAL; the supplementary SphericalSfM experiment further changes τ_par^cam per device (1.8 vs. 0.8) and τ_ra=7.1°. The paper does not report sensitivity of Table 1 to these thresholds. If the large gains appear only within narrow threshold bands, the 'drop-in replacement' claim is weakened. Please add a threshold-sensitivity study over τ_ra, τ_par^cam, and τ_nbr, showing that the reported results are not a consequence of per-dataset tuning.
minor comments (5)
  1. [Throughout] Several table captions contain a typo: 'T able 1' etc. should be 'Table 1'.
  2. [Table 1] The text describes CSS-BA focal calibration as 'competitive but dataset-dependent,' yet on iPhone13Mini CSS-BA AFE is 158.01 vs. 115.56 for Normal-LM and 169.43 for PoBA. Please qualify the calibration claim with this direct comparison.
  3. [Sec. 4.3] The concrete values of τ_shared, τ_edge_par, and ε_g are not reported, although they affect the support construction and the complement-direction threshold. List all default hyperparameter values in one place.
  4. [Eq. (14)] The formula uses H~_{λ,ii}^{-1}, which may be singular for degenerate single-camera blocks. State whether a pseudo-inverse or a damped diagonal is used in the implementation.
  5. [Table 5] The two timed sequences are unnamed; wall-clock under a fixed 150-iteration cap may not reflect convergence behavior. Report timestamps for converged runs and name the sequences.

Circularity Check

0 steps flagged

No significant circularity: the projected Schur-LM step is an algebraic restriction, and reported gains are externally evaluated against ground truth.

full rationale

The paper's central step is Eq. (5), obtained by substituting Δc = V_t y into the damped Schur system (3). This is a standard algebraic projection and does not presuppose the reported accuracy gains. The CSS score (Eq. 14) is a per-block quadratic model decrease used only as a basis-selection heuristic; it is not later reported as an empirical prediction, and the evaluation metrics (RRA/RTA/AUC/AFE) are defined from ground-truth poses independently of internal scores. Gate thresholds are preprocessing parameters tuned per dataset, not fitted outputs, so this is a tuning/soundness concern rather than a fitted-input-called-prediction issue. There is no load-bearing self-citation: CSS is attributed to external work [8] and the solver builds on the external PoBA codebase [27]. The one flagged limitation is Algorithm 1 line 25 / Sec. E, where a hybrid fallback is asserted to trigger 'only when the projected CSS step fails to provide a sufficiently reliable predicted decrease' without specifying the trigger predicate, fallback step, or trigger counts. This is an attribution/soundness limitation for the experiments, not a circular reduction: the fallback is not defined in terms of the reported accuracy metric, and no equation equates the CSS step to the fallback. Overall, the derivation chain is self-contained and the empirical claims are not forced by construction.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central claim rests on standard BA linear algebra plus a set of hand-tuned gate thresholds and an unspecified fallback. No new physical entity is introduced. The gate thresholds are adjusted per dataset/device, which weakens the claim that the method is parameter-free.

free parameters (6)
  • k_max (top-k budget) = 10
    Default in all experiments; Table 4 shows little sensitivity over k=4..16, but it is a hand-set hyperparameter.
  • m (Lanczos iterations) = 32
    Hand-set; no ablation of m is reported.
  • tau_ra (rotation-agreement gate) = 8.0/20/7.1 deg by regime
    Main text says only the rotation gate is adapted to motion regime; device-specific values are used in Supp. B.
  • tau_par_cam (camera-level parallax gate) = 2.0 deg; 1.8/0.8 deg by device in Supp. B
    Hand-set and device-dependent in the supplementary SphericalSfM experiment.
  • tau_nbr (min filtered neighbors) = 2
    Fixed across datasets.
  • tau_shared, tau_edge_par, epsilon_g = not reported
    Thresholds in Eqs. (7), (10), and the z_g complement test are referenced but concrete values are never given.
axioms (4)
  • standard math Schur complement and damped normal equations (Eqs. 2-3) are a valid linearization for BA around the current estimate.
    Standard BA theory; the method does not derive or modify it.
  • domain assumption Parallax, rotation agreement, and view connectivity computed from current estimated poses are reliable proxies for the conditioning of the damped Schur camera system.
    Entry into support S in Eq. (13); if noisy estimates violate this, the gate may exclude useful directions.
  • domain assumption The top-k Ritz vectors of the localized Schur operator approximate the dominant, well-conditioned update directions.
    Sec. 3.3; no convergence guarantee tied to BA weak geometry is provided.
  • ad hoc to paper The unspecified prediction safeguards/hybrid fallback do not alter the effective update on successful iterations.
    Asserted in Sec. E but no mechanism or values are provided; this is load-bearing for attributing gains.

pith-pipeline@v1.3.0-alltime-deepseek · 13873 in / 13250 out tokens · 122233 ms · 2026-08-01T22:40:00.555854+00:00 · methodology

0 comments
read the original abstract

Bundle adjustment (BA) remains a critical refinement module for image-based 3D reconstruction and continues to improve geometric accuracy even in learning-based pipelines. However, in low-parallax and near-rotational regimes, classical Schur-based Levenberg--Marquardt (LM) often becomes ill-conditioned and yields unreliable pose and calibration estimates. We propose Gate-Guided CSS-BA, a solver-side modification of Schur-LM that preserves the classical BA objective and trust-region framework while constraining each update to a geometrically informed low-dimensional subspace. By integrating Column Space Search (CSS) with geometry-aware gating, the method stabilizes the Schur-LM update without altering the estimation problem. In contrast to keyframe or state-selection approaches, all camera and point parameters remain in the optimization problem; only the update direction is restricted. The method serves as a drop-in replacement for existing BA pipelines. Experiments on both generic and challenging weak-geometry scenarios show more stable optimization, improved relative pose accuracy, and competitive calibration behavior while maintaining reprojection quality.

Figures

Figures reproduced from arXiv: 2607.15652 by Ayano Kaneda, Shigeo Morishima, Shugo Yamaguchi, Takafumi Taketomi.

Figure 1
Figure 1. Figure 1: Comparison between standard BA and the proposed CSS-BA on the damped Schur camera system. Standard BA directly solves the full damped Schur system to obtain a full-dimensional camera update. In contrast, CSS-BA first selects a top-k camera-block support Kt, constructs a low-dimensional Lanczos/Ritz basis on the lo￾calized Schur operator, and solves the projected LM system in this subspace. The resulting st… view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative comparison of reconstructed camera trajectories on two PhoneSweep sequences: Nexus5X/engineering_west (top) and iPhone13Mini/scott_shand (bottom). Ground-truth camera layouts exhibit the characteristic near-spherical circular trajectory of PhoneSweep. Normal-LM and PoBA often fail to preserve this geometry, producing collapsed, fragmented, or strongly distorted camera configurations. In contras… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 4 linked inside Pith

  1. [1]

    Agarwal, S., Mierle, K., Team, T.C.S.: Ceres solver.http://ceres-solver.org (2022), accessed: 2026-03-01

    2022

  2. [2]

    Agarwal, S., Snavely, N., Seitz, S.M., Szeliski, R.: Bundle adjustment in the large. In: Eur. Conf. Comput. Vis. (2010)

    2010

  3. [3]

    Chui, J., Cremers, D.: Proba: Probabilistic bundle adjustment with the bhat- tacharyya coefficient (2025), arXiv preprint arXiv:2505.20858

    Pith/arXiv arXiv 2025

  4. [4]

    In: IEEE Conf

    Demmel, N., Sommer, C., Cremers, D., Usenko, V.: Square root bundle adjustment for large-scale reconstruction. In: IEEE Conf. Comput. Vis. Pattern Recog. (2021)

    2021

  5. [5]

    In: Proceedings of International Conference on 3D Vision (3DV)

    Duisterhof, B.P., Zust, L., Weinzaepfel, P., Leroy, V., Cabon, Y., Revaud, J.: MASt3R-SfM: A fully-integrated solution for unconstrained structure-from- motion. In: Proceedings of International Conference on 3D Vision (3DV). pp. 1–10 (2025)

    2025

  6. [6]

    Hong, J.H., Zach, C., Fitzgibbon, A., Cipolla, R.: Projective bundle adjustment from arbitrary initialization using the variable projection method. In: Eur. Conf. Comput. Vis. Lecture Notes in Computer Science, vol. 9905, pp. 477–493 (2016)

    2016

  7. [7]

    In: IEEE Conf

    Hong, J.H., Zach, C., Fitzgibbon, A., Cipolla, R.: pOSE: Pseudo object space error for initialization-free bundle adjustment. In: IEEE Conf. Comput. Vis. Pattern Recog. (2018)

    2018

  8. [8]

    Journal of Computational Physics443, 110498 (2021)

    Hyde, D.A.B., Bao, M., Fedkiw, R.: On obtaining sparse semantic solutions for inverse problems, control, and neural network training. Journal of Computational Physics443, 110498 (2021)

    2021

  9. [9]

    In: IEEE Conf

    Iglesias, J.P., Nilsson, A., Olsson, C.: expOSE: Accurate initialization-free projec- tive factorization using exponential regularization. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023) 16 A. Kaneda et al

    2023

  10. [10]

    The International Jour- nal of Robotics Research31(2), 216–235 (2012)

    Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J.J., Dellaert, F.: iSAM2: Incremental smoothing and mapping using the bayes tree. The International Jour- nal of Robotics Research31(2), 216–235 (2012)

    2012

  11. [11]

    Journal of Computational and Applied Mathematics172, 375–397 (2004)

    Kanzow, C., Yamashita, N., Fukushima, M.: Levenberg–Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints. Journal of Computational and Applied Mathematics172, 375–397 (2004)

    2004

  12. [12]

    In: ICRA (2011)

    Kümmerle, R., Grisetti,G., Strasdat,H., Konolige,K., Burgard,W.: g2o: Ageneral framework for graph optimization. In: ICRA (2011)

    2011

  13. [13]

    In: IEEE Conf

    Liao, B., Qu, D., Xue, Y., Zhang, H., Lao, Y.: Revisiting rolling shutter bundle ad- justment: Toward accurate and fast solution. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

    2023

  14. [14]

    ACM Transactions on Mathematical Software36(1) (2009)

    Lourakis,M.I.A.,Argyros,A.A.:SBA:Asoftwarepackageforgenericsparsebundle adjustment. ACM Transactions on Mathematical Software36(1) (2009)

    2009

  15. [15]

    Ni, K., Steedly, D., Dellaert, F.: Out-of-core bundle adjustment for large-scale 3D reconstruction. In: Int. Conf. Comput. Vis. (2007)

    2007

  16. [16]

    Olsson, C., Nilsson, A.: Towards initialization-free calibrated bundle adjustment (2025), arXiv preprint arXiv:2506.23808

    Pith/arXiv arXiv 2025

  17. [17]

    Pan, L., Barath, D., Pollefeys, M., Schönberger, J.L.: Global structure-from-motion revisited. In: Eur. Conf. Comput. Vis. (2024)

    2024

  18. [18]

    Ren, J., Liang, W., Yan, R., Mai, L., Liu, S., Liu, X.: MegBA: A GPU-based distributed library for large-scale bundle adjustment. In: Eur. Conf. Comput. Vis. (2022)

    2022

  19. [19]

    In: IEEE Conf

    Safari, D.: Matrix-free shared intrinsics bundle adjustment. In: IEEE Conf. Com- put. Vis. Pattern Recog. (2025)

    2025

  20. [20]

    In: IEEE Conf

    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: IEEE Conf. Comput. Vis. Pattern Recog. (2016)

    2016

  21. [21]

    Sibley, G.: Relative bundle adjustment. Tech. Rep. Technical Report 2307/09, Ox- ford University Computing Laboratory (2009)

    2009

  22. [22]

    Strasdat, H., Davison, A.J., Montiel, J.M.M., Konolige, K.: Double window opti- misation for constant time visual SLAM. In: Int. Conf. Comput. Vis. (2011)

    2011

  23. [23]

    In: Vision Algorithms: Theory and Practice, pp

    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Vision Algorithms: Theory and Practice, pp. 298–372. Springer (2000)

    2000

  24. [24]

    Ventura, J., Larsson, V., Kahl, F.: Uncalibrated structure from motion on a sphere. In: Int. Conf. Comput. Vis. (2025)

    2025

  25. [25]

    In: IEEE Conf

    Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: VGGT: Visual geometry grounded transformer. In: IEEE Conf. Comput. Vis. Pattern Recog. (2025)

    2025

  26. [26]

    In: IEEE Conf

    Wang, S., Leroy, V., Cabon, Y., Chidlovskii, B., Revaud, J.: DUSt3R: Geometric 3D vision made easy. In: IEEE Conf. Comput. Vis. Pattern Recog. (2024)

    2024

  27. [27]

    In: IEEE Conf

    Weber, S., Demmel, N., Chan, T.C., Cremers, D.: Power bundle adjustment for large-scale 3D reconstruction. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

    2023

  28. [28]

    Weber, S., Hong, J.H., Cremers, D.: Power variable projection for initialization-free large-scale bundle adjustment. In: Eur. Conf. Comput. Vis. (2024)

    2024

  29. [29]

    In: IEEE Conf

    Wu, C., Agarwal, S., Curless, B., Seitz, S.M.: Multicore bundle adjustment. In: IEEE Conf. Comput. Vis. Pattern Recog. (2011)

    2011

  30. [30]

    Zhan, Z., Xu, H., Fang, Z., Wei, X., Hu, Y., Wang, C.: Bundle adjustment in the eager mode (2024), arXiv preprint arXiv:2409.12190

    Pith/arXiv arXiv 2024

  31. [31]

    Zhang, R., Zhu, S., Fang, T., Quan, L.: Distributed very large scale bundle adjust- ment by global camera consensus. In: Int. Conf. Comput. Vis. (2017) CSS-BA 17

    2017

  32. [32]

    Zhong, J., Zhan, Z., Gao, Q., Chen, Z., Lou, H., Mao, J., Neumann, U., Wang, C., Wang, Y.: InstantSfM: Towards GPU-native SfM for the deep learning era (2025), arXiv preprint arXiv:2510.13310

    Pith/arXiv arXiv 2025

  33. [33]

    CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment

    Zhou, L., Luo, Z., Zhen, M., Shen, T., Li, S., Huang, Z., Fang, T., Quan, L.: Stochastic bundle adjustment for efficient and scalable 3D reconstruction. In: Eur. Conf. Comput. Vis. (2020) Supplementary Material for “CSS-BA: Gate-Guided Column Space Search for Bundle Adjustment” Ayano Kaneda1, Takafumi Taketomi2, Shugo Yamaguchi1, and Shigeo Morishima1 1 W...

    2020