Pith. sign in

REVIEW 4 major objections 6 minor 13 references

Adam SLAM - the last mile of camera calibration with 3DGS

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Camera fine-tuning with the rendering loss adds 0.4 dB to 3DGS reference scenes

desk verdict A plausible calibration fine-tuning recipe for 3DGS, but the reported +0.43 dB gain is compromised because the paper never states that test-view cameras are excluded from the fine-tuning loop. read the letter →

arxiv 2508.20526 v1 pith:P3TN3FO7 submitted 2025-08-28 cs.CV

Matthieu Gendrin , Stéphane Pateux , Xiaoran Jiang , Théo Ladune , Luce Morin This is my paper · ORCID
classification cs.CV
keywords 3DGaussiansplattingcameracalibrationnovelviewsynthesisbackpropagationPSNRHessianreparameterizationAdamoptimizerposefine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

3D Gaussian splatting (3DGS) is a differentiable rendering model that represents a scene as a collection of colored Gaussian points; this paper tries to establish that the camera calibration fed into it is not the best possible calibration for novel view synthesis, and that post-hoc optimization can recover part of that gap. It treats each camera's translation, rotation, and fields of view as trainable parameters, updated by backpropagating the rendering color loss through the model. Interleaving these camera updates with ordinary model training raises average PSNR on the eleven reference scenes of the original 3DGS paper from 28.36 to 28.79 dB, with the improved calibration alone responsible for the gain. This matters because benchmark comparisons of view synthesis methods currently mix model quality with calibration quality, and a 0.4 dB shift is as large as some published model improvements.

What carries the argument

The machinery is the differentiable pinhole projection inside the 3DGS renderer: the color loss is differentiated with respect to each camera parameter through the projected Gaussian centers, with the covariance term neglected under the assumption that Gaussians are small relative to their distance to the camera. Because depth and focal length are correlated through $(u,v)=(f_x x/z, f_y y/z)$, the paper reparameterizes $(z_c, \phi_x, \phi_y)$ as coordinates in the eigenbasis of the Hessian of the loss, so Adam's momentum can move independently in low- and high-curvature directions. A phased schedule coordinates the two optimizations: $M=3000$ model steps, then per-camera fine-tuning stopped early by an EMA of PSNR progress once it falls below a threshold, saving about 20% of camera-tuning iterations.

What would settle it

On a synthetic scene with known ground-truth cameras, add controlled errors to the poses, run the proposed fine-tuning, and check whether the recovered camera parameters move toward ground truth while held-out PSNR improves; if the training-view loss drops but the estimated poses move away from the truth, the central claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that a trained 3DGS model can double as a calibration oracle: starting from the reference calibration, alternating between model training and per-camera fine-tuning of pose and fields of view, using L2 color loss and a Hessian-based reparameterization, improves the average test PSNR from 28.36 to 28.79 dB on the reference scenes of the 3DGS paper. The gains come from the new calibration alone, because the final model is retrained and evaluated with the original code and rendering pipeline. The paper also reports that most scenes reach equal or better PSNR with fewer Gaussians, and that using L1 loss for camera parameters can diverge, while L2 is more reliable because the camera gradient is averaged over millions of Gaussians.

Load-bearing premise

The load-bearing premise is that minimizing the color error on the training views reliably improves the camera calibration instead of merely overfitting those views, so that held-out novel-view PSNR benefits as well.

Editorial extensions

If this is right

  • Reference numbers for novel view synthesis should state which calibration they used, since the calibration itself accounts for roughly 0.4 dB of the score.
  • Calibration becomes a final polishing step that can be applied to an existing 3DGS training run by optimizing only camera parameters.
  • The interleaved schedule makes the extra cost controllable: cameras that show no early PSNR progress stop after a minimum of 100 steps, saving about 20% of camera-tuning time.
  • Because most scenes reach equal or better PSNR with fewer Gaussians after fine-tuning, calibration quality and model compactness are linked, not independent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the color-loss signal transfers reliably to held-out views, the same camera-fine-tuning loop could be applied after any radiance-field training, not only 3DGS, and could be iterated to a fixed point; the paper only demonstrates a single pass.
  • The Hessian-eigenvector reparameterization is a generic cure for depth-focal-length correlation, so it could transfer to monocular depth estimation or SLAM backends where scale and focal length are entangled.
  • A testable extension is to stress-test the initialization basin: measure how far the initial calibration can be perturbed before fine-tuning stops recovering, which would reveal whether the method is a local polish or a global pose optimizer.
  • One could also check whether the 0.4 dB gain is additive to model-side improvements, such as densification or regularization changes, or whether better calibration and better models saturate the same error.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a post-hoc camera-calibration refinement method for 3D Gaussian Splatting. Starting from a COLMAP calibration, the method interleaves 3DGS model training with fine-tuning of each camera's translation, rotation (quaternion), and field-of-view parameters by backpropagating a color loss. It uses L2 loss for camera steps, an early-stopping schedule based on EMA of PSNR progress, and a Hessian-eigenvector reparameterization of (z_c, phi_x, phi_y). On the Mip-NeRF 360, Tanks & Temples, and Deep Blending scenes used as 3DGS reference data, the authors report an average PSNR improvement from 28.36 dB to 28.79 dB over the COLMAP calibration, with the same 3DGS training code. Ablations remove the reparameterization, FOV training, L2 loss, and the repeated interleaved schedule.

Significance. If the result holds, it is practically useful: benchmark calibration for novel-view synthesis could be improved by roughly 0.4 dB without changing the renderer, and the method is simple to implement. The gradient derivation in Eqs. (1)-(2) is clean and the approximation is justified for typical Gaussian sizes and distances. The comparison against the external COLMAP baseline on held-out views is the right evaluation design, and the improvement is not forced by construction. However, the manuscript currently does not explicitly state that the held-out test cameras are excluded from fine-tuning, reports no repeated runs or error bars, and contains an internal table conflict for the kitchen scene. These issues directly affect the validity of the central quantitative claim, so the contribution cannot be accepted as-is.

major comments (4)
  1. [Section 2.3 and Section 3] The paper never states whether the held-out test views are excluded from camera fine-tuning. Section 2.3 says 'training of all the cameras parameters, one camera after another,' and Section 3 says PSNR is computed by rendering test images using 'our calibration.' If a test camera's pose is optimized against the color loss computed from its own ground-truth held-out image, then the reported PSNR is a fitting metric, not a novel-view metric, and the +0.43 dB gain could be caused by test-set leakage. Please state explicitly the train/test camera split, confirm that test cameras are frozen during fine-tuning, and report the evaluation protocol precisely enough to rule out this possibility.
  2. [Table 1 and Table 3] The kitchen scene is reported as 31.50 dB 'ours' in Table 1 but as 31.14 dB in the 'full' row of Table 3. Only the 31.14 value reproduces the stated average of 28.79 dB across the 11 scenes, while the Table 1 value gives 28.82 dB. This internal inconsistency means the per-scene effect of the method is currently ambiguous. Please correct the tables, recompute all averages, and verify each entry against the raw per-scene outputs.
  3. [Section 3, general evaluation] No error bars, repeated seeds, or run-to-run variability are reported. 3DGS training is stochastic, and several per-scene gains are small (counter +0.15 dB, room +0.22 dB); without multiple runs it is unclear whether the average +0.43 dB improvement is distinguishable from training noise. Please report the mean and standard deviation over at least a few seeds, or otherwise provide evidence that the differences are stable.
  4. [Section 2.4 and Table 3] The Hessian-based reparameterization is a central component, but its contribution is not isolated in the ablations. The 'w/o abc' row removes the reparameterization while keeping FOV training, but the 'w/o fov' row removes FOV training and therefore also removes the reparameterization, so the two effects are confounded. In addition, the Hessian is approximated by finite differences because second derivatives are 'unstable', yet no sensitivity analysis is given for the finite-difference step epsilon, and the stability of the eigenbasis across training is not examined. Please provide an ablation that keeps FOV training but disables only the reparameterization, and report sensitivity to epsilon.
minor comments (6)
  1. [Abstract] The abstract contains the misspelling 'criticity' and the phrase 'the stake of novel view quality is the most important' should be reworded for clarity.
  2. [Keywords] The keywords line reads 'Example, model, template.' and appears to be a leftover placeholder; it should be replaced with actual keywords such as camera calibration, 3D Gaussian Splatting, novel view synthesis, pose optimization.
  3. [Section 2.4, Eq. (4)] Equation (4) has spacing and subscript issues ('(fx/z, fy/z)'); please typeset it as (f_x x / z, f_y y / z) and ensure all variables are defined consistently.
  4. [Table 2] The text says the fine-tuned calibration enables 'generally better PSNR with less gaussians', but the Train scene shows an increase from 1.01M to 1.13M Gaussians; please qualify the statement or explain the exception.
  5. [Section 3, Figure 1] The caption of Figure 1 does not define what the subgraphs represent quantitatively; please state the metric, the camera index, and the convergence criterion used for the early stop.
  6. [Section 3, ablation table] The phrase 'once runs only on phase of cameras fine-tuning' appears to mean 'runs only one phase of camera fine-tuning'; please correct the wording.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported +0.43 dB PSNR gain may be the fitted objective itself: the paper trains 'all the cameras parameters' and never states that the test cameras are frozen.

  1. fitted input called prediction [Section 2.3 (Training schedule) and Section 3 (Results)]
    "we proceed in a loop with M iterations of 3DGS model training, followed by the training of all the cameras parameters, one camera after another. ... we first train the camera positions and model together, as described in 2, then we re-train the model, render the test images and process metrics with the original code of [3]."

    The paper's protocol trains 'all the cameras parameters' and does not list test cameras as an exception. For any camera, the color loss is computed by rendering from that camera and comparing to its ground-truth image; PSNR is 10 log10(1/MSE). If a test camera is included in the fine-tuning loop, minimizing the color loss on that test image is exactly maximizing the PSNR later reported for that same image. The headline +0.43 dB improvement is then the fitted objective for the evaluation cameras, not an independent held-out measure of novel-view quality.

full rationale

The paper's derivation chain is: take a COLMAP calibration, train a 3DGS model, interleave fine-tuning of camera parameters by backpropagating the color loss, then re-train the model and report PSNR on test images. The load-bearing problem is the evaluation protocol: Section 2.3 says 'the training of all the cameras parameters, one camera after another' and Section 3 does not state that the test/hold-out views are excluded from this loop. If test camera poses are optimized against the color loss on the same test images that are later rendered for PSNR, the reported improvement is at least partly the result of fitting the evaluation cameras to the evaluation images. No equation in the paper separates 'test' from 'train' cameras in the fine-tuning phase, and no sentence freezes the test cameras. The external COLMAP baseline and the fact that the 3DGS model itself is trained only on the training views prevent a complete collapse, so this is a partial, not total, circularity. There are no self-citation chains or imported uniqueness theorems. Additional correctness concerns, not circularity: Table 1 lists kitchen 'ours' as 31.50 dB while Table 3 'full' lists 31.14 dB, and only the latter reproduces the stated 28.79 dB average; no error bars or repeated-seed statistics are given, and the reported gains are within the range of 3DGS training stochasticity.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The method depends on several hand-set schedule hyperparameters and two unreported values (camera learning rate, finite-difference epsilon). No new physical entities are introduced; the Hessian reparameterization is a mathematical transformation of existing camera parameters.

free parameters (7)
  • EMA smoothing factor beta = 1/50
    Controls the early-stopping progress score in Eq. (3); set experimentally in Section 2.3.
  • Model training steps between camera fine-tuning phases M = 3000
    Chosen as the natural pace matching the opacity-reset interval of 3DGS (Section 2.3).
  • Minimum camera fine-tuning steps = 100
    Hand-set lower bound for per-camera training (Section 2.3).
  • Maximum camera fine-tuning steps = 1000
    Hand-set upper bound for per-camera training (Section 2.3).
  • Early-stopping progress threshold t = 0.0002
    Identified as a good trade-off 'experimentally' (Section 2.3).
  • Finite-difference step epsilon for Hessian approximation = not reported
    Used in Section 2.4 to estimate the Hessian; exact value not given.
  • Camera optimizer learning rate = not reported
    The Adam learning rate for camera parameters is never stated, yet it is critical for convergence of the fine-tuning.
assumptions (6)
  • domain assumption The pinhole camera model with world-to-camera translation/rotation and fields of view accurately models the capture process.
    Used in Eq. (4) and throughout the method; if the model is wrong, the reparameterization and gradients are invalid.
  • domain assumption The color loss on training views is an adequate proxy for calibration quality and transfers to held-out novel-view PSNR.
    Underlies the entire fine-tuning approach (Sections 2.2, 2.3); the evaluation in Section 3 treats PSNR improvement as calibration improvement.
  • domain assumption In Eq. (2), the derivative through the projected covariance is negligible because gaussian covariances are small relative to their distance from the camera.
    Stated in Section 2.1; if many gaussians violate this, gradients will be inaccurate.
  • ad hoc to paper The loss landscape near the optimum is locally quadratic, so the Hessian eigenbasis decouples parameters and improves Adam convergence.
    Section 2.4; the paper admits second derivatives are unstable and uses finite differences, so this is an unproved heuristic.
  • domain assumption The 3DGS rendering and training pipeline from [3] is correct and differentiable.
    The method builds on 3DGS without re-deriving it.
  • standard math Gradient backpropagation through the pinhole projection follows the standard chain rule.
    Eqs. (1)-(2) and (4); standard calculus.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adam SLAM - the last mile of camera calibration with 3DGS." pith.science (2026). https://pith.science/paper/P3TN3FO7

@misc{pith2026250820526,
  author       = {Pith},
  title        = {Pith review of: Adam SLAM - the last mile of camera calibration with 3DGS},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P3TN3FO7}},
  note         = {Machine review of arXiv:2508.20526}
}
read the original abstract

The quality of the camera calibration is of major importance for evaluating progresses in novel view synthesis, as a 1-pixel error on the calibration has a significant impact on the reconstruction quality. While there is no ground truth for real scenes, the quality of the calibration is assessed by the quality of the novel view synthesis. This paper proposes to use a 3DGS model to fine tune calibration by backpropagation of novel view color loss with respect to the cameras parameters. The new calibration alone brings an average improvement of 0.4 dB PSNR on the dataset used as reference by 3DGS. The fine tuning may be long and its suitability depends on the criticity of training time, but for calibration of reference scenes, such as Mip-NeRF 360, the stake of novel view quality is the most important.

Figures

Figures reproduced from arXiv: 2508.20526 by the authors.

Figure 1
Figure 1. Tracking of the PSNR improvement during the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Reference (top) vs ours (bottom) on bicycle view. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Reference (left) vs ours (right) on DrJohnson [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Reference (blue) vs ours (orange) motion flow [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Example of failure. A detailed zone (the tree), [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 7 canonical work pages

  1. [1]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 5470– 5479, 2022

    work page 2022

  2. [2]

    Deep blending for free-viewpoint image-based rendering

    Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. Deep blending for free-viewpoint image-based rendering. ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings), 37(6), November 2018

    work page 2018

  3. [3]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1, 2023

    2023

  4. [4]

    3d gaussian splatting as markov chain monte carlo.arXiv preprint arXiv:2404.09591, 2024

    Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Jeff Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splatting as markov chain monte carlo.arXiv preprint arXiv:2404.09591, 2024

    arXiv 2024

  5. [5]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    arXiv 2014

  6. [6]

    Tanks and temples: Benchmarking large-scale scene reconstruction

    Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4):1–13, 2017

    2017

  7. [7]

    Pixel-perfect structure- from-motion with featuremetric refinement

    Philipp Lindenberger, Paul-Edouard Sarlin, Viktor Larsson, and Marc Pollefeys. Pixel-perfect structure- from-motion with featuremetric refinement. In Pro- ceedings of the IEEE/CVF international conference on computer vision, pages 5987–5997, 2021

    work page 2021

  8. [8]

    Robust methods for accu- rate and efficient 3D modeling from unstructured im- agery

    Johannes L Schönberger. Robust methods for accu- rate and efficient 3D modeling from unstructured im- agery. PhD thesis, ETH Zurich, 2018

    work page 2018

Show all 13 references
  1. [9]

    On the importance of initialization and momentum in deep learning

    Ilya Sutskever, James Martens, George Dahl, and Ge- offrey Hinton. On the importance of initialization and momentum in deep learning. In International confer- ence on machine learning , pages 1139–1147. pmlr, 2013

    2013

  2. [10]

    RAFT: recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. RAFT: recurrent all-pairs field transforms for optical flow. CoRR, abs/2003.12039, 2020

    2003 arXiv

  3. [11]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pages 20697–20709, 2024

    2024

  4. [12]

    gsplat: An open-source library for gaussian splatting

    Vickie Ye, Ruilong Li, Justin Kerr, Matias Turku- lainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, et al. gsplat: An open-source library for gaussian splatting. arXiv preprint arXiv:2409.06765, 2024

    2024 arXiv

  5. [13]

    Gsplatloc: Ultra-precise camera lo- calization via 3d gaussian splatting

    Atticus J Zeller. Gsplatloc: Ultra-precise camera lo- calization via 3d gaussian splatting. arXiv preprint arXiv:2412.20056, 2024

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.