{"id":"cdacd274-bc97-4a21-87e3-6b5abc4a8654","arxiv_id":"2508.20526","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Backpropagating the 3D Gaussian Splatting color loss into camera parameters, with an L2 loss, focal-length training, and a Hessian-based reparameterization, raises average PSNR by 0.4 dB over COLMAP calibration on reference scenes.","lead":"This paper fine-tunes camera positions and focal lengths using a 3D Gaussian Splatting model's color loss, improving average novel-view PSNR by 0.4 dB on common benchmarks. It is a practical calibration 'last mile' that could change how reference datasets are prepared for novel-view synthesis comparisons.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"If test-camera poses are fine-tuned using the color loss on test views, the reported +0.43 dB PSNR gain is not a valid novel-view measurement; the paper never states that test cameras are frozen.","rationale":"The reader's weakest assumption was whether minimizing the color loss on training views transfers to held-out novel-view PSNR. My concern goes one step further: the paper never explicitly states that the fine-tuning is restricted to training views. Section 2.3's 'all the cameras parameters' and Section 3's use of 'our calibration' for rendering test images leave open the possibility that test-camera poses are also updated using the color loss on those test images. If that is the case, the evaluation is not a valid measurement of calibration quality at all, because the evaluation cameras have been fit to the exact views being scored. This is the most load-bearing issue because every other contribution—the L2 loss, the Hessian reparameterization, the interleaved schedule—is subservient to the validity of the reported benchmark number. The internal kitchen inconsistency between Table 1 and Table 3 further weakens the empirical support, and the absence of error bars means the 0.4 dB effect could be within run-to-run variation. None of this proves the method is wrong; it shows that the central claim currently rests on an ambiguous protocol and an unreconciled table. The reader's CONDITIONAL verdict is therefore appropriate, and my analysis reinforces it rather than changing it. The concrete test is to log or diff test-camera poses before and after fine-tuning; this single check would settle whether the evaluation is contaminated. If test cameras are untouched, the main remaining issue is the table conflict and missing error bars, which are fixable. If test cameras are updated, the headline result must be recomputed under a clean protocol before the paper can be accepted.","tokens_in":5882,"tokens_out":7467,"duration_ms":70521,"concrete_test":"Instrument the pipeline to log which cameras are updated during fine-tuning, or diff the saved COLMAP poses (translation, quaternion, fx/fy/phi) for every test image before and after the camera fine-tuning phase. If any test-view pose differs from the initial COLMAP value, evaluation is contaminated. A clean test: fix all test-camera parameters, fine-tune only training cameras, then re-run the Section 3 protocol; recompute the average PSNR gain, and ideally repeat over 5 seeds to assess stochasticity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that fine-tuning camera parameters with the 3DGS color loss improves calibration quality, as measured by PSNR on the 3DGS reference scenes. Section 2.3 says 'training of all the cameras parameters, one camera after another,' and Section 3 says PSNR is computed by rendering test images using 'our calibration.' The paper never states whether the test/hold-out views are excluded from the camera fine-tuning loop. If the color loss for a test camera is computed by rendering that held-out view and comparing it to its ground-truth image, then the test-camera pose is optimized against the exact image on which PSNR is later reported. This is a form of test-set leakage: the reported 0.43 dB improvement could be caused by fitting the evaluation cameras to the model rather than by a genuinely better calibration. The improvement magnitude is also within the range of 3DGS training stochasticity, and no error bars are given. Additionally, Table 1 lists kitchen as 31.50 dB 'ours' while Table 3 'full' lists 31.14 dB, and only the 31.14 value reproduces the reported average 28.79 dB; this internal inconsistency makes it impossible to determine the true per-scene effect.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a post-hoc camera-calibration refinement method for 3D Gaussian Splatting. Starting from a COLMAP calibration, the method interleaves 3DGS model training with fine-tuning of each camera's translation, rotation (quaternion), and field-of-view parameters by backpropagating a color loss. It uses L2 loss for camera steps, an early-stopping schedule based on EMA of PSNR progress, and a Hessian-eigenvector reparameterization of (z_c, phi_x, phi_y). On the Mip-NeRF 360, Tanks & Temples, and Deep Blending scenes used as 3DGS reference data, the authors report an average PSNR improvement from 28.36 dB to 28.79 dB over the COLMAP calibration, with the same 3DGS training code. Ablations remove the reparameterization, FOV training, L2 loss, and the repeated interleaved schedule.","tokens_in":6194,"tokens_out":3400,"duration_ms":31985,"significance":"If the result holds, it is practically useful: benchmark calibration for novel-view synthesis could be improved by roughly 0.4 dB without changing the renderer, and the method is simple to implement. The gradient derivation in Eqs. (1)-(2) is clean and the approximation is justified for typical Gaussian sizes and distances. The comparison against the external COLMAP baseline on held-out views is the right evaluation design, and the improvement is not forced by construction. However, the manuscript currently does not explicitly state that the held-out test cameras are excluded from fine-tuning, reports no repeated runs or error bars, and contains an internal table conflict for the kitchen scene. These issues directly affect the validity of the central quantitative claim, so the contribution cannot be accepted as-is.","major_comments":[{"comment":"The paper never states whether the held-out test views are excluded from camera fine-tuning. Section 2.3 says 'training of all the cameras parameters, one camera after another,' and Section 3 says PSNR is computed by rendering test images using 'our calibration.' If a test camera's pose is optimized against the color loss computed from its own ground-truth held-out image, then the reported PSNR is a fitting metric, not a novel-view metric, and the +0.43 dB gain could be caused by test-set leakage. Please state explicitly the train/test camera split, confirm that test cameras are frozen during fine-tuning, and report the evaluation protocol precisely enough to rule out this possibility.","section":"Section 2.3 and Section 3"},{"comment":"The kitchen scene is reported as 31.50 dB 'ours' in Table 1 but as 31.14 dB in the 'full' row of Table 3. Only the 31.14 value reproduces the stated average of 28.79 dB across the 11 scenes, while the Table 1 value gives 28.82 dB. This internal inconsistency means the per-scene effect of the method is currently ambiguous. Please correct the tables, recompute all averages, and verify each entry against the raw per-scene outputs.","section":"Table 1 and Table 3"},{"comment":"No error bars, repeated seeds, or run-to-run variability are reported. 3DGS training is stochastic, and several per-scene gains are small (counter +0.15 dB, room +0.22 dB); without multiple runs it is unclear whether the average +0.43 dB improvement is distinguishable from training noise. Please report the mean and standard deviation over at least a few seeds, or otherwise provide evidence that the differences are stable.","section":"Section 3, general evaluation"},{"comment":"The Hessian-based reparameterization is a central component, but its contribution is not isolated in the ablations. The 'w/o abc' row removes the reparameterization while keeping FOV training, but the 'w/o fov' row removes FOV training and therefore also removes the reparameterization, so the two effects are confounded. In addition, the Hessian is approximated by finite differences because second derivatives are 'unstable', yet no sensitivity analysis is given for the finite-difference step epsilon, and the stability of the eigenbasis across training is not examined. Please provide an ablation that keeps FOV training but disables only the reparameterization, and report sensitivity to epsilon.","section":"Section 2.4 and Table 3"}],"minor_comments":[{"comment":"The abstract contains the misspelling 'criticity' and the phrase 'the stake of novel view quality is the most important' should be reworded for clarity.","section":"Abstract"},{"comment":"The keywords line reads 'Example, model, template.' and appears to be a leftover placeholder; it should be replaced with actual keywords such as camera calibration, 3D Gaussian Splatting, novel view synthesis, pose optimization.","section":"Keywords"},{"comment":"Equation (4) has spacing and subscript issues ('(fx/z, fy/z)'); please typeset it as (f_x x / z, f_y y / z) and ensure all variables are defined consistently.","section":"Section 2.4, Eq. (4)"},{"comment":"The text says the fine-tuned calibration enables 'generally better PSNR with less gaussians', but the Train scene shows an increase from 1.01M to 1.13M Gaussians; please qualify the statement or explain the exception.","section":"Table 2"},{"comment":"The caption of Figure 1 does not define what the subgraphs represent quantitatively; please state the metric, the camera index, and the convergence criterion used for the early stop.","section":"Section 3, Figure 1"},{"comment":"The phrase 'once runs only on phase of cameras fine-tuning' appears to mean 'runs only one phase of camera fine-tuning'; please correct the wording.","section":"Section 3, ablation table"}],"recommendation":"major_revision","confidential_remarks":"The central idea is close in spirit to the pose optimization already present in gsplat, so the novelty rests on the specific training schedule, loss choice, and reparameterization, plus the claimed benchmark improvement. The evaluation-protocol ambiguity and the table inconsistency are the main blockers; both are fixable in revision. I would also ask the editor to require the authors to state whether any code or configuration will be released, since the method depends on several manually chosen hyperparameters (beta, M, thresholds, epsilon, learning rate) and the current text gives no reproducibility details."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline: the paper promises a +0.43 dB PSNR gain on the 3DGS reference scenes by fine-tuning camera parameters through the color loss, and the ablation suggests each of L2 loss, focal training, and the Hessian reparameterization contributes. That part is plausible and worth a look. But the evaluation is compromised: the text says cameras are trained \"one camera after another\" and then PSNR is computed on \"test images\" with \"our calibration,\" yet it never states that test-view cameras are excluded from the fine-tuning. If a held-out camera pose is optimized against the color loss of its own ground-truth image, the reported gain is not a novel-view measurement. Given the protocol described, I think that is what happens. This is a load-bearing flaw, not a minor gap.\n\nWhat is genuinely useful: the derivation of the gradient with the covariance term dropped is clean and sensibly justified, the interleaved schedule with early stopping is a reasonable engineering choice, and the Hessian eigenbasis reparameterization is an interesting trick. The ablations in Table 3 mostly support the design choices, and the paper cites gsplat and gsplatloc honestly.\n\nThe soft spots, in order. First, the leakage issue above. Second, Table 1 reports kitchen at 31.50 dB while Table 3's \"full\" column lists 31.14; only the 31.14 value reproduces the stated average 28.79 dB. Third, there are no error bars, and the gain is within the range of 3DGS training stochasticity. Fourth, the Hessian approximation is admitted unstable, and several hyperparameters (learning rates, epsilon, thresholds) are not stated. These are fixable, but the evaluation problem is not.\n\nIf the authors rerun with test cameras frozen and the improvement holds, this is a solid practical contribution for fairer benchmark comparisons. As it stands, the central claim is unverified. I'd send it to review to force a clean evaluation, but I would not accept it now. A careful reader should not cite the +0.43 dB number.","headline":"A plausible calibration fine-tuning recipe for 3DGS, but the reported +0.43 dB gain is compromised because the paper never states that test-view cameras are excluded from the fine-tuning loop.","tokens_in":6748,"tokens_out":3752,"would_cite":false,"duration_ms":33414,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Camera fine-tuning with the rendering loss adds 0.4 dB to 3DGS reference scenes","keywords":["3D Gaussian splatting","camera calibration","novel view synthesis","backpropagation","PSNR","Hessian reparameterization","Adam optimizer","pose fine-tuning"],"falsifier":"On a synthetic scene with known ground-truth cameras, add controlled errors to the poses, run the proposed fine-tuning, and check whether the recovered camera parameters move toward ground truth while held-out PSNR improves; if the training-view loss drops but the estimated poses move away from the truth, the central claim is refuted.","tokens_in":5689,"feed_emoji":"📷","tokens_out":8820,"duration_ms":75733,"temperature":0.7,"pith_summary":"3D Gaussian splatting (3DGS) is a differentiable rendering model that represents a scene as a collection of colored Gaussian points; this paper tries to establish that the camera calibration fed into it is not the best possible calibration for novel view synthesis, and that post-hoc optimization can recover part of that gap. It treats each camera's translation, rotation, and fields of view as trainable parameters, updated by backpropagating the rendering color loss through the model. Interleaving these camera updates with ordinary model training raises average PSNR on the eleven reference scenes of the original 3DGS paper from 28.36 to 28.79 dB, with the improved calibration alone responsible for the gain. This matters because benchmark comparisons of view synthesis methods currently mix model quality with calibration quality, and a 0.4 dB shift is as large as some published model improvements.","feed_headline":"Camera fine-tuning adds 0.4 dB to 3DGS reference scenes","feed_subtitle":"Backpropagating the novel-view color loss into camera poses and focal lengths beats the reference calibration.","key_machinery":"The machinery is the differentiable pinhole projection inside the 3DGS renderer: the color loss is differentiated with respect to each camera parameter through the projected Gaussian centers, with the covariance term neglected under the assumption that Gaussians are small relative to their distance to the camera. Because depth and focal length are correlated through $(u,v)=(f_x x/z, f_y y/z)$, the paper reparameterizes $(z_c, \\phi_x, \\phi_y)$ as coordinates in the eigenbasis of the Hessian of the loss, so Adam's momentum can move independently in low- and high-curvature directions. A phased schedule coordinates the two optimizations: $M=3000$ model steps, then per-camera fine-tuning stopped early by an EMA of PSNR progress once it falls below a threshold, saving about 20% of camera-tuning iterations.","core_discovery":"The central claim is that a trained 3DGS model can double as a calibration oracle: starting from the reference calibration, alternating between model training and per-camera fine-tuning of pose and fields of view, using L2 color loss and a Hessian-based reparameterization, improves the average test PSNR from 28.36 to 28.79 dB on the reference scenes of the 3DGS paper. The gains come from the new calibration alone, because the final model is retrained and evaluated with the original code and rendering pipeline. The paper also reports that most scenes reach equal or better PSNR with fewer Gaussians, and that using L1 loss for camera parameters can diverge, while L2 is more reliable because the camera gradient is averaged over millions of Gaussians.","pith_inferences":["If the color-loss signal transfers reliably to held-out views, the same camera-fine-tuning loop could be applied after any radiance-field training, not only 3DGS, and could be iterated to a fixed point; the paper only demonstrates a single pass.","The Hessian-eigenvector reparameterization is a generic cure for depth-focal-length correlation, so it could transfer to monocular depth estimation or SLAM backends where scale and focal length are entangled.","A testable extension is to stress-test the initialization basin: measure how far the initial calibration can be perturbed before fine-tuning stops recovering, which would reveal whether the method is a local polish or a global pose optimizer.","One could also check whether the 0.4 dB gain is additive to model-side improvements, such as densification or regularization changes, or whether better calibration and better models saturate the same error."],"forward_implications":["Reference numbers for novel view synthesis should state which calibration they used, since the calibration itself accounts for roughly 0.4 dB of the score.","Calibration becomes a final polishing step that can be applied to an existing 3DGS training run by optimizing only camera parameters.","The interleaved schedule makes the extra cost controllable: cameras that show no early PSNR progress stop after a minimum of 100 steps, saving about 20% of camera-tuning time.","Because most scenes reach equal or better PSNR with fewer Gaussians after fine-tuning, calibration quality and model compactness are linked, not independent."],"supporting_citations":[{"why":"Supplies the differentiable Gaussian splatting renderer, the reference training pipeline, and the benchmark scenes on which the 0.4 dB improvement is measured.","marker":"[3]"},{"why":"Provides the initial feature-based camera calibration that the paper fine-tunes and uses as the baseline 'dataset calibration'.","marker":"[8]"},{"why":"The prior work that backpropagates through pose during 3DGS training; the paper differentiates its loss choice, schedule, and focal-length training from this baseline.","marker":"[12]"},{"why":"Adam is the optimizer used for camera parameters, and its momentum behavior motivates the Hessian-eigenvector reparameterization.","marker":"[5]"},{"why":"Explains momentum as accumulation of persistent gradient directions, the rationale for decoupling low- and high-curvature parameter directions.","marker":"[9]"},{"why":"Provides seven of the eleven reference scenes used for evaluation.","marker":"[1]"},{"why":"Provides two of the eleven reference scenes used for evaluation.","marker":"[6]"},{"why":"Provides two of the eleven reference scenes used for evaluation.","marker":"[2]"}],"fun_headline_variants":["3DGS backprop fine-tunes cameras, gains 0.4 dB PSNR","Camera calibration via 3DGS color loss adds 0.4 dB PSNR","3DGS fine-tunes camera poses and fovs for +0.4 dB PSNR","Backpropagate color loss into cameras with 3DGS, gain 0.4 dB"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that minimizing the color error on the training views reliably improves the camera calibration instead of merely overfitting those views, so that held-out novel-view PSNR benefits as well.","fun_headline_variants_meta":{"raw":{"variants":["3DGS backprop fine-tunes cameras, gains 0.4 dB PSNR","Camera calibration via 3DGS color loss adds 0.4 dB PSNR","3DGS fine-tunes camera poses and fovs for +0.4 dB PSNR","Backpropagate color loss into cameras with 3DGS, gain 0.4 dB"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000745,"raw_usage":{"total_tokens":3272,"prompt_tokens":846,"completion_tokens":2426,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":2325}},"tokens_in":462,"tokens_out":2426,"duration_ms":15437,"temperature":1.0,"reasoning_tokens":2325,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:44:18.797911+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a synthetic scene with known ground-truth cameras, add controlled errors to the poses, run the proposed fine-tuning, and check whether the recovered camera parameters move toward ground truth while held-out PSNR improves; if the training-view loss drops but the estimated poses move away from the truth, the central claim is refuted.","supporting_citations":[{"cited_title":"Robust methods for accu- rate and efficient 3D modeling from unstructured im- agery","cited_arxiv_id":null,"evidence_quote":"Provides the initial feature-based camera calibration that the paper fine-tunes and uses as the baseline 'dataset calibration'."},{"cited_title":"On the importance of initialization and momentum in deep learning","cited_arxiv_id":null,"evidence_quote":"Explains momentum as accumulation of persistent gradient directions, the rationale for decoupling low- and high-curvature parameter directions."},{"cited_title":"Mip-nerf 360: Unbounded anti-aliased neural radiance fields","cited_arxiv_id":null,"evidence_quote":"Provides seven of the eleven reference scenes used for evaluation."},{"cited_title":"Deep blending for free-viewpoint image-based rendering","cited_arxiv_id":null,"evidence_quote":"Provides two of the eleven reference scenes used for evaluation."}],"review_version":2}