{"id":"608d2af6-949d-454e-90a1-8fa6472021da","arxiv_id":"2501.18994","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A monocular positioning system that combines absolute and relative pose regression with a Kalman filter, using learned covariances, reports improved temporal accuracy on 7-Scenes and Oxford RobotCar.","lead":"VKFPos blends two ways of estimating a camera's position from images, an absolute guess and a relative motion guess, using a Kalman filter with learned uncertainty. On standard indoor and outdoor benchmarks it reports more accurate trajectories than several existing methods, especially on outdoor routes.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's factorization assumes APR and RPR errors are conditionally independent, but both branches share the current image, so the errors are correlated; the paper neither verifies nor accounts for this, making the EKF misspecified.","rationale":"The reader's weakest assumption is that the RPR branch produces a statistically consistent control-input model, focusing on bias or miscalibration of the RPR mean and covariance. My concern is a distinct but related failure mode: even if each branch is individually well-calibrated, the APR and RPR errors are likely cross-correlated because both branches process the same current image. This violates the conditional independence assumption explicitly used in Eq. (9) of Theorem 1, which is the foundation of the claimed posterior decomposition and of the independent training scheme. This is more directly load-bearing than the calibration concern because it challenges the exactness of the factorization that motivates the entire method. The paper provides no evidence that the cross-covariance is negligible. A simple empirical test can settle whether the concern lands. If the cross-correlation is small, the decomposition is a acceptable approximation and the paper's central claim stands; if it is large, the EKF should incorporate it or the theoretical framing must be weakened. This does not change the overall verdict from CONDITIONAL: the empirical results may still hold, but the theoretical foundation requires additional validation or revision.","tokens_in":13968,"tokens_out":14713,"duration_ms":141589,"concrete_test":"On a held-out validation split of 7-Scenes, compute the per-frame APR residual e_z = z_t - z_truth and the RPR residual e_u = u_{t,t-1} - u_truth. Estimate the 6×6 sample cross-covariance matrix C = E[e_z e_u^T]. If the Frobenius norm of the off-diagonal blocks exceeds, say, 10% of the geometric mean of the diagonal blocks of the APR and RPR covariance estimates, the independence assumption in Eq. (9) is violated. Then modify the EKF to a correlated-noise update that incorporates the estimated C (using the standard cross-covariance terms in the prediction and update equations) and re-evaluate VKFPos on the same benchmark. If the median translation or rotation error changes by more than 5%, or if the filter's estimated trajectory covariance changes materially, the concern is confirmed and the paper's theoretical claim needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim, expressed in Theorem 1 and used to justify independent training of the APR and RPR branches, is the factorization p(x_t | x_{t-1}, u, z_t) ∝ p(z_t | x_t) p(u | x_t, x_{t-1}). This requires the measurement z_t (APR output) and the control u (RPR output) to be conditionally independent given the true state. Eq. (9) makes exactly this assumption: p(z_t | u, x_{t-1}, x_t) = p(z_t | x_t). In VKFPos, however, the APR branch takes the current image I_t as input, and the RPR branch takes (I_{t-1}, I_t). The same current image therefore drives both predictions. Even if the two branches do not share encoder weights, their regression errors are driven by the same scene content, lighting, dynamic objects, and network architectural biases; the generic case is that the APR residual e_z = z_t - z_truth and the RPR residual e_u = u_{t,t-1} - u_truth are cross-correlated. The variational losses in Eqs. (14)-(15) train the two marginals independently, and the EKF update in Eqs. (16)-(25) does not include any cross-covariance term between process and measurement noise. Consequently, the decomposition is not exact for this architecture; the EKF is misspecified, the Kalman gain is suboptimal, and the predicted state covariance can be overconfident. The paper provides no empirical check of the independence assumption, so the 'theoretically-founded' claim rests on an unvalidated structural assumption.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VKFPos, a monocular camera positioning method that combines an Absolute Pose Regression (APR) branch, which regresses the 6DoF pose of a single image and its covariance, with a Relative Pose Regression (RPR) branch, which regresses the relative pose between consecutive images and its covariance. The two branches are trained independently with negative log-likelihood losses, and their outputs are fused at inference time through an Extended Kalman Filter (EKF) in which the RPR output acts as the control input and the APR output acts as the measurement. The authors claim that the posterior of the positioning problem decomposes into APR and RPR components, that the covariance predictions improve generalization, and that the EKF integration yields superior temporal positioning accuracy on the 7-Scenes and Oxford RobotCar datasets compared with existing temporal APR and model-based integration methods.","tokens_in":14285,"tokens_out":3507,"duration_ms":34620,"significance":"If the claims hold, VKFPos would offer a practical and lightweight integration of learned absolute and relative pose estimators, with the important practical virtue of being trainable without a filtering objective and of releasing source code. The covariance prediction is a useful ingredient, and the experimental results on two standard benchmarks are encouraging. However, the paper's central theoretical justification contains a flawed proof, the key conditional-independence assumption is not validated, and the experimental comparison omits the most closely related Kalman-filter baseline. The contribution is therefore potentially valuable but is not yet established at the level claimed.","major_comments":[{"comment":"The proof of Theorem 1 is not mathematically correct. In Eq. (8), the denominator should be the conditional density p(z_t | u_{t,t-1}, x_{t-1}), not the marginal p(z_t), because the posterior being decomposed conditions on u_{t,t-1} and x_{t-1}. In Eq. (10), the factor p(x_{t-1}, x_t) is dropped without justification; the line 'p(x_t, x_{t-1}, u) ∝ p(u|x_t, x_{t-1})' is not a valid conditioning step unless additional assumptions are stated. As written, the chain of equalities and proportionalities does not establish Eq. (6). The authors should provide a correct Bayesian derivation or explicitly present Eq. (6) as a modeling assumption rather than a theorem.","section":"Theorem 1 and Eqs. (8)-(11)"},{"comment":"The factorization in Eq. (6) relies on the conditional independence p(z_t | u_{t,t-1}, x_{t-1}, x_t) = p(z_t | x_t), meaning that the APR measurement noise and the RPR control noise are independent given the true state. In VKFPos, the APR branch takes the current image I_t, while the RPR branch takes (I_{t-1}, I_t); because both branches depend on I_t, their regression residuals will in general be correlated through scene content, lighting, dynamic objects, and network biases. This is not merely a technicality: correlated process and measurement noise makes the EKF misspecified, the Kalman gain suboptimal, and the state covariance potentially overconfident. The paper provides no empirical check of this independence assumption and no measure of cross-correlation between the APR and RPR residuals. This is load-bearing for the claim that the decomposition is theoretically founded.","section":"Eq. (9) and the shared-image architecture"},{"comment":"The APR and RPR branches are trained to minimize their respective NLL losses in Eqs. (14)-(15), but at inference the predicted covariances Σ_z and Σ_u are used directly as the measurement and process noise covariances in the EKF update Eqs. (19)-(25). There is no check that the predicted covariances are statistically calibrated for filtering, e.g., via normalized innovation squared or consistency tests. If the RPR covariance is miscalibrated, the EKF correction can degrade rather than improve the trajectory; the paper reports only final pose errors, not filter consistency. The authors should add a quantitative assessment of covariance calibration and, ideally, an ablation that replaces the learned covariances with fixed constants to show that the learned values are what drive the improvement.","section":"Training/inference mismatch and covariance calibration"},{"comment":"The experimental section does not compare against KFNet (Zhou et al. 2020), which is cited and discussed in the Introduction as the most closely related learning-based EKF integration method for temporal camera relocalization. Because KFNet is the natural baseline for the central claim 'outperforms ... model-based integration methods', its omission leaves the comparative claim incomplete. Additionally, Tables 1 and 3 report no error bars, standard deviations, or number of runs, so it is unclear whether the reported differences, some of which are small, are statistically meaningful. Finally, Table 3's caption states 'translation(cm)' while the values are clearly in meters (e.g., 4.7m), which should be corrected.","section":"Tables 1-3 and baseline choice"}],"minor_comments":[{"comment":"The caption spells 'AtLoc+' as 'AtLot+', which should be corrected.","section":"Figure 4"},{"comment":"The equation contains an extraneous comma and a misplaced plus sign: 'f (ˆxt−1, ut,t−1), +N (0, Σut,t−1 )' should read 'f (x_{t-1}, u_{t,t-1}) + N(0, Σ_{u_{t,t-1}})'.","section":"Eq. (16)"},{"comment":"Some table entries contain repeated or concatenated numbers (e.g., '0.10m0.10m0.10m, 4.07°4.07°4.07°' and '5.27°5.27°5.27°'), which appears to be a formatting artifact and should be cleaned.","section":"Tables 1 and 2"},{"comment":"The reference 'Charles, A. 2018. Kalman Filtering: A Bayesian Approach' is incomplete; it lacks a venue, volume, or publisher information.","section":"References"},{"comment":"The sentence 'The results of performance evaluations ... delivering superior accuracy' in the contribution list is immediately followed by a footnote marker and a URL; the footnote should be placed at the end of the sentence without disrupting the list.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a sensible integration of APR and RPR with an EKF, using learned covariances in both branches. The reported numbers look good, but the paper never compares against its closest relative KFNet, the proof of the central theorem is wrong as written, and the conditional-independence assumption is likely violated. I'd send it to review, but it needs real revisions before acceptance.\n\nWhat's genuinely useful: the design is clean. Train an APR branch and an RPR branch independently, each with an NLL loss that includes predicted covariances, then feed the relative pose as control input and the absolute pose as measurement into an EKF. The covariance prediction is a nice touch, and the single-shot results are comparable to AtLoc while the temporal results improve over MapNet and AtLoc+ on both 7-Scenes and Oxford. They also provide a code link, which helps reproducibility.\n\nNow the soft spots. The proof of Theorem 1 has a concrete error: Eq. (10) drops p(x_{t-1}, x_t) while claiming proportionality to p(u | x_t, x_{t-1}), but that joint still contains x_t, so the proportionality does not hold. The denominator p(z_t) is also discarded without comment. The theorem's statement is true in spirit—it's the standard Bayes-filter factorization—but as written it is not rigorous. That is fixable, but it undercuts the 'theoretically-founded' phrasing.\n\nThe bigger issue is the missing KFNet baseline. KFNet already did APR+RPR+EKF integration with learned covariances, and the paper cites it while explaining why they diverge. That's exactly the comparison a reviewer would demand, and its absence leaves the central improvement claim unsubstantiated. There are also no error bars, no ablations (EKF on/off, covariance learning on/off), and no check on whether the APR and RPR residuals are conditionally independent given the state. The stress-test is right: both branches see the same current image, so their errors are almost certainly correlated, and the EKF is misspecified in principle. The paper does not test whether that miscalibration matters in practice.\n\nWho is this for? Researchers working on learning-based camera localization or deep sensor fusion. It is a useful reference point, and the empirical results suggest the approach has merit, but it is not a breakthrough.\n\nRecommendation: send it to peer review. The missing comparison and sloppy proof are repairable, and the core idea deserves expert scrutiny. A serious referee should demand the KFNet comparison, error bars and ablations, and a direct discussion of the independence assumption.","headline":"VKFPos is a workmanlike EKF-based APR+RPR fusion with competitive numbers, but the missing KFNet comparison, a faulty proof, and an unvalidated independence assumption keep it from being a clear advance.","tokens_in":14838,"tokens_out":2291,"would_cite":false,"duration_ms":22757,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that monocular positioning posterior decomposes into absolute and relative pose factors, and fusing the two learned estimators with an extended Kalman filter outperforms temporal pose regression.","keywords":["monocular positioning","absolute pose regression","relative pose regression","extended Kalman filter","variational Bayesian inference","covariance learning","camera relocalization","se(3) pose representation"],"falsifier":"On a held-out trajectory, compare the relative-pose branch's predicted covariance $\\Sigma_{u,t-1}$ with the empirical covariance of its residuals against ground-truth relative poses. If the predicted uncertainty is systematically off-scale or the residuals have non-zero mean, the EKF-corrected trajectory should degrade relative to the APR-only branch, which would show that the learned control-input model is not statistically consistent enough for the filter to do the work the paper claims.","tokens_in":13780,"feed_emoji":"📷","tokens_out":9834,"duration_ms":78397,"temperature":0.7,"pith_summary":"VKFPos sets out to solve monocular camera positioning—locating a camera from single or consecutive images—by fusing two learned predictions usually used separately: absolute pose regression (APR), which locates the camera in a global frame but is noisy, and relative pose regression (RPR), which tracks motion between frames but accumulates drift. The paper's central claim is that the posterior distribution of the camera pose factorizes into an APR likelihood and an RPR transition prior, so each branch can be trained independently with its own predicted covariance and then combined optimally by an extended Kalman filter. If this decomposition holds, it turns model-based integration—which the paper finds unstable—into a principled Bayesian calculation cheap enough for real-time use. On the indoor 7-Scenes and outdoor Oxford RobotCar benchmarks, the authors report that the single-shot branch matches state-of-the-art APR accuracy while the temporal output beats temporal APR and RNN-based integration methods in translation and rotation accuracy.","feed_headline":"Two pose estimators, one Kalman filter, sharper camera tracking","feed_subtitle":"Learning both uncertainties and fusing them with an EKF lifts temporal pose accuracy beyond APR and RNN baselines.","key_machinery":"The load-bearing object is the posterior factorization of Eq. (6), $p(\\hat{x}_t | \\hat{x}_{t-1}, u_{t,t-1}, z_t) \\propto p(z_t | \\hat{x}_t) \\, p(u_{t,t-1} | \\hat{x}_t, \\hat{x}_{t-1})$, which turns the extended Kalman filter into a Bayes-rule product of learned factors. Each branch approximates its factor with a Gaussian $q(\\cdot)$ on $\\mathfrak{se}(3)$ whose diagonal covariance is predicted by the network; the per-sample negative-log-likelihood losses of Eq. (14) and Eq. (15) train mean and covariance together. In the EKF, the prediction step composes the previous estimate with the relative pose through a manifold update $\\oplus$, and the correction step treats the absolute pose as an identity measurement. The learned covariances enter directly as $\\Sigma_{u,t-1}$ in the covariance prediction and as $\\Sigma_{z,t}$ in the Kalman gain, so the filter's trust in each branch is learned from data.","core_discovery":"At the heart of the paper is Theorem 1: when the current state is conditioned on the latest measurement, the monocular positioning posterior $p(x_t | x_{t-1}, u_{t,t-1}, z_t)$ is proportional to $p(z_t | x_t) \\, p(u_{t,t-1} | x_t, x_{t-1})$. The paper reads this as a decomposition of the positioning problem into an absolute-pose likelihood and a relative-pose transition, and it realizes each factor as a neural network that outputs a Gaussian mean and covariance on the Lie algebra $\\mathfrak{se}(3)$. The branches are trained by minimizing their negative log-likelihood losses, so the covariances are learned rather than hand-tuned and then serve directly as the measurement noise and process noise in an extended Kalman filter. With the APR prediction as measurement and the RPR prediction as control input, the EKF produces a temporally smoothed pose; the reported consequence is single-shot accuracy on par with AtLoc and temporal accuracy that beats MapNet, AtLoc+, ViPR, and LSTM-KF on both benchmark datasets.","pith_inferences":["This decomposition suggests a general recipe: any learned measurement model and any learned motion model that output calibrated Gaussians could be fused in the same way, independent of the particular APR and RPR architectures.","A natural stress test is to replace the learned RPR branch with a visual odometry or inertial motion estimate; the EKF equations do not depend on how the relative pose is generated.","The diagonal-covariance assumption is a simplifying restriction; permitting correlations between translation and rotation might change how much the correction step trusts each branch.","Part of the reported gain may come from the covariance weighting acting as learned loss weights; isolating that effect would require ablating covariance prediction while keeping the EKF integration fixed."],"forward_implications":["A single-shot APR branch trained with covariance prediction can match attention-based APR baselines, so adding uncertainty outputs does not sacrifice accuracy.","Temporal positioning with EKF fusion beats temporal APR methods (MapNet, AtLoc+) and RNN-based integration (LSTM-KF, ViPR), indicating the learned filter, not extra trainable recurrency, is the effective ingredient.","Because the APR and RPR branches are trained independently, one branch can be swapped or upgraded without retraining the entire system.","The predicted covariances give a confidence estimate for each pose, which can inform downstream decisions in navigation, loop closure, and sensor fusion.","EKF-based fusion is computationally lighter than pose-graph optimization, making the accuracy gain available in real-time applications."],"supporting_citations":[{"why":"Defines the APR task and provides the PoseNet baseline that single-shot VKFPos is compared against.","marker":"(Kendall, Grimes, and Cipolla 2015)"},{"why":"Introduces learned loss weighting for APR, the line of uncertainty-aware training VKFPos extends with full covariance prediction.","marker":"(Kendall and Cipolla 2017)"},{"why":"AtLoc, the attention-guided APR baseline whose architecture VKFPos adopts and whose performance it matches.","marker":"(Wang et al. 2020)"},{"why":"Provides the diagonal-covariance assumption and uncertainty-aware pose regression that VKFPos reuses for its covariance matrices.","marker":"(Moreau et al. 2022)"},{"why":"KFNet, the prior EKF-based learning integration method VKFPos distinguishes itself from on Bayesian-principle grounds.","marker":"(Zhou et al. 2020)"},{"why":"LSTM-KF, a model-based RNN integration baseline VKFPos outperforms in temporal positioning.","marker":"(Coskun et al. 2017)"},{"why":"ViPR, an RNN-based APR/RPR integration baseline VKFPos beats and uses to motivate optimization-based fusion.","marker":"(Ott et al. 2020)"},{"why":"Supplies the se(3) Lie-algebra representation and manifold operations used in the pose heads and EKF update.","marker":"(Sola, Deray, and Atchuthan 2018)"},{"why":"Supports the claim that EKF is computationally efficient and suitable for real-time integration.","marker":"(LaViola 2003)"}],"fun_headline_variants":["Learning to fuse absolute and relative poses with an EKF","Monocular positioning gets a Kalman boost from learned covariances","EKF integration of APR and RPR yields sharper pose tracking","Variational Bayesian EKF: fusing pose estimators for better localization","Learning both pose and uncertainty for EKF-based monocular tracking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The relative-pose branch's predicted means and covariances are trustworthy enough to serve as the filter's motion model, even though that branch is trained only to minimize its own likelihood loss and is never calibrated for filtering.","fun_headline_variants_meta":{"raw":{"variants":["Learning to fuse absolute and relative poses with an EKF","Monocular positioning gets a Kalman boost from learned covariances","EKF integration of APR and RPR yields sharper pose tracking","Variational Bayesian EKF: fusing pose estimators for better localization","Learning both pose and uncertainty for EKF-based monocular tracking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000884,"raw_usage":{"total_tokens":3814,"prompt_tokens":939,"completion_tokens":2875,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":2798}},"tokens_in":555,"tokens_out":2875,"duration_ms":17587,"temperature":1.0,"reasoning_tokens":2798,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T21:41:44.166345+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out trajectory, compare the relative-pose branch's predicted covariance $\\Sigma_{u,t-1}$ with the empirical covariance of its residuals against ground-truth relative poses. If the predicted uncertainty is systematically off-scale or the residuals have non-zero mean, the EKF-corrected trajectory should degrade relative to the APR-only branch, which would show that the learned control-input model is not statistically consistent enough for the filter to do the work the paper claims.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the claim that EKF is computationally efficient and suitable for real-time integration."},{"cited_title":"X.; Zhao, P.; Trigoni, N.; and Markham, A","cited_arxiv_id":null,"evidence_quote":"AtLoc, the attention-guided APR baseline whose architecture VKFPos adopts and whose performance it matches."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the diagonal-covariance assumption and uncertainty-aware pose regression that VKFPos reuses for its covariance matrices."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"KFNet, the prior EKF-based learning integration method VKFPos distinguishes itself from on Bayesian-principle grounds."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LSTM-KF, a model-based RNN integration baseline VKFPos outperforms in temporal positioning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ViPR, an RNN-based APR/RPR integration baseline VKFPos beats and uses to motivate optimization-based fusion."}],"review_version":1}