{"id":"37b4ce7a-2f14-4a5c-8635-3d8118672677","arxiv_id":"2501.08659","paper_version":4,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A brightness-guided Transformer with IMU-based pose graph optimization improves monocular VO accuracy on synthetic low-light and KITTI benchmarks, though the evaluation has significant flaws.","lead":"This paper introduces BrightVO, a visual odometry model that pairs a Transformer network with a brightness estimation module and an IMU-based optimization backend, and reports improved camera pose accuracy in normal and low-light driving scenes. It also presents KiC4R, a synthetic CARLA dataset spanning dusk, night, midnight, and extreme weather, which could help standardize low-light VO evaluation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"KITTI SOTA claim is not supported: Table 2 includes training sequences and, on held-out sequences, BrightVO loses to ORB-SLAM2 on 5 of 7 sequences.","rationale":"The reader's weakest_assumption is the fidelity of the synthetic KiC4R dataset to real low-light conditions. This is a valid concern for the low-light claim, but I find a more directly load-bearing flaw in the KITTI evaluation that supports the normal-light SOTA claim. The paper trains on KITTI sequences 01, 03, 07, 08 and then reports Table 2 over all ten sequences, including three training sequences. This inflates the average and contradicts the claim of consistent superiority: on held-out sequences, ORB-SLAM2 wins on five of seven. The reader's rationale does mention the training/test split issue, but the formal weakest_assumption field focuses on synthetic fidelity. Therefore I partially agree: the reader identified the problem in the rationale, but the identified weakest assumption is not the one I consider most load-bearing. My concern is concrete and testable: recomputing the average over held-out sequences only would settle it. This reinforces the REJECT verdict rather than changing it, hence UNCHANGED.","tokens_in":12197,"tokens_out":3470,"duration_ms":34284,"concrete_test":"Recompute the mean ATE over only the seven held-out KITTI sequences (00, 02, 04, 05, 06, 09, 10) using the values in Table 2, and compare BrightVO against ORB-SLAM2 and other baselines. If BrightVO's average is not below ORB-SLAM2, or if it loses on a majority of held-out sequences, the claimed 20% improvement is not established. Additionally, rerun the released DPVO model on the same sequences with the same scale alignment to verify whether the reported DPVO baseline numbers in Table 2 are reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of a 20% average improvement in normal outdoor pose estimation on KITTI rests on a flawed evaluation protocol. Section 4.2 states the model was trained on KITTI sequences 01, 03, 07, and 08. Table 2 then reports ATE on all ten sequences, including the training sequences 01, 07, and 08. Including training sequences in the test set inflates the reported average and is not a valid measure of generalization. More importantly, the paper's own table contradicts the claim that BrightVO 'consistently outperforms' ORB-SLAM2. On the held-out sequences, ORB-SLAM2 achieves lower ATE on 00 (1.3 vs 2.12), 04 (0.2 vs 0.44), 05 (0.8 vs 2.31), 06 (0.8 vs 2.7), and 10 (1.0 vs 2.11), while BrightVO only wins on 02 and 09. The 20% average improvement is therefore an artifact of mixing training sequences with test sequences and of comparing to implausible baseline numbers, such as DPVO's 111.97 m ATE on sequence 00, which is far outside published DPVO results. Since the KITTI evaluation is the sole support for the normal-light SOTA claim, the headline claim collapses once the training/test overlap is removed. The low-light claim on KiC4R is also not independently validated on real data, but the KITTI flaw alone is sufficient to reject the paper's central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BrightVO, a monocular visual odometry system combining a brightness-guided Vision Transformer front-end with a pose-graph-optimization back-end that fuses IMU (and, in the stated formulation, GNSS) measurements. The authors also introduce KiC4R, a synthetic CARLA-based dataset with dusk, night, midnight, and extreme-weather sequences. They report evaluations on KITTI and KiC4R and claim state-of-the-art performance, including a 20% average improvement over ORB-SLAM2 in normal outdoor scenes and a 25% improvement in low-light conditions, with code released publicly.","tokens_in":12525,"tokens_out":6579,"duration_ms":57731,"significance":"If the empirical claims were valid, BrightVO would be a useful contribution: combining a low-light-aware transformer front-end with a classical pose-graph refinement backend is a sensible design, and the release of code and a synthetic low-light dataset is a tangible asset for the community. The paper also evaluates with standard metrics (ATE, RPE) and includes an ablation of the refinement module. However, the central performance claims are not supported by the evidence as presented: the KITTI evaluation includes training sequences in the test set, the comparison with ORB-SLAM2 reverses on held-out sequences, baseline numbers are not credible without further protocol details, and the low-light claim is validated only on synthetic data from the same simulator used for training. These issues undermine both the normal-light and low-light state-of-the-art conclusions.","major_comments":[{"comment":"The KITTI evaluation is contaminated by training/test overlap. Section 4.2 states that the model was trained on KITTI sequences 01, 03, 07, and 08, and Table 2 then reports ATE on all ten sequences, including training sequences 01, 07, and 08. The claimed 20% average improvement over ORB-SLAM2 is an artifact of this overlap. On the seven held-out sequences (00, 02, 04, 05, 06, 09, 10), ORB-SLAM2 has lower ATE than BrightVO on five sequences (00: 1.3 vs 2.12; 04: 0.2 vs 0.44; 05: 0.8 vs 2.31; 06: 0.8 vs 2.7; 10: 1.0 vs 2.11). Removing the training sequences flips the comparison: the average ATE over the held-out sequences is about 1.86 m for ORB-SLAM2 versus about 2.07 m for BrightVO. The statement in Section 4.3 that BrightVO \"consistently outperforms\" ORB-SLAM2 is directly contradicted by the paper's own table.","section":"Section 4.2 / Section 4.3 / Table 2"},{"comment":"The baseline numbers are implausible as reported and no evaluation protocol is given. DPVO is listed with ATE values of 111.97 m on sequence 00, 123.40 m on sequence 02, and 115.90 m on sequence 08, which are far outside published DPVO results under standard KITTI evaluation protocols. DeepVO's 95.92 m on sequence 00 is similarly not contextualized. The paper does not state how scale alignment was performed for each baseline, whether the same trajectory alignment (e.g., evo with Sim(3)) was applied consistently, or which exact sensor data each baseline received. Without this information, the comparison is not reproducible. In addition, no repeated runs or error bars are reported for the learning-based methods, so the claimed margins cannot be assessed for statistical significance.","section":"Table 2 / Section 4.3"},{"comment":"The low-light state-of-the-art claim is supported only by experiments on the synthetic KiC4R dataset. The test sequences (00-03) are CARLA-generated scenes from the same simulator and follow the same collection pipeline as the training sequences (04-06), and no real-world low-light dataset is used for validation. The paper's abstract and conclusion generalize to \"low-light conditions\" broadly, but the evidence only covers synthetic dusk, night, midnight, and extreme-weather scenes. This external-validity gap is load-bearing for the paper's central low-light claim, especially given that the method is intended for real-world autonomous driving and robotics.","section":"Section 4.3 / Table 4"},{"comment":"The use of GNSS in the refinement module is underspecified. Equation (10) defines the backend objective with a GNSS-based IMU pose term, but Section 4.4 states that KiC4R does not include GNSS data. The paper never states whether GNSS corrections are active for the KITTI experiments, how the weighting parameter λ is chosen, or how the IMU integration (Eqs. (7)-(9)) handles bias and noise. Since the multi-modality refinement module is a central claimed contribution, the absence of this information makes the contribution difficult to evaluate and reproduce.","section":"Section 3.2 / Eq. (10) / Section 4.4"}],"minor_comments":[{"comment":"There is an inconsistency in the claimed low-light improvement: the abstract in the paper header says \"259%\" while the abstract body, the introduction, and the contributions list say \"25%\". This needs to be corrected.","section":"Abstract / Introduction"},{"comment":"TartanVO is listed in Section 4.2 as a comparison method but has no row in Table 2; it appears only in Tables 3 and 4 for selected sequences, with no explanation of why it is absent from the full KITTI ATE table.","section":"Table 2"},{"comment":"The RPE formula is ambiguous: T and \\hat{T} are described as translation vectors but are written as transformation matrices, and the metric mixes rotational and translational components without a clear definition. The selection of sequences 06, 07, 09, 10 for the KITTI RPE comparison in Table 3 is also not justified.","section":"Eq. (12) / Section 4.2"},{"comment":"The attention formulation in Eq. (4) uses V both for image-derived values and for the reshaped brightness feature Fbr, and the notation Atten(Q, K, V, V) with (V ⊙ V) makes the roles of the two V terms unclear. The learnable scaling α should at least be described in terms of initialization and dimension (e.g., relation to 1/sqrt(d_k)).","section":"Eq. (4)"},{"comment":"The paper cites [Keimel et al., 2012] for the TUM dataset, but that reference is the TUM high-definition video dataset, not the TUM RGB-D dataset commonly used for VO/SLAM evaluation; the correct reference should be provided. The Light-SLAM discussion also cites Sarlin et al. for LightGlue, but LightGlue is a different work and should be cited correctly.","section":"Section 4.1 / References"},{"comment":"There are several typographical errors and unclear captions, including \"Sence\" in Table 3, \"adpoted\" in Section 4.2, \"optimizor\" in Section 4.2, and the Figure 6 caption stating \"(d)-(f) represent sequence 00-03\" while showing three plots for four sequences.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The training/test overlap in the KITTI evaluation is the decisive issue: removing the training sequences reverses the headline comparison against ORB-SLAM2, so the central normal-light claim is not merely under-supported but contradicted by the paper's own data. The implausible baseline values and the absence of real-world low-light validation compound the problem. If the authors re-run the evaluation on a clean held-out split, provide reproducible baseline numbers, and add real-world low-light validation, a substantially revised submission could be reconsidered, but the current manuscript does not support its stated contributions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the central claim doesn't survive its own table. The model was trained on KITTI 01, 07, and 08, and Table 2 includes those very sequences in the test set. On the held-out seven sequences (00, 02, 04, 05, 06, 09, 10), ORB-SLAM2 has lower ATE on five of them; BrightVO only wins on 02 and 09. The paper's sentence claiming it \"consistently outperforms\" ORB-SLAM2 is simply false by the numbers printed next to it. The DPVO baseline of 111.97 m on sequence 00 is also far outside any published DPVO result, so the whole comparison table is unreliable.\n\nThe genuinely new piece is KiC4R—a synthetic low-light VO dataset with 37k CARLA images in dusk/night/midnight/rain, plus IMU data in KITTI format. That is a useful resource. The architecture, a brightness-guided ViT on the front end and pose-graph optimization with IMU on the back end, is a reasonable combination of known components, and the ablation shows the refinement module helps. They also released code and data, which counts in their favor.\n\nSoft spots beyond the evaluation: Eq. (4) defines attention as (V ⊙ V) softmax(K^T Q / α), which is dimensionally inconsistent unless V is reshaped; as written it's not a standard attention and would be hard to reproduce. The low-light claim is validated only on synthetic KiC4R, with no real-world night driving data, so the generalization claim is unproven. Minor: the abstract says 259% low-light improvement while the body says 25%.\n\nWho is this for: anyone working on learning-based VO under poor lighting, particularly those interested in synthetic training data. The dataset and code are worth a look, but the reported numbers should not be cited. I'd send this to peer review rather than desk reject, because the dataset is concrete and the evaluation mistakes are fixable—but the referee should demand a corrected protocol, real baselines, error bars, and a worked-out attention definition. As is, it's not acceptable.","headline":"The KITTI SOTA claim is an artifact of testing on training sequences; the paper's own table shows ORB-SLAM2 winning on 5 of 7 held-out sequences, but the new synthetic low-light dataset is a genuine contribution.","tokens_in":13045,"tokens_out":5753,"would_cite":false,"duration_ms":50766,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BrightVO claims a 20% average pose-error reduction outdoors and 25% in low-light conditions by combining a brightness-guided Vision Transformer with IMU-based pose-graph refinement.","keywords":["visual odometry","low-light driving","Transformer","brightness estimation","IMU fusion","pose graph optimization","synthetic dataset","CARLA"],"falsifier":"Evaluate the released BrightVO model on a real-world nighttime driving sequence with ground-truth poses and compare its ATE/RPE against the same baselines used in the paper; if the roughly 20-25% error reduction does not appear, the synthetic-to-real assumption fails. A second check is to ablate the brightness estimator on KiC4R and see whether relative pose error changes at all.","tokens_in":11975,"feed_emoji":"🌙","tokens_out":9715,"duration_ms":79259,"temperature":0.7,"pith_summary":"The paper proposes BrightVO, a visual odometry model aimed at low-light conditions. It uses a Vision Transformer whose self-attention is guided by a brightness estimator, plus a back-end pose graph optimization that fuses IMU data to refine trajectories. The authors argue this combination yields state-of-the-art pose accuracy on the KITTI benchmark (about 20% average reduction in absolute trajectory error) and on a new synthetic low-light dataset called KiC4R (about 25% reduction in relative pose error). They also introduce KiC4R, a CARLA-simulated dataset with dusk, night, midnight, and extreme-weather sequences, to enable low-light VO training and evaluation.","feed_headline":"BrightVO cuts pose error 20% outdoors, 25% in low light","feed_subtitle":"A brightness-guided ViT plus IMU pose-graph refinement lifts accuracy on KITTI and on a synthetic low-light benchmark.","key_machinery":"BrightVO has two main components. The front-end is a ViT-based encoder where a brightness estimator computes a per-pixel illumination prior, and self-attention is computed as $\\operatorname{Atten}(Q,K,V,V) = (V \\odot V)\\operatorname{softmax}(K^\\top Q / \\alpha)$, with the brightness features folded into the values so the model can attend to illumination structure. The back-end is a pose graph optimization (PGO) solved with Levenberg–Marquardt using the PyPose library, which fuses the VO motion with pre-integrated IMU measurements (and GNSS position when available) through a weighted error term, iteratively refining the poses. The brightness estimator is an adaptation of Retinexformer's illumination module, simplified for the VO task.","core_discovery":"On its own terms, the paper's central claim is that jointly learning brightness features inside a Vision Transformer and then refining VO estimates with an IMU-driven pose-graph optimizer produces consistently lower pose error than existing geometry-based, learning-based, and Transformer-based VO systems, in both normal outdoor and low-light driving scenes. The evidence is the ATE/RPE tables on KITTI and KiC4R, where BrightVO reports an average ATE of 2.18 m on KITTI sequences versus 2.75 m for ORB-SLAM2, and smaller relative errors on all four KiC4R low-light test sequences. The authors attribute the gains to the brightness-guided self-attention for feature extraction and the iterative multi-modality refinement that corrects drift, especially on long sequences.","pith_inferences":["If the synthetic-to-real gap for low-light scenes is small, the same brightness-guided attention design could transfer to other nighttime perception tasks, such as place recognition or object detection, though the paper does not test those settings.","Because the refinement error term includes GNSS positions when available, the KITTI results may partly reflect GNSS-informed corrections; how much of the gain comes from GNSS rather than IMU is not isolated by an ablation.","A real-world nighttime evaluation would be the decisive test of the 25% low-light improvement, since KiC4R is synthetic and the paper reports no real low-light validation.","The value-side brightness injection is a minimal architectural change that could be lifted into other Transformer vision models for illumination-degraded inputs; the paper does not explore that transfer."],"forward_implications":["A single BrightVO model can serve both normal and low-light driving scenes without switching between algorithms, since the same brightness-guided front-end and refinement back-end handle both conditions.","The KiC4R dataset provides a KITTI-format benchmark with dusk, night, midnight, and extreme-weather sequences, enabling direct comparison of VO methods under low-light conditions.","The combination of a learned front-end and a classical graph-optimization back-end shows a reusable pattern: deep pose proposals refined by inertial constraints reduce drift on long sequences.","Removing the refinement module causes significant drift in both normal and low-light conditions, while short sequences remain accurate, so the IMU back-end matters most for long trajectories."],"supporting_citations":[{"why":"Supplies the pretrained ViT base ('vit-base-patch16-224') that BrightVO fine-tunes for pose estimation.","marker":"[Dosovitskiy, 2020]"},{"why":"Provides the Transformer/self-attention architecture that the VO encoder is built on.","marker":"[Vaswani, 2017]"},{"why":"Source of the brightness estimator design, which BrightVO simplifies for low-light feature extraction.","marker":"[Cai et al., 2023]"},{"why":"PyPose, used to implement and solve the pose graph optimization in the refinement module.","marker":"[Wang et al., 2023]"},{"why":"ORB-SLAM3, a geometry-based baseline that BrightVO compares against on KITTI and KiC4R.","marker":"[Campos et al., 2021]"},{"why":"TartanVO, a learning-based baseline used for RPE comparison on both datasets.","marker":"[Wang et al., 2021]"},{"why":"DPVO, a recent deep VO baseline included in the KITTI and KiC4R comparisons.","marker":"[Teed et al., 2024]"},{"why":"DPV-SLAM, a deep patch visual SLAM baseline that BrightVO compares against on KITTI.","marker":"[Lipson et al., 2025]"},{"why":"KITTI benchmark, the standard dataset that grounds the normal-outdoor pose accuracy claims.","marker":"[Geiger et al., 2012]"},{"why":"CARLA simulator, used to generate the KiC4R low-light dataset with dusk, night, midnight, and extreme-weather scenes.","marker":"[Dosovitskiy et al., 2017]"}],"fun_headline_variants":["BrightVO: Transformer VO fuses brightness and IMU to cut drift","BrightVO: Brightness-guided attention plus IMU pose graph for VO","BrightVO: IMU-refined Transformer VO improves pose accuracy in dark scenes","BrightVO: Low-light VO gains from brightness-aware Transformer and IMU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The synthetic KiC4R dataset is assumed to faithfully represent real low-light driving, so the measured 25% low-light improvement would carry over to real nighttime operation.","fun_headline_variants_meta":{"raw":{"variants":["BrightVO: Transformer VO fuses brightness and IMU to cut drift","BrightVO: Brightness-guided attention plus IMU pose graph for VO","BrightVO: IMU-refined Transformer VO improves pose accuracy in dark scenes","BrightVO: Low-light VO gains from brightness-aware Transformer and IMU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001609,"raw_usage":{"total_tokens":6423,"prompt_tokens":979,"completion_tokens":5444,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":5364}},"tokens_in":595,"tokens_out":5444,"duration_ms":38920,"temperature":1.0,"reasoning_tokens":5364,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:20:38.377734+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the released BrightVO model on a real-world nighttime driving sequence with ground-truth poses and compare its ATE/RPE against the same baselines used in the paper; if the roughly 20-25% error reduction does not appear, the synthetic-to-real assumption fails. A second check is to ablate the brightness estimator on KiC4R and see whether relative pose error changes at all.","supporting_citations":[{"cited_title":"Retinexformer: One-stage retinex-based transformer for low-light image enhancement","cited_arxiv_id":null,"evidence_quote":"Source of the brightness estimator design, which BrightVO simplifies for low-light feature extraction."},{"cited_title":"Orb-slam3: An accurate open-source library for visual, visual--inertial, and multimap slam","cited_arxiv_id":null,"evidence_quote":"ORB-SLAM3, a geometry-based baseline that BrightVO compares against on KITTI and KiC4R."},{"cited_title":"Deep patch visual odometry","cited_arxiv_id":null,"evidence_quote":"DPVO, a recent deep VO baseline included in the KITTI and KiC4R comparisons."},{"cited_title":"Deep patch visual slam","cited_arxiv_id":null,"evidence_quote":"DPV-SLAM, a deep patch visual SLAM baseline that BrightVO compares against on KITTI."},{"cited_title":"Are we ready for autonomous driving? the kitti vision benchmark suite","cited_arxiv_id":null,"evidence_quote":"KITTI benchmark, the standard dataset that grounds the normal-outdoor pose accuracy claims."}],"review_version":1}