{"id":"e47ed7b7-79f1-4c5c-9844-1abbdb9f1293","arxiv_id":"2506.18204","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FMF-SLAM extends DROID-SLAM with Fourier-domain attention and cross-modal knowledge distillation for RGB-depth fusion, reporting fewer tracking failures and lower average trajectory error on dark, dim, and fast-moving test sequences.","lead":"FMF-SLAM combines RGB and depth cameras inside a visual SLAM system, using fast Fourier transforms in its attention modules to keep computation light while tracking in dark or changing light. A reader might care because robust, low-cost navigation in bad lighting is a practical bottleneck for security robots and autonomous vehicles.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline SOTA comparison uses RGB-only DROID-SLAM against RGB-D FMF-SLAM; without a depth-enabled DROID-SLAM baseline the claimed 1.07 vs 6.02 cm advantage is uncontrolled.","rationale":"The reader's weakest assumption focused on whether Fourier attention preserves correlation information and on the absence of a standard attention baseline. That is a legitimate concern, but it is secondary to a more fundamental experimental control problem: the headline comparisons in Tables I and II do not give DROID-SLAM access to depth even though DROID-SLAM supports RGB-D input. The paper's own Table I header explicitly lists DROID-SLAM as RGB and FMF-SLAM as RGBD. Since the central claim is state-of-the-art localization under challenging conditions, the burden is on the authors to show the advantage persists when the baseline receives the same sensor information. The current evidence cannot distinguish between the effect of the proposed architecture and the effect of an additional input channel. This is not an accusation of dishonesty; it is a request for a controlled baseline that the paper omits. The ablation study in Table IV only shows incremental improvements within the proposed encoder, and it does not include a softmax attention condition, so it cannot substantiate the attention-specific claims either. The concrete test of running DROID-SLAM(RGB-D) is feasible, fast, and directly settles whether the 1.07 cm versus 6.02 cm comparison is meaningful. If the RGB-D baseline closes the gap, then the paper's contribution reduces to a marginal engineering improvement rather than a state-of-the-art result. Therefore the verdict should remain conditional, but with the condition explicitly requiring a depth-enabled DROID-SLAM baseline and a standard attention ablation before the SOTA claim is accepted. My read partially agrees with the reader because the reader identified an evidence gap in the ablation, but the missing modality-controlled baseline is a distinct and more load-bearing issue.","tokens_in":11740,"tokens_out":3963,"duration_ms":39993,"concrete_test":"Run DROID-SLAM in its RGB-D mode on every sequence of Tables I and II using the same TartanAir training split and the same initialization as the paper, then compute per-sequence ATE, the number of outliers under the paper's criterion, and the average ATE without outliers. If DROID-SLAM(RGB-D) achieves comparable or lower average ATE or at least as many successfully tracked sequences, the claimed state-of-the-art advantage is not supported. As a second check, add an ablation to Table IV that replaces Fourier attention in FMF-SLAM with standard dot-product softmax attention while keeping all other components fixed; if the softmax variant matches or beats the Fourier variant, the paper's central architectural claim is contradicted.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Table I and Table II label DROID-SLAM as using RGB input only, while FMF-SLAM uses RGB-D input. DROID-SLAM is explicitly designed to support RGB-D input, and its original evaluation includes RGB-D sequences from TUM. The paper never compares against DROID-SLAM with depth on the same TartanAir or Realtest-GT sequences. Consequently, the headline claim of state-of-the-art robustness and accuracy, including the zero-outlier result and the average ATE of 1.07 cm versus 6.02 cm, may be fully explained by the additional depth modality rather than by the proposed Fourier attention, multi-scale distillation, or any other architectural contribution. The ablation study in Table IV is internal to the proposed encoder and does not include a depth-aware DROID-SLAM baseline or a standard softmax attention variant, so it cannot isolate whether the reported gains come from the Fourier attention mechanism. On standard TUM sequences in Table I, FMF-SLAM is actually slightly worse than RGB-only DROID-SLAM (e.g., fr1/desk 1.73 versus 1.63, fr3/office 1.28 versus 1.18), which further suggests the TartanAir advantage may stem from the extra depth stream rather than from the claimed architectural novelty. The central claim is therefore not supported by a controlled comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FMF-SLAM, a learning-based RGB-D SLAM system built on DROID-SLAM, with three claimed contributions: a Fourier-based self- and cross-attention mechanism for multimodal feature extraction, a multi-scale knowledge distillation loss between RGB and depth branches, and integration with GNSS-RTK and global bundle adjustment for deployment on a security robot. The method is evaluated on TUM, TartanAir (with held-out challenging sequences), and the authors' own Realtest-GT and Realtest-Visual datasets. The central claim is that FMF-SLAM achieves state-of-the-art localization robustness and accuracy under fast motion, dim lighting, and darkness, with zero tracking failures on the selected challenging sequences and an average ATE of 1.07 cm versus 6.02 cm for DROID-SLAM, while running at about 100 ms per frame on a robot.","tokens_in":12004,"tokens_out":4217,"duration_ms":42816,"significance":"If the central claims hold, the paper would be a useful demonstration that an efficient Fourier-domain mixing operation can replace dot-product attention in a learning-based SLAM front-end and that RGB-D multimodal fusion improves robustness in visually degraded scenes. The paper has notable strengths: it reports results on external benchmarks (TUM, TartanAir), uses a held-out TartanAir test split disjoint from training, provides a quantitative ablation (Table IV), reports parameter counts and inference times, and states that code and datasets will be released. However, the headline comparison is not controlled: FMF-SLAM uses RGB-D input while DROID-SLAM is evaluated with RGB only, although DROID-SLAM natively supports RGB-D input. The ablation does not include a standard softmax-attention baseline, so the specific benefit of the Fourier attention operation is not isolated. These issues are load-bearing for the claims of state-of-the-art robustness and of Fourier attention as the cause of the gains.","major_comments":[{"comment":"The principal comparison is uncontrolled with respect to input modality. DROID-SLAM is listed as using RGB input only while FMF-SLAM uses RGB-D, and DROID-SLAM natively supports RGB-D cameras (Teed and Deng, NeurIPS 2021). Without a DROID-SLAM RGB-D baseline evaluated on the same TartanAir and Realtest-GT sequences, the zero-outlier result and the average ATE of 1.07 cm versus 6.02 cm cannot be attributed to the proposed Fourier attention or distillation; the additional depth stream alone could explain the difference. The TUM rows, where FMF-SLAM is slightly worse than RGB-only DROID-SLAM (fr1/desk 1.73 vs. 1.63, fr3/office 1.28 vs. 1.18), reinforce this concern. Please add the missing RGB-D baseline or substantially rephrase the claims so that they do not imply an architectural advantage over DROID-SLAM.","section":"Tables I and II"},{"comment":"The Fourier attention mechanism is not fully specified and is not demonstrated to act as attention. As written, Eq. (2) is an element-wise product in the Fourier domain followed by inverse FFT, which is a circular correlation/convolution operation; there is no softmax and no learned attention weight. The text says \"FFT(·) denotes the conjugate transpose operation\" but the symbols in Eqs. (2) and (5) do not make clear which operand is conjugated or along which axis the FFT is applied (spatial dimensions, channel dimension, or both). This makes the method irreproducible as stated. In addition, Table IV Exp. 3 vs. Exp. 4 only compares Fourier attention against no attention; a standard dot-product/softmax attention or a linear attention baseline is needed to support the claim that the Fourier-domain mixing preserves the correlation information needed for optical flow in SLAM.","section":"Section III-A, Eqs. (2)-(6)"},{"comment":"Only the cross-attention output for the RGB branch (V_r) is defined. The claim that cross-attention is applied bidirectionally (\"RGB ↔ depth\") requires an analogous equation for the depth branch, V_d, which is never given. Without this equation, the bidirectional exchange and the exact inputs to the distillation losses are not fully specified. Please provide the missing expression and clarify how the two cross-attention branches are combined in the loss.","section":"Section III-A, Eq. (5)"},{"comment":"All quantitative results are reported as single runs with no error bars, standard deviations, or number of random seeds. Several reported differences are small relative to plausible run-to-run variation (e.g., fr1/xyz 0.96 vs. 1.04, OfficeP05 0.22 vs. 0.21, EndofworldP06 0.45 vs. 0.58), so the claimed improvements are not statistically supported. Please report mean ± std over multiple runs or otherwise justify that the evaluation procedure is deterministic and the differences are significant.","section":"Tables I, II, and IV"},{"comment":"The real-time and deployment claims need more context. \"100.85 ms per frame\" is stated without specifying the sensor frame rate, the hardware pipeline, the input resolution, or whether the time includes depth estimation. The TartanAir figure of 77.76 ms per frame is similarly reported without a runtime comparison to DROID-SLAM on the same machine. Moreover, the outdoor robot experiments use ZoeDepth-estimated depth rather than sensor depth, so they do not directly validate the RGB-D front-end. Please provide a runtime comparison table and state the estimated-depth limitation explicitly.","section":"Section IV-D"}],"minor_comments":[{"comment":"Reference [9] attributes \"Attention is all you need\" to Mayer et al. at NIPS; the correct authors are Vaswani et al., NeurIPS 2017. Reference [15] gives the TartanAir year as 2012, but the dataset was published in 2020. Please correct these citations.","section":"References"},{"comment":"There is a typo, \"direct direct\", in the first paragraph; please fix it.","section":"Section II-A"},{"comment":"The distillation losses are hard to parse because of ambiguous norm notation and unclear summation ranges; for example, in Eq. (8) it is not clear whether the 1/HW average is applied to each feature map before the norm, and in Eq. (9) the argument of the norm mixes tensor and scalar quantities. Please rewrite these equations with explicit indexing.","section":"Eqs. (7)-(9)"},{"comment":"The block diagram is nearly unreadable in the PDF: the fonts are tiny and the connections are dense. Please provide a higher-resolution figure with larger labels.","section":"Figure 1"},{"comment":"The claim \"the first learning-based multimodal SLAM system\" is too strong; DROID-SLAM already supports RGB-D input, and there are other learning-based RGB-D SLAM systems. Please qualify this as \"the first to our knowledge\" and cite the relevant prior work.","section":"Contributions, page 2"},{"comment":"The definitions of ACC1px, AEPE2D, Rot0.1°, and Tra0.01m cite reference [7], which is the authors' own paper. Please define these metrics directly in the text or cite the original sources.","section":"Section IV-E"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an accepted RAL version, so my comments are aimed at the archival version. The main issue, the missing DROID-SLAM RGB-D baseline, is fixable within the scope of the paper, and the code/data release commitment is a positive signal. I do not see grounds for rejection, but the central architectural claim needs the additional controlled comparison before the paper can be considered sound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper is a serious system effort: FMF-SLAM takes DROID-SLAM and replaces the front-end with a two-branch RGB-D encoder using FFT-based self/cross-attention and multi-scale knowledge distillation, then validates on TUM, TartanAir, self-collected indoor data, and an outdoor robot with GNSS-RTK and global BA. Code and datasets are public. But the headline robustness claim has a load-bearing comparison gap: DROID-SLAM is evaluated RGB-only while FMF-SLAM uses RGB-D. Since DROID-SLAM natively supports depth, the zero-outlier result and the 1.07 vs 6.02 cm average ATE may simply reflect the extra depth stream, not the Fourier attention or distillation. On TUM, FMF-SLAM is actually slightly worse than RGB-only DROID-SLAM on several sequences, which strengthens the suspicion that the TartanAir advantage is modality-driven.\n\nWhat's genuinely new is the integrated system, not the ingredients. FFT-based token mixing and knowledge distillation are known, but combining them in a DROID-style SLAM pipeline for RGB-D localization is a reasonable contribution. The evaluation is broad, and the ablation in Table IV shows each module adds a small positive increment—that's honest. The real-world deployment with ~100 ms/frame and low accumulation error is useful evidence.\n\nThe soft spots are real but mostly fixable. The missing depth-enabled DROID baseline is the main one and should be ground for revision. There are also no error bars or multiple runs, so the ablation increments could be noise. No softmax-attention baseline appears in the ablation, so the efficiency/accuracy claim for Fourier attention is not proven. The Fourier attention notation in Eq. (2)–(5) is ambiguous—what exactly is conjugated and on which axes? Loss weights alpha, beta, delta are unreported. The abstract overstates the TUM results, where the method is roughly on par with DROID.\n\nWho is this for: people working on learning-based SLAM front-ends and multimodal fusion. It deserves serious peer review because the system is plausible and the code is available, but the SOTA claim needs a controlled baseline before the numbers should be trusted. I'd want to see that before citing it.","headline":"A useful RGB-D SLAM system, but the headline robustness claim compares RGB-only DROID-SLAM against RGB-D FMF-SLAM, so the advantage is uncontrolled.","tokens_in":12547,"tokens_out":3855,"would_cite":false,"duration_ms":33457,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FMF-SLAM claims that a SLAM front-end with Fourier-based attention and RGB-depth fusion can localize reliably in dark, dim, and fast-moving scenes, reporting zero tracking failures and an average trajectory error of 1.07 cm on the…","keywords":["visual SLAM","RGB-D fusion","Fourier attention","fast Fourier transform","knowledge distillation","optical flow","GNSS-RTK","real-time robotics"],"falsifier":"Train the identical two-branch encoder, loss schedule, and DROID-SLAM back-end, but replace Fourier attention with standard dot-product self- and cross-attention (with softmax). If that variant matches or exceeds FMF-SLAM's ACC1px of 82.04% and AEPE2D of 1.51 on TartanAir, or achieves comparable ATE on the challenging test split, then the Fourier operation is not the source of the reported accuracy and the efficiency claim needs a new benchmark. A second check is to replace the conjugate product in Eq. (2) with a plain FFT product and see whether flow quality collapses.","tokens_in":11506,"feed_emoji":"🤖","tokens_out":12167,"duration_ms":102672,"temperature":0.7,"pith_summary":"FMF-SLAM is a learning-based visual SLAM system designed to keep localizing reliably when the RGB image is noisy, dim, or dark. It claims that replacing the dot-product attention inside a DROID-SLAM-style front-end with attention computed via the fast Fourier transform, and fusing RGB with depth features through multi-scale knowledge distillation, gives both higher robustness and lower compute. On TUM and the challenging TartanAir splits it reports zero outliers across the 20 evaluated sequences and an average absolute trajectory error of 1.07 cm, against 6.02 cm for DROID-SLAM. The same front-end runs on a security robot at about 100 ms per frame, fusing GNSS-RTK positions through global bundle adjustment. If these results hold, frequency-domain attention is sufficient for dense optical flow in SLAM, which makes multimodal fusion practical on real robots.","feed_headline":"Fourier attention drives SLAM error down to 1.07 cm in dark scenes","feed_subtitle":"A depth-plus-RGB front-end with FFT attention reports zero tracking failures; DROID-SLAM's average error is 6.02 cm.","key_machinery":"The load-bearing component is the Fourier-based multimodal fusion encoder (FMF encoder), a two-branch RGB/depth feature extractor that replaces the front-end of DROID-SLAM. Its defining operation is Eqs. (2)--(5): for each branch, self-attention is computed as $\\mathrm{IFFT}\\big(\\mathrm{FFT}(Q_i)\\,\\overline{\\mathrm{FFT}(K_i)}\\big)$, a fast correlation in the frequency domain with no softmax and no learned attention weights, after which cross-attention exchanges queries and keys between branches. The multi-scale knowledge distillation loss then pulls the two branches into agreement across three granularities. This assembly is what carries the efficiency claim (77.76 ms per frame on TartanAir, 100.85 ms on the robot) and the robustness claim (zero tracking failures in the challenging test set).","core_discovery":"The paper's central discovery is that a SLAM front-end can compute self-attention and cross-modal attention entirely in the frequency domain, as element-wise products of FFTs rather than matrix dot products, and still extract the correlation information needed for dense optical flow. With RGB and depth branches, the self-attention map is $A_i = \\mathrm{IFFT}\\big(\\mathrm{FFT}(Q_i)\\,\\overline{\\mathrm{FFT}(K_i)}\\big)$, normalized and applied to values as $V_i = \\mathcal{N}(A_i)\\,V_i$, with a $1\\times1$ convolution residual; cross-attention then mixes the two modalities bidirectionally. A multi-scale distillation loss $L_k = \\alpha L_{L2} + \\beta L_s + \\delta L_c$ aligns the RGB and depth feature maps in $L2$, spatial, and channel terms. On TUM and on the TartanAir fast-moving, dimmed, and dark splits, FMF-SLAM reports zero outliers across the 20 evaluated sequences and an average ATE of 1.07 cm, versus one outlier and 6.02 cm for DROID-SLAM. The authors present this as evidence that an efficient Fourier-based multimodal encoder can replace expensive dot-product attention inside a learning-based SLAM front-end without sacrificing accuracy.","pith_inferences":["The paper does not compare its Fourier attention against standard dot-product attention under identical training, so the accuracy gains from the attention block could come from the depth branch and distillation rather than from the Fourier operation itself; a direct head-to-head would settle this.","Because the Fourier attention has no learned weights, it behaves as a fixed frequency-domain correlation; if that is the true source of the gains, similar FFT-based correlation layers could replace other costly matching modules in SLAM front-ends beyond attention.","The same two-branch encoder could transfer to other modality pairs, such as stereo or thermal-plus-RGB, wherever a second depth-like signal is available; this is a testable extension the paper does not pursue."],"forward_implications":["A learning-based SLAM front-end that consumes depth together with RGB can remain accurate where RGB-only systems fail: across the 20 tested sequences in Table I the FMF encoder produced zero tracking outliers.","Fourier-domain attention is an efficiency lever worth taking seriously: the full pipeline runs at 77.76 ms per frame on TartanAir and 100.85 ms per frame on the robot, within range of real-time deployment.","Fusing GNSS-RTK through global bundle adjustment keeps large outdoor loops tight, with accumulation errors of 0.09 m over 147.45 m and 0.26 m over 391.82 m.","On real indoor data under light-changing and dark conditions, the depth branch supplies information the RGB stream lacks, giving average ATE of 11.03 cm and 16.84 cm where RGB-only baselines diverge or fail."],"supporting_citations":[{"why":"DROID-SLAM is the learning-based SLAM system that FMF-SLAM extends and the principal comparison baseline.","marker":"[10]"},{"why":"Prior attentive multimodal fusion work that motivates the RGB-depth encoder and provides the ACC1px/AEPE2D evaluation measures.","marker":"[7]"},{"why":"TartanAir supplies the training data and the fast-moving, dimmed, and dark test splits behind the headline zero-outlier results.","marker":"[15]"},{"why":"TUM provides the standard RGB-D benchmark sequences on which the method is compared.","marker":"[14]"},{"why":"The fast Fourier transform is the efficiency mechanism that replaces dot-product attention in the proposed attention blocks.","marker":"[11]"},{"why":"Fast-FNet supports the memory-saving claim that conjugate symmetry allows storing only half of the FFT tensor.","marker":"[28]"},{"why":"ZoeDepth estimates depth monocularly for the outdoor robot runs, enabling the real-world deployment demonstration.","marker":"[33]"},{"why":"GNSS-RTK provides the global position measurements fused with visual SLAM for the robot integration.","marker":"[13]"},{"why":"Cross-modal knowledge distillation work that supports the multi-scale distillation loss aligning RGB and depth features.","marker":"[29]"}],"fun_headline_variants":["FFT attention SLAM: zero failures, 1.07 cm ATE","FMF-SLAM: Fourier attention for robust visual odometry","Frequency-domain attention cuts SLAM error to 1.07 cm","Multimodal fusion SLAM with FFT attention beats DROID","Efficient Fourier attention SLAM for dark scenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system's gains rest on the assumption that swapping the usual learned attention weights for a fixed frequency-domain multiplication costs no information needed for accurate optical flow, and the experiments never compare the two attention schemes directly.","fun_headline_variants_meta":{"raw":{"variants":["FFT attention SLAM: zero failures, 1.07 cm ATE","FMF-SLAM: Fourier attention for robust visual odometry","Frequency-domain attention cuts SLAM error to 1.07 cm","Multimodal fusion SLAM with FFT attention beats DROID","Efficient Fourier attention SLAM for dark scenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1505,"prompt_tokens":1028,"completion_tokens":477,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":386}},"tokens_in":644,"tokens_out":477,"duration_ms":4785,"temperature":1.0,"reasoning_tokens":386,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:54:08.715678+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the identical two-branch encoder, loss schedule, and DROID-SLAM back-end, but replace Fourier attention with standard dot-product self- and cross-attention (with softmax). If that variant matches or exceeds FMF-SLAM's ACC1px of 82.04% and AEPE2D of 1.51 on TartanAir, or achieves comparable ATE on the challenging test split, then the Fourier operation is not the source of the reported accuracy and the efficiency claim needs a new benchmark. A second check is to replace the conjugate product in Eq. (2) with a plain FFT product and see whether flow quality collapses.","supporting_citations":[{"cited_title":"DROID-SLAM: Deep Visual SLAM for Monoc- ular, Stereo, and RGB-D Cameras,","cited_arxiv_id":null,"evidence_quote":"DROID-SLAM is the learning-based SLAM system that FMF-SLAM extends and the principal comparison baseline."},{"cited_title":"Attentive multimodal fusion for optical and scene flow,","cited_arxiv_id":null,"evidence_quote":"Prior attentive multimodal fusion work that motivates the RGB-depth encoder and provides the ACC1px/AEPE2D evaluation measures."},{"cited_title":"TartanAir: A dataset to push the limits of visual slam,","cited_arxiv_id":null,"evidence_quote":"TartanAir supplies the training data and the fast-moving, dimmed, and dark test splits behind the headline zero-outlier results."},{"cited_title":"A bench mark for the evaluation of RGB-D SLAM systems,","cited_arxiv_id":null,"evidence_quote":"TUM provides the standard RGB-D benchmark sequences on which the method is compared."},{"cited_title":"Guest editors introduction to the top 10 algorithms,","cited_arxiv_id":null,"evidence_quote":"The fast Fourier transform is the efficiency mechanism that replaces dot-product attention in the proposed attention blocks."},{"cited_title":"3d mapping database-aided gnss rtk and its assessments in urban canyons,","cited_arxiv_id":null,"evidence_quote":"GNSS-RTK provides the global position measurements fused with visual SLAM for the robot integration."},{"cited_title":"DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distil- lation,","cited_arxiv_id":null,"evidence_quote":"Cross-modal knowledge distillation work that supports the multi-scale distillation loss aligning RGB and depth features."}],"review_version":1}