{"id":"9b97cbfc-8d6b-407b-9f85-14772bac5e53","arxiv_id":"2505.06483","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A hierarchical multi-modal SLAM system, proven in the DARPA SubT finals, is described and released with code and a 740-meter underground dataset.","lead":"CompSLAM fuses visual, thermal, LiDAR, inertial, and leg-kinematic data into a hierarchical SLAM system that kept Team Cerberus's robots localized during the winning DARPA Subterranean Challenge run. The paper releases the code and a 740-meter ANYmal dataset from the finals course, giving the robotics community a benchmark for degraded underground environments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Resilience claim rests on a leg-odometry fallback that is neither quality-monitored nor exercised under combined multi-modal degradation.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing vulnerability: CompSLAM's resilience depends on the premise that not all sensor inputs degrade at once, and the final fallback, leg odometry, is assumed rather than demonstrated. My examination confirms this and sharpens it with a specific mechanism: the architecture has no monitor for leg-odometry quality. The LO and LM consistency checks only reject implausibly large motion deviations, not slow drift or slip bias, so a degraded leg-odometry estimate can silently become the active pose source and propagate error into VTIO re-initialization and the map. The evaluation does not exercise this failure mode: the dataset is challenging but not adversarially combined, and no ablation or instrumentation shows when and how often the fallback actually contributes. This does not invalidate the system-description contribution or the dataset release, which are valuable and independently supported by the DARPA SubT deployment. It does, however, mean the quantitative robustness and accuracy claims are conditional on a premise the evidence does not yet test. Since the reader already reached CONDITIONAL and my concern reinforces that position, I recommend no change to the verdict.","tokens_in":11739,"tokens_out":3872,"duration_ms":44060,"concrete_test":"Run CompSLAM on the released dataset with all non-kinematic modalities degraded at once in a selected self-similar tunnel segment: blank or darken visual and thermal images, downsample or remove LiDAR returns in one field of view to simulate dust/occlusion, and keep only IMU plus leg odometry. Compare the produced trajectory to the provided DARPA ground-truth/ICP reference, with and without the leg-odometry prior enabled. If the leg-odometry-only fallback yields RPE above, say, 5% over 25 m in that segment, or if removing the leg-odometry prior changes nothing because it is never selected, the claimed resilience under simultaneous sensor degradation is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"CompSLAM's central resilience claim is load-bearing on the premise in Section III.A that not all sensor inputs degrade simultaneously, but the fallback that remains when they do — leg odometry — is an unmonitored single point of failure. In III.C.a, the LO accepts an external pose only if its relative motion is 'within motion bounds allowed by the robot's controller'; in III.C.d, the LM 'independently validates' fallback pose consistency. Both checks can catch gross jumps but cannot detect slowly accumulating drift or slip-induced bias in kinematic odometry, which is exactly the failure mode of legged odometry on uneven or slippery terrain. The VTIO D-optimality re-initialization (III.B.e) re-anchors VTIO to the broader CompSLAM pose, so if that pose has drifted due to a bad leg-odometry fallback, the failure propagates. The released dataset is genuinely challenging (Section IV.A lists darkness, self-similar tunnels, IMU saturation, motion blur, moving people), but it is one run in which the system happened to stay accurate; Section IV.B reports no segment where LiDAR degeneracy fires and leg odometry is actually the active source, no sensor-degradation injection, no leg-odometry ablation, and no statistical repetition. Thus the paper supports 'CompSLAM worked on this SubT sequence' but not the stronger claim 'regardless of which sensors fail, pose remains reliable.' The DARPA deployment is credible evidence, but a single successful run cannot establish the redundancy premise under simultaneous multi-modal degradation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"CompSLAM is a hierarchical multi-modal SLAM framework that fuses visual, thermal, LiDAR, inertial, and leg-kinematic measurements through three stages: an IEKF-based Visual-Thermal-Inertial Odometry module (Section III.B), a LiDAR-Inertial Odometry module with a LOAM-style scan-to-scan front-end and scan-to-submap mapping (Section III.C), and external fallback pose handling. The paper claims real-time, robust onboard pose estimation in degraded underground environments, supported by deployment on all Team Cerberus robots in the DARPA SubT winning final run and by an evaluation on a new 740 m ANYmal dataset from the finals course. The dataset release and open-source code are presented as contributions.","tokens_in":12007,"tokens_out":3350,"duration_ms":33053,"significance":"Verified hardware deployment in a DARPA-winning team is meaningful evidence of practical utility. If the robustness claims held, the work would be a useful reference architecture for complementary multi-modal sensor fusion in subterranean robotics, and the dataset would be a valuable benchmark for degraded underground perception. The manuscript is unusually open about its limitations (e.g., no dynamic-object filtering, D-optimality re-initialization not triggered in the final event), and it ships machine-checkable artifacts: public code and public dataset. However, the quantitative evaluation is too thin to sustain the full resilience claim.","major_comments":[{"comment":"The trajectory reference is an offline ICP registration of LiDAR point clouds with the dense ground-truth map. Because CompSLAM is itself LiDAR-based, this is a same-modality comparison that can mask drift, especially in the self-similar tunnel sections where LiDAR degeneracy is claimed. The map metrics in Tables II–III are also computed after dense ICP alignment of S_C to S_GT, which absorbs global misalignment. Please provide an independent reference (e.g., surveyed markers, visual/thermal structure-from-motion, or at least a comparison against a non-LiDAR trajectory estimate) and report absolute trajectory error without the alignment step.","section":"Section IV.B, Tables II–IV"},{"comment":"The evaluation is a single teleoperated run with no baseline SLAM system, no ablation of the modules, and no repeated trials. The core claim (Section III.A) is that resilience holds when sensor inputs degrade, but Tables II–IV give one aggregate snapshot. Please add controlled ablations: disabling VTIO, disabling the leg-odometry fallback, disabling degeneracy detection; and report per-segment statistics on the sections where LiDAR degeneracy, darkness, IMU saturation, and motion blur actually occur. The competition deployment is not a substitute for this controlled evidence.","section":"Section IV.B and IV.A"},{"comment":"The quality checks on the leg-odometry fallback only verify that relative motion lies within controller bounds and that the LM's consistency check passes; neither detects slow drift or slip-induced bias in kinematic odometry, the likely failure mode on uneven or slippery terrain. Since the D-optimality re-initialization in III.B.e re-anchors VTIO to the broader CompSLAM pose, any drift entering through the fallback propagates. The paper never reports a segment in which LiDAR degeneracy actually fired and leg odometry was the active source, so the redundancy premise of Section III.A remains untested under exactly the combined degradation it is meant to handle.","section":"Section III.C.a and III.C.d"},{"comment":"The paper notes that the D-optimality pose quality check 'was not triggered during the actual DARPA SubT Finals event' and the released dataset evaluation reports no activation statistics. As a safety mechanism central to the resilience story, its behavior should be demonstrated on the released sequence (or another) where the filter is actually degraded.","section":"Section III.B.e and IV"}],"minor_comments":[{"comment":"The caption appears garbled: 'the different modules and passing of the estimated odometry as the initial guess of the next module.' Please rephrase.","section":"Figure 2 caption"},{"comment":"The entries '3× Camera Grayscale 20' and '4× Camera RGB 0.5' are ambiguous; specify the number of cameras, resolution, and whether the listed frequency is per camera.","section":"Table I"},{"comment":"The sentence 'segmenting the first tunnel, including the three-way junction from S_C and S_GT' is missing a comma or connecting word; please clarify whether both point clouds are segmented.","section":"Section IV.B"}],"recommendation":"major_revision","confidential_remarks":"The paper is a system paper with strong field validation but thin controlled evaluation. I recommend major revision, primarily to add ablations and an independent trajectory reference. The authors' candor about untriggered mechanisms is appreciated; the revision should follow through by exercising those mechanisms. The dataset release is a significant community asset."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead CompSLAM. The genuinely new thing here is the public DARPA SubT Finals dataset—a challenging underground sequence with darkness, self-similar tunnels, IMU saturation, motion blur, and moving people—plus the first unified write-up of the full CompSLAM stack. That is a useful contribution on its own. The system itself is a sensible integration of ROVIO-style VTIO, LOAM-style LIO, factor-graph smoothing, and a priority-based fallback hierarchy, and the fact that it ran on all Team Cerberus robots in the winning SubT run is strong qualitative evidence. Code and dataset are public, and the paper is honest about limitations like the absence of dynamic-object filtering.\n\nThe soft spots are in the evaluation. It is a single teleoperated run, with no baselines, no ablations, and no error bars. The reference trajectory for RPE comes from offline ICP of the same LiDAR point clouds against the ground-truth map, which shares the system's modality; map errors are computed after a dense ICP alignment to the same ground truth, so the numbers are more about consistency than absolute accuracy. More importantly, the central resilience claim—that complementary modalities make the system robust because not all sensors degrade at once—is never actually stressed. The paper reports no segment where LiDAR degeneracy fires and leg odometry becomes the active source, no sensor-degradation injection, and no leg-odometry ablation. The checks on external poses catch gross jumps but not slow drift or slip-induced bias, which is exactly the failure mode of kinematic odometry on loose or slippery terrain. So the phrase \"highly resilient\" is a step ahead of what the evidence supports. That isn't a fatal flaw in the system—it's a gap between the claim and the test.\n\nThe paper is best read as a systems description with a valuable benchmark attached. If I worked on degraded-environment SLAM, I would want this dataset, and I would cite it. I also think a serious referee should engage with it: the method description is detailed, the deployment is legitimate, and the dataset release deserves scrutiny and uptake. My recommendation is to send it to peer review, with the expectation that the authors either soften the resilience claims or add a proper evaluation with baselines, degeneracy injections, and a leg-odometry ablation.","headline":"CompSLAM is a field-proven systems paper whose real value is the public SubT dataset; the resilience claims outrun a single-run evaluation that never stresses the leg-odometry fallback.","tokens_in":12594,"tokens_out":2904,"would_cite":true,"duration_ms":27733,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A chain of complementary pose estimators keeps robots localized in dark underground environments.","keywords":["multi-modal SLAM","underground robotics","sensor fusion","lidar-inertial odometry","visual-thermal-inertial odometry","legged robots","DARPA SubT finals dataset","perception degradation"],"falsifier":"Run CompSLAM in a controlled underground course engineered to degrade every perception channel simultaneously—unlit featureless walls, dense artificial dust, a geometrically self-similar tunnel, and rough terrain causing motion blur and IMU saturation—and compare the estimated trajectory against surveyed ground truth. If the pose error grows without bound or the map splits where the leg-odometry fallback drifts, the central resilience claim is falsified; a weaker version would blank out overlapping LiDAR and camera intervals in the released dataset and measure how long the reported 1.794% per 25 m accuracy is preserved.","tokens_in":11528,"feed_emoji":"🧭","tokens_out":9698,"duration_ms":90451,"temperature":0.7,"pith_summary":"This paper claims that an underground robot can be kept reliably localized and mapping by arranging several independent pose estimators in a hierarchy, so that each estimator uses the previous one's output and can be bypassed when its sensor inputs degrade. The authors present CompSLAM as that system and support the claim with its deployment on all Team Cerberus robots during the winning final run of the DARPA Subterranean Challenge and with a quantitative evaluation on a newly released 740-meter dataset collected by a quadruped in the same course. The load-bearing design premise is that environmental conditions rarely degrade every sensor modality at the same time, making redundancy the primary safety mechanism. If the claim is correct, the released code and dataset give the robotics community a reference implementation and a demanding benchmark for perception-degraded subterranean operation.","feed_headline":"CompSLAM kept every Team Cerberus robot on track underground","feed_subtitle":"The open-source system fuses vision, thermal, LiDAR, and leg odometry, and a new 740 m cave dataset lets others test it.","key_machinery":"The carrying mechanism is the fallback chain: a hierarchy of estimators in which each stage validates the quality of its input, produces a pose estimate, and passes that estimate to the next stage as a prior. The visual-thermal-inertial stage uses an iterated extended Kalman filter with visual and 16-bit thermal features and optional LiDAR-initialized inverse depth; the LiDAR-inertial stage performs feature-based scan-to-scan and scan-to-submap registration; and external kinematic estimates are inserted as prioritized priors. The degeneracy checks—image texture quality, D-optimality growth in the filter covariance, and eigenvalue thresholds on the registration Hessian—decide when a stage is bypassed, which is what converts redundant sensors into a graceful degradation chain.","core_discovery":"The paper's core claim is that resilience in underground SLAM comes from complementary redundancy rather than from a single tightly coupled estimator. CompSLAM links a visual-thermal-inertial odometry module, a LiDAR-inertial odometry module with scan-to-scan and scan-to-submap stages, and external kinematic estimates from the robot's legs into a coarse-to-fine chain, where each stage hands its pose estimate downstream as an initial guess and can be skipped if it detects degeneration. Degeneration is detected through image-quality gating, D-optimality covariance monitoring, and eigenvalue analysis of the registration Hessian. On the released DARPA SubT finals dataset, the paper reports an Average Wasserstein Distance of 0.221 m at 1 m voxel resolution, a Spatial Consistency Score of 0.491, and a relative translation error of 1.794% over 25 m trajectory segments, while the robot traversed 740 m in 35 minutes through self-similar tunnels, caves, and warehouse sections with missing LiDAR frames, motion blur, IMU saturation, and moving people present.","pith_inferences":["Beyond the paper, the ordered fallback design could be applied generally to autonomy stacks: rank independent estimators by cost and reliability, pass priors downstream, and let each stage validate its own input, rather than building one monolithic estimator.","Beyond the paper, because CompSLAM does not filter moving people, the released dataset can be used to measure how much dynamic-object filtering would improve map accuracy by comparing the published map against one rebuilt from static structures only.","Beyond the paper, the fact that the D-optimality reinitialization was never triggered during the finals suggests the quality thresholds are conservative; a systematic sweep of those thresholds on the released dataset could reveal earlier detection of sensor degradation."],"forward_implications":["If one sensor modality fails, the framework can continue delivering pose and map updates by using the previous valid estimate as a prior for the next stage, instead of stopping.","Because the modules are replaceable, learning-based components such as a learned scan-to-scan registration and a localizability classifier can be swapped into the LiDAR odometry stage without rebuilding the system.","The released dataset, with synchronized LiDAR, grayscale, RGB, IMU, and leg-odometry data spanning tunnels, caves, and warehouse areas, gives the community a common testbed for individual degradations including motion blur and dynamic people.","The same architecture has been extended for inter-robot map sharing, marsupial ground-air deployments, and collaborative multi-robot mapping, indicating the fallback chain is reusable beyond the competition course."],"supporting_citations":[{"why":"Gives the DARPA Subterranean Challenge context that defines the deployment and evaluation environment.","marker":"[1]"},{"why":"Documents the Team Cerberus sensor suite and the winning final run in which CompSLAM was deployed on all robots.","marker":"[12]"},{"why":"Supplies the iterated extended Kalman filter formulation and feature parametrization that CompSLAM's visual-thermal-inertial odometry adapts.","marker":"[16]"},{"why":"Provides the leg-kinematic-inertial estimator used as the external fallback pose source on the ANYmal robot.","marker":"[18]"},{"why":"Supplies the line-and-plane feature extraction and the scan-to-scan / scan-to-submap decomposition used by the LiDAR-inertial odometry module.","marker":"[22]"},{"why":"Provides the eigenvalue-based degeneracy test that CompSLAM uses to decide when a LiDAR registration is ill-conditioned and a fallback is needed.","marker":"[34]"},{"why":"Defines the ANYmal quadruped platform whose onboard sensors produced the released dataset and the DARPA runs.","marker":"[41]"},{"why":"Supplies the Average Wasserstein Distance and Spatial Consistency Score metrics used to compare the estimated map to ground truth.","marker":"[46]"}],"fun_headline_variants":["CompSLAM: complementary odometry fusion for underground resilience","Open-source CompSLAM wins DARPA SubT with redundant fusion","New 740m cave dataset stresses CompSLAM's multi-modal odometry","CompSLAM: redundant sensors for robust underground localization","Resilient underground SLAM from complementary vision, LiDAR, and leg odometry"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's resilience rests on the premise that environmental conditions will not degrade all sensor inputs at the same time; if darkness, dust, self-similar geometry, and motion blur arrive together, the final fallback is leg odometry, whose behavior under that combined stress is assumed rather than demonstrated by the evaluation.","fun_headline_variants_meta":{"raw":{"variants":["CompSLAM: complementary odometry fusion for underground resilience","Open-source CompSLAM wins DARPA SubT with redundant fusion","New 740m cave dataset stresses CompSLAM's multi-modal odometry","CompSLAM: redundant sensors for robust underground localization","Resilient underground SLAM from complementary vision, LiDAR, and leg odometry"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000733,"raw_usage":{"total_tokens":3321,"prompt_tokens":1032,"completion_tokens":2289,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":648,"completion_tokens_details":{"reasoning_tokens":2196}},"tokens_in":648,"tokens_out":2289,"duration_ms":16698,"temperature":1.0,"reasoning_tokens":2196,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:41:03.496044+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CompSLAM in a controlled underground course engineered to degrade every perception channel simultaneously—unlit featureless walls, dense artificial dust, a geometrically self-similar tunnel, and rough terrain causing motion blur and IMU saturation—and compare the estimated trajectory against surveyed ground truth. If the pose error grows without bound or the map splits where the leg-odometry fallback drifts, the central resilience claim is falsified; a weaker version would blank out overlapping LiDAR and camera intervals in the released dataset and measure how long the reported 1.794% per 25 m accuracy is preserved.","supporting_citations":[{"cited_title":"Into the robotic depths: Analy- sis and insights from the darpa subterranean challenge,","cited_arxiv_id":null,"evidence_quote":"Gives the DARPA Subterranean Challenge context that defines the deployment and evaluation environment."},{"cited_title":"Team cerberus wins the darpa subterranean challenge: Technical overview and lessons learned,","cited_arxiv_id":null,"evidence_quote":"Documents the Team Cerberus sensor suite and the winning final run in which CompSLAM was deployed on all robots."},{"cited_title":"Iterated extended kalman filter based visual-inertial odometry using direct photometric feedback,","cited_arxiv_id":null,"evidence_quote":"Supplies the iterated extended Kalman filter formulation and feature parametrization that CompSLAM's visual-thermal-inertial odometry adapts."},{"cited_title":"The two-state implicit filter recursive estimation for mobile robots,","cited_arxiv_id":null,"evidence_quote":"Provides the leg-kinematic-inertial estimator used as the external fallback pose source on the ANYmal robot."},{"cited_title":"Loam: Lidar odometry and mapping in real- time","cited_arxiv_id":null,"evidence_quote":"Supplies the line-and-plane feature extraction and the scan-to-scan / scan-to-submap decomposition used by the LiDAR-inertial odometry module."},{"cited_title":"On degeneracy of optimization-based state estimation problems,","cited_arxiv_id":null,"evidence_quote":"Provides the eigenvalue-based degeneracy test that CompSLAM uses to decide when a LiDAR registration is ill-conditioned and a fallback is needed."},{"cited_title":"Anymal - a highly mobile and dynamic quadrupedal robot,","cited_arxiv_id":null,"evidence_quote":"Defines the ANYmal quadruped platform whose onboard sensors produced the released dataset and the DARPA runs."},{"cited_title":"MapEval: Towards Unified, Robust and Efficient SLAM Map Evaluation Framework","cited_arxiv_id":"2411.17928","evidence_quote":"Supplies the Average Wasserstein Distance and Spatial Consistency Score metrics used to compare the estimated map to ground truth."}],"review_version":1}