{"id":"f4329350-c057-493e-bbd9-bebee6a95aef","arxiv_id":"2501.14147","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A multi-robot system aligns heterogeneous camera streams into a common frame and continually trains a semantic 3D Gaussian Splatting map for language-guided navigation.","lead":"HAMMER is a server-based system that merges image and depth streams from multiple robots and wearable devices into one shared 3D map using Gaussian Splatting, without knowing where the robots started. It also embeds language semantics into the map so robots can navigate to goals like 'the couch'.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is gated by one-time 32-image SfM alignment, yet no alignment success/failure or residual statistics are reported; the real-world fidelity results are therefore conditional on an unmeasured alignment success rate.","rationale":"The paper is well written, the system architecture is credible, and the one-time-alignment-plus-online-3DGS idea is a reasonable design. The most load-bearing part of the argument is not the 3DGS training itself—which is standard online 3DGS with pose refinement—but the claim that a single 32-image localized SfM alignment, computed once per device, is sufficient to fuse arbitrary heterogeneous streams with no prior relative pose. The reader's weakest-assumption analysis identifies the same module, but I want to sharpen it: the paper provides no empirical distribution over alignment outcomes. It describes rejection thresholds, but not how often alignments are rejected, how many SfM attempts are needed, or what residual error remains in accepted transforms. Without that, the hardware results are only conditional on an alignment having succeeded, and the abstract's phrasing overstates the robustness of the method. The self-admitted limitations about drift and short deployments reinforce that the alignment module is the fragile core rather than a solved component. My proposed test is a repeated-trial experiment with randomized join times and a deliberately non-overlapping start; it would directly measure whether the one-time alignment assumption holds in the cases that matter. No change to the reader's CONDITIONAL verdict is needed; the concern is consistent with it and justifies adding the missing evidence rather than rejecting the system outright.","tokens_in":12156,"tokens_out":5987,"duration_ms":62850,"concrete_test":"Run the two hardware ROS bags through HAMMER at least 10 times with randomized device join order/time offsets, and separately with the second device's first 30 s in a non-overlapping region, matching the failure regime named in the limitations. Report, per trial: number of SfM attempts, accepted/rejected alignments, Eq. 1 residual rotation/translation, and final map PSNR and pose ATE against the Oracle COLMAP poses. Also sweep W in {8,16,32} on Replica Off-0. If the acceptance rate is not high (e.g., at least 90%) or accepted ATE is not well below the 0.1 m/10 deg rejection thresholds, the one-time alignment premise fails and the comparative claims must be re-scoped.","verdict_should_be":"UNCHANGED","load_bearing_attack":"HAMMER's headline claim—that heterogeneous, asynchronous devices can be fused with no prior relative poses—rests entirely on Sec. III-B's one-time alignment. For every non-origin device, a single NETVLAD/LightGlue correspondence selects one temporal window of W=16 images per robot; localized SfM on those 32 images must reconstruct all poses, and two absolute-orientation solves (Eq. 1) freeze a similarity transform that is used for the rest of the deployment. Three unstated conditions must hold: the window has enough parallax and overlap for SfM to succeed; the local SLAM poses inside that window are drift-free enough to define the target frame; and all later SLAM drift is small enough that the regularized per-frame SE(3) offsets of Sec. III-C.3 can absorb it. The paper reports rejection criteria (0.1 m, 10 deg) but never reports how many SfM attempts occurred, what fraction of alignments were rejected, what the residuals of accepted alignments were, or how alignment error propagates to map ATE. The Hardware Trials (Sec. IV-B.2) then compare only the successful alignments. If the acceptance rate is low or accepted transforms carry errors near the rejection thresholds, the '2x better' claim and the 'no prior relative pose' claim are not established for the general ad-hoc deployment scenario. The conclusion even concedes that drift and very short deployments can break the method, which is precisely the regime the alignment module must handle.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"HAMMER proposes a server-based, online multi-robot 3D Gaussian Splatting mapping system that fuses asynchronous, heterogeneous robot and edge-device data streams (e.g., ground robots and Aria glasses) without prior knowledge of relative poses. The method has two main components: (i) a one-time inter-robot frame alignment module that uses NETVLAD/SuperPoint/LightGlue correspondences, localized SfM on a 32-image window, and two absolute-orientation solves to estimate a similarity transform from each robot's local SLAM frame to the global map frame; and (ii) an online semantic 3DGS training module that continuously optimizes a map from the aligned streams, with appearance compensation for different ISP pipelines, depth supervision, and CLIP semantic embedding distillation for open-vocabulary queries. The paper reports evaluations on the ReplicaMultiAgent dataset against CP-SLAM and MAGiC-SLAM, and on two real-world hardware scenes against several baselines, including a modified Di-NeRF baseline and an Oracle upper bound. The authors claim HAMMER is the first online multi-robot 3DGS pipeline for heterogeneous teams and that it produces higher-fidelity maps than competing baselines while requiring far less computation.","tokens_in":1960,"tokens_out":1871,"duration_ms":94445,"significance":"If the central claim is supported, HAMMER would be a practically valuable contribution: it offers a single-GPU server architecture that works with off-the-shelf robot pose estimators and ROS infrastructure, which is a meaningful step toward deployable collaborative semantic mapping. The paper's strengths include a well-motivated system design, a genuinely one-time and black-box alignment procedure, real hardware experiments with heterogeneous devices, and an explicit semantic field that enables language-guided downstream tasks. The authors also provide useful limitation statements, notably that devices without pose estimates, significant SLAM drift, and very short deployments are not handled. However, the quantitative evidence for the headline claims has important gaps: the central alignment module is not evaluated with any success/failure statistics, the main benchmark table reports training-view rather than held-out novel-view quality, and the real-world baselines are substantially modified or partially self-referential. These gaps are load-bearing, because the paper's principal contributions are the alignment method and the claimed fidelity advantage over existing baselines.","major_comments":[{"comment":"The one-time alignment module is the load-bearing component for the 'no prior relative pose' claim, yet the paper reports no statistics on how often alignment succeeded, how many SfM attempts were made, how many were rejected by the 0.1 m / 10 deg thresholds, or what the residuals of the accepted Eq. (1) solves were. Without this information, the hardware results in Sec. IV-B.2 are conditional on an unmeasured alignment success rate, and the conclusion's own admission that drift and very short deployments 'can break the method' highlights the regime in which the alignment must work. I ask the authors to report alignment outcome distributions, residuals, and a sensitivity analysis over the window size W, the match-ratio xi, and the rejection thresholds, including at least one case where alignment fails or is marginal.","section":"III-B.3, III-B.4, Sec. V"},{"comment":"The caption of Table I states that the reported numbers are 'training view synthesis' metrics, not held-out novel-view synthesis. Training-view PSNR/SSIM/LPIPS can be inflated by overfitting, so the claim that HAMMER 'outperforms both baselines' on reconstruction quality is not established by this table. Additionally, the CP-SLAM metrics are copied from [13] and the runtimes are estimated rather than measured under the same protocol, making the comparison uncontrolled. Please either evaluate all methods on a common held-out view set or clearly state that Table I is an approximate cross-paper comparison rather than a controlled benchmark.","section":"Table I, Sec. IV-B.1"},{"comment":"The Di-NeRF* baseline is so heavily modified (centralized optimization, training on cached HAMMER data, and replacement of the NeRF with 3DGS) that the comparison does not evaluate the actual Di-NeRF method; it evaluates a custom centralized 3DGS variant with gradient-based alignment. This weakens the claim that HAMMER 'dramatically outperforms Di-NeRF*' as a statement about the published baseline. Relatedly, the Oracle baseline is trained on the full dataset with SfM poses derived from the same sensor data, so it is not an independent ground truth; it is an upper bound only relative to the SfM pose pipeline. The authors should clarify what each baseline is intended to show and, if possible, include an unmodified Di-NeRF or a centralized 3DGS without HAMMER's pose refinement to isolate the contributions.","section":"IV-A.3, Fig. 5"},{"comment":"The semantic contribution is evaluated only through a single qualitative case study, with no quantitative measure of language-guided goal localization accuracy, and the contribution of the online pose refinement in Sec. III-C.3 is not ablated. Since the abstract and introduction emphasize both the semantic capability and the continual online training, I would expect at least a small ablation (e.g., with and without pose refinement, with and without ISP compensation) and a quantitative semantic localization metric (e.g., success rate or distance error for language-specified goals).","section":"Sec. IV-B.4, Sec. III-C.3"}],"minor_comments":[{"comment":"The abstract claims '2x' higher-fidelity maps while the introduction says 'more than 40% better Mean-Squared-Error'; these are inconsistent and the '2x' is not defined anywhere in the paper. Please state the metric and the exact comparison.","section":"Abstract, Introduction"},{"comment":"The value of the rotation regularization weight epsilon in Eq. (1) is never specified, and the statement 'for small epsilon, an approximately optimal solution can be computed' is vague. Please report the chosen value and the number of iterations or the stopping criterion for the alternating optimization.","section":"III-B.4"},{"comment":"There is a typo: 'streaming' appears as 'steaming' in the text, and the sampling-weighting scheme for data tuples is described only qualitatively. Please clarify how the sampling probability is computed.","section":"III-C.2"},{"comment":"The paper states that neither public implementation of CP-SLAM or MAGiC-SLAM is real-time capable, but MAGiC-SLAM is described elsewhere as requiring N+1 GPUs; this should be clarified so that the runtime comparison in Table I is not misinterpreted as an unfair hardware handicap.","section":"IV-A.3"},{"comment":"The figure plots 'Map quality over time' but the axes and the exact metric (PSNR on held-out frames? per-device averages?) are not clearly labeled in the text. Please add explicit axis labels and describe how the curves were computed.","section":"Fig. 5"},{"comment":"The held-out evaluation set consists of 10 frames per device, but the paper does not report standard deviations or significance tests over these small samples. Given the small number of frames, confidence intervals would help assess whether the reported differences are meaningful.","section":"IV-B.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong systems contribution with a plausible central idea, but the evaluation currently overstates the evidence for the main claims. The alignment success-rate gap and the training-view/copied-baseline issues are fixable with additional experiments or with more careful claims, so rejection is not warranted. I would also encourage the authors to release code at the project page, as the paper does not currently include a code repository link, which would materially improve reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about HAMMER. First, it is a real, working multi-robot Gaussian splatting system: a single GPU server aligns streams from heterogeneous devices (ground robots with ZED cameras, Aria glasses) with no prior relative pose, and keeps a semantic 3DGS map updated online. The hardware trials in two indoor scenes show the architecture actually runs, and the qualitative results are convincing. Second, the evaluation is not as strong as the abstract implies. The headline comparisons against CP-SLAM and MAGiC-SLAM rest on numbers taken from another paper and training-view metrics, and the Di-NeRF baseline is modified into something quite different. The system likely works; the quantitative evidence for \"2x better\" is thinner than it looks.\n\nWhat is actually new: the specific integration of NetVLAD, SuperPoint/LightGlue, localized SfM, and two absolute-orientation solves to align a new device once, then continually train a metric-semantic 3DGS map with CLIP codes. The bilateral-grid ISP compensation for mixed cameras is a nice touch. The paper is clearly written, and the limitations section is honest: it admits that significant drift or very short deployments break the method. That candor is to their credit.\n\nThe soft spots, in order of importance. (1) The alignment module is the load-bearing component, and the paper reports no statistics on how often it succeeds, how many candidates were rejected, or the residuals of accepted transforms. The stress-test note is right: if the success rate is low or accepted errors are near the 0.1m/10deg thresholds, the \"ad-hoc deployment\" claim is unproven. This is not just a minor omission, but it does not invalidate the demonstration on the two scenes where it did succeed. (2) Table I is training-view synthesis, not held-out novel views. That overstates map quality. (3) Baselines are handled unevenly: CP-SLAM numbers are copied, Di-NeRF is modified into Di-NeRF*, and Oracle is trained on the full dataset. These are all defensible choices, but they should be labeled clearly as reimplementations or upper bounds. (4) No repeated trials, so no error bars on the hardware results.\n\nWho is this for: robotics researchers working on multi-robot mapping or online 3DGS, and anyone building semantic maps with wearable devices. It deserves a serious referee. The system is engineered well and the core idea is credible; the evaluation gaps are fixable with more transparency and a few more experiments. I would not desk-reject it.\n\nRecommendation: send it to review, ask for a revised version that reports alignment success rates and residuals, runs the baselines directly or labels them clearly as proxies, and adds held-out novel-view metrics. With those changes, it would be a solid systems contribution.","headline":"A well-engineered multi-robot 3DGS system whose core alignment statistics are under-reported; the evaluation gaps are real but fixable, and the paper deserves review.","tokens_in":13014,"tokens_out":3809,"would_cite":true,"duration_ms":32834,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T40","68T45"],"pacs":[],"model":"deepseek-v4-flash","headline":"HAMMER shows a single server can fuse asynchronous heterogeneous robot streams into one live metric-semantic 3D Gaussian map, without any prior relative poses.","keywords":["multi-robot mapping","3D Gaussian splatting","metric-semantic mapping","frame alignment","online mapping","heterogeneous robots","CLIP embeddings","language-guided navigation"],"falsifier":"Deploy two robots in a scene with deliberately disjoint coverage so the only overlap is a single narrow doorway, or inject growing drift into one robot's SLAM poses; if the one-time alignment is rejected or the final map's PSNR drops sharply compared to an Oracle bulk-SfM map, the claimed independence from coverage and drift would be falsified.","tokens_in":11898,"feed_emoji":"🗺️","tokens_out":6942,"duration_ms":56314,"temperature":0.7,"pith_summary":"HAMMER proposes a server-based pipeline in which a team of robots and wearable devices, each running its own onboard SLAM in its own local coordinate frame, streams posed color and depth data over ROS to a single GPU server. The server performs a one-time alignment of each new device's frame to a global frame using image correspondences and localized structure-from-motion, with no prior knowledge of relative poses. It then continuously trains a metric-semantic 3D Gaussian Splatting map from the fused streams, distilling CLIP embeddings into a feature field that supports open-vocabulary queries. The paper reports that this yields higher-fidelity maps than competing multi-robot mapping baselines in simulation, and better novel-view reconstruction than a Di-NeRF-based baseline in hardware trials with Aria glasses and ground robots, at a fraction of the compute time.","feed_headline":"One server fuses a robot team's streams into one live 3D map","feed_subtitle":"Each device's own SLAM is aligned once in ~36 seconds, then a live semantic map answers text queries.","key_machinery":"The load-bearing component is the one-time inter-robot frame alignment module. It builds on a rotation-aware absolute orientation problem, Eq. (1), which for a window of $W=16$ poses per robot jointly optimizes scale, rotation, and translation between local SLAM poses and localized SfM poses with a small rotation regularization term; composing two such solutions maps the new device's frame $T_j$ into the global frame $T_g$. Because the SfM step ignores onboard poses entirely, the method treats each robot's localization as a black box and only requires one robot to provide metric-scale poses.","core_discovery":"The paper's central claim is that heterogeneous multi-robot mapping can be decoupled into a cheap one-time per-robot alignment step plus an online, continually trained 3D Gaussian Splatting map, so that no robot needs a GPU and no prior inter-robot pose knowledge is required. HAMMER finds candidate correspondences between a new device and already-aligned devices with NetVLAD and SuperPoint/LightGlue, runs localized SfM on a 32-image window to get an independent relative pose estimate, and solves a rotation-aware absolute orientation problem (its Eq. (1)) to compose a metric similarity transform from the new robot's local frame into the global frame. After alignment, all streams are fused into a 3DGS representation with online pose refinement, bilateral-grid ISP compensation to handle differing camera pipelines, and a hash-grid feature field supervised by CLIP embeddings for language semantics. Against CP-SLAM and MAGiC-SLAM on ReplicaMultiAgent it reports higher average PSNR/SSIM and lower LPIPS at 8 minutes versus 133+ minutes, and in real-world trials it outperforms a centralized Di-NeRF* baseline while approaching an Oracle bulk-SfM upper bound.","pith_inferences":["Beyond the paper, if the 32-image localized SfM window is the bottleneck, then scenes with low texture or small overlaps will need larger windows or periodic re-alignment; this scaling condition is not explored in the paper.","A further implication is that because only the origin robot must be metric, teams mixing AR glasses, drones, and ground robots can all contribute to one map, which could enable wearable-plus-robot search-and-rescue or home assistance without dedicated infrastructure.","One testable extension is whether the same ISP compensation that handles device-to-device appearance differences also handles large lighting changes over time within a single device, since the paper's scenes include such variation but do not isolate it.","The paper names RGB-only devices as future work; an immediate test would be replacing depth input with monocular metric depth estimation and comparing alignment and map quality against the RGB-D results."],"forward_implications":["Any device that can stream posed RGB-D data or point clouds can join an existing map after a single roughly 36-second alignment, regardless of its camera or SLAM software.","A single server GPU can sustain 26-66 Hz training updates over the fused stream, making the map usable during deployment rather than after offline batch optimization.","Semantic queries such as \"go to the couch\" run in about 20 ms by matching CLIP embeddings against the feature field, and Splat-Nav plans collision-free trajectories in about 0.5 s for the whole team.","On ReplicaMultiAgent, the reported reconstruction quality is higher than CP-SLAM and MAGiC-SLAM while using under a tenth of the computation time."],"supporting_citations":[{"why":"Supplies the 3D Gaussian Splatting representation and differentiable rasterization that HAMMER uses for the online map.","marker":"[3]"},{"why":"COLMAP backend used for localized SfM, giving independent relative pose estimates between robots.","marker":"[18]"},{"why":"NetVLAD place recognition features that propose candidate image correspondences between aligned and unaligned streams.","marker":"[25]"},{"why":"LightGlue feature matching used to verify candidate correspondences before SfM.","marker":"[27]"},{"why":"MAGiC-SLAM, the concurrent multi-robot 3DGS SLAM baseline HAMMER compares against in Replica experiments.","marker":"[13]"},{"why":"CP-SLAM, the collaborative point-based SLAM baseline HAMMER outperforms on ReplicaMultiAgent.","marker":"[14]"},{"why":"Di-NeRF, the collaborative NeRF baseline with gradient-based alignment that HAMMER is compared against in hardware trials as Di-NeRF*.","marker":"[16]"},{"why":"Bilateral guided radiance field processing whose ISP compensation scheme HAMMER adapts to suppress floaters from differing cameras.","marker":"[35]"},{"why":"CLIP, the vision-language model whose per-pixel embeddings are distilled into HAMMER's semantic feature field.","marker":"[31]"},{"why":"Splat-Nav, used in the case study to plan collision-free trajectories to language-specified goals in the semantic map.","marker":"[7]"}],"fun_headline_variants":["No GPS, no problem: robot team builds one 3D map","One server, many robots, one live semantic 3D map","Robot swarm maps without pose sharing, on a server","HAMMER: async robot streams, one aligned 3D map","Server-built 3D map from robots with no prior poses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"HAMMER's central assumption is that each new device's path has enough visual overlap with already-aligned devices that a 32-image window yields a correct similarity transform, and that onboard SLAM drift stays small enough for the online pose refinement to correct; a robot that explores brand-new territory or drifts heavily can fail to align or can bias the global map.","fun_headline_variants_meta":{"raw":{"variants":["No GPS, no problem: robot team builds one 3D map","One server, many robots, one live semantic 3D map","Robot swarm maps without pose sharing, on a server","HAMMER: async robot streams, one aligned 3D map","Server-built 3D map from robots with no prior poses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1385,"prompt_tokens":1022,"completion_tokens":363,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":638,"completion_tokens_details":{"reasoning_tokens":275}},"tokens_in":638,"tokens_out":363,"duration_ms":3921,"temperature":1.0,"reasoning_tokens":275,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:20:39.600770+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Deploy two robots in a scene with deliberately disjoint coverage so the only overlap is a single narrow doorway, or inject growing drift into one robot's SLAM poses; if the one-time alignment is rejected or the final map's PSNR drops sharply compared to an Oracle bulk-SfM map, the claimed independence from coverage and drift would be falsified.","supporting_citations":[{"cited_title":"Structure-from-motion revisited,","cited_arxiv_id":null,"evidence_quote":"COLMAP backend used for localized SfM, giving independent relative pose estimates between robots."},{"cited_title":"Netvlad: Cnn architecture for weakly supervised place recognition,","cited_arxiv_id":null,"evidence_quote":"NetVLAD place recognition features that propose candidate image correspondences between aligned and unaligned streams."},{"cited_title":"Lightglue: Local feature matching at light speed,","cited_arxiv_id":null,"evidence_quote":"LightGlue feature matching used to verify candidate correspondences before SfM."},{"cited_title":"Cp-slam: Collabora- tive neural point-based slam system,","cited_arxiv_id":null,"evidence_quote":"CP-SLAM, the collaborative point-based SLAM baseline HAMMER outperforms on ReplicaMultiAgent."},{"cited_title":"Di-nerf: Distributed nerf for collaborative learning with relative pose refinement,","cited_arxiv_id":null,"evidence_quote":"Di-NeRF, the collaborative NeRF baseline with gradient-based alignment that HAMMER is compared against in hardware trials as Di-NeRF*."},{"cited_title":"Bilateral guided radiance field processing,","cited_arxiv_id":null,"evidence_quote":"Bilateral guided radiance field processing whose ISP compensation scheme HAMMER adapts to suppress floaters from differing cameras."}],"review_version":1}