{"id":"b7154777-505f-48f8-a2de-31f0bb6f242e","arxiv_id":"2607.12630","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"Instance-enriched 2.5D maps with LLM expert-fusion retrieval improve zero-shot object retrieval and navigation over HOV-SG while cutting map storage by ~96%.","lead":"This paper presents a compact 2.5D semantic map that stores multiple objects per grid cell, instance captions, and room context, and reports that it lets a simulated robot follow varied natural-language navigation commands better than 3D scene-graph baselines. The practical hook is a ~96% smaller map with more robust query handling for zero-shot visual-language navigation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline '2.5D outperforms 3D by 27%' is confounded: Ours uses SEEM, HOV-SG uses SAM+CLIP; the gain may stem from the segmentation backbone, not the representation.","rationale":"The reader's weakest_assumption is camera-pose drift, which is a legitimate applicability concern but is not the most load-bearing issue for the central claim as stated. The central claim is a quantitative superiority of the 2.5D representation over a 3D baseline, and every experiment uses simulator poses, so the pose assumption does not undermine the in-simulation comparison. The more serious problem is the confounding of representation with segmentation backbone: the headline 27% AUC improvement is computed against HOV-SG, which uses SAM+CLIP, while Ours uses SEEM. The paper even demonstrates that switching from LSeg to SEEM improves a 2D map substantially (Table 3), suggesting that much of the benefit attributed to '2.5D' may actually come from SEEM's superior instance segmentation. The matching-normalized AUC (Table 1) still favors Ours, but by a much smaller margin, and that metric does not control for backbone either. This is a correctable experimental design issue: running a SEEM-based HOV-SG or a SAM+CLIP-based version of Ours would settle whether the representation itself is the driver. The reader's verdict of CONDITIONAL is appropriate; my concern does not change the verdict direction but sharpens the condition: the authors should provide a backbone-matched comparison before claiming the 2.5D representation is the cause of the improvement. I agree partially with the reader because they did list the confound as a structural weakness in the rationale, but their explicit 'weakest_assumption' singled out pose drift, which I consider secondary to the claim's internal validity.","tokens_in":26163,"tokens_out":5785,"duration_ms":55381,"concrete_test":"Run the Table 1 mapping evaluation with HOV-SG rebuilt on SEEM masks/embeddings (same 3D scene graph, same thresholds, same normalization), or equivalently rebuild Ours with SAM+CLIP-derived instance features. If the prediction-normalized AUC gap over HOV-SG collapses below 27% or reverses, the representation-level claim is unsupported; if the gap persists, the 2.5D representation is vindicated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the proposed 2.5D representation outperforms the 3D scene-graph baseline by >27% average prediction-normalized AUC (Abstract; Sec. 4.1.1, Table 1) is not attributable to the representation because the two systems differ simultaneously in the perception backbone. Ours uses SEEM (Focal-L) for panoptic segmentation and instance embeddings; HOV-SG uses SAM+CLIP (ViT-H-14). The paper itself shows SEEM materially improves mapping over LSeg/CLIP (Table 3, VLMaps vs VLMaps+SEEM), and the large AUC_pred gap (0.356 vs 0.093 on Replica; 0.371 vs 0.085 on HM3DSEM) is driven by HOV-SG's over-segmentation (Instance Diff 86.6 vs 2.6; 417.6 vs 67.4). The only matched-backbone comparison, VLMaps+SEEM vs Ours (Table 3), is not the one cited in the headline. Thus the abstract/conclusion overstate what the experiments isolate. The navigation claim is also a full-system comparison, not an isolated test of MTEFR; the ablation Full+SPR vs Full+MTEFR is a cleaner isolation, but HOV-SG does not use MTEFR at all.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Instance-Enriched Semantic Maps (IESM), a zero-shot VLN framework that builds a compact 2.5D instance-level map from RGB-D observations using SEEM panoptic segmentation, enriches instances with room context and LLM-generated captions, and performs goal selection via a Multi-Type Expert Fusion Retrieval (MTEFR) module. The mapping, room segmentation, captioning, and retrieval components are evaluated on MP3D, Replica, and HM3DSEM against HOV-SG and VLMaps. The paper claims that the 2.5D representation outperforms a 3D scene-graph baseline by over 27% in prediction-normalized AUC, that MTEFR yields over 17% object-retrieval and 23% navigation-success improvements, and that the representation reduces storage by ~96%.","tokens_in":26534,"tokens_out":6020,"duration_ms":58086,"significance":"If the claims are substantiated, the paper would demonstrate a practical alternative to 3D scene graphs: a 10 MB instance-enriched map with vertical stacking support and LLM-based retrieval can outperform a 248 MB 3D map on mapping and navigation. The manuscript has notable strengths: hyperparameters and prompts are listed, the system is modular and clearly described, and the authors explicitly acknowledge failure modes such as over-segmentation (Fig. 5), open-plan room failures (Fig. 6), and low absolute success on abstract queries (Sec. 4.3.2). The matched-backbone comparison (VLMaps+SEEM vs. Ours in Table 3) and the Full+SPR ablation are the right controls, and the storage comparison is concrete. However, the headline comparisons are confounded across multiple components and lack statistical support, so the central attribution claims are not yet tightly established.","major_comments":[{"comment":"The headline claim that the 2.5D representation outperforms the 3D baseline by >27% in prediction-normalized AUC is confounded: 'Ours' uses SEEM (Focal-L) while HOV-SG uses SAM+CLIP (ViT-H-14). Table 3 shows that replacing LSeg with SEEM alone improves VLMaps on both MP3D and Replica (mIoU 0.198→0.226 and 0.175→0.200), so the perception backbone materially affects instance quality. The large AUC_pred gaps (0.356 vs 0.093; 0.371 vs 0.085) and Instance Diff gaps (2.6 vs 86.6; 67.4 vs 417.6) are consistent with HOV-SG's over-segmentation, not necessarily with the 2.5D representation. The matched-backbone comparison (VLMaps+SEEM vs Ours, Table 3) is not the comparison cited in the abstract/conclusion. To support the headline, please report HOV-SG with SEEM or otherwise isolate representation from perception backbone.","section":"Sec. 4.1.1, Table 1; Abstract"},{"comment":"The claimed 17%/23% navigation improvements are full-system comparisons against HOV-SG: the two systems differ in perception backbone, map representation, captioning, retrieval, and navigation. HOV-SG (w/ desc.) adds only captions and shows marginal gains, but it does not isolate MTEFR because HOV-SG's SAM+CLIP over-segmentation may prevent caption assignment at the same object granularity. The internal ablation Full+MTEFR vs Full+SPR is the cleaner isolation, yet the gains are uneven: on (o) O-SR improves 70 vs 58, on (a,r,d) 14 vs 6, while conditional N-SR is lower for MTEFR on (o,d), (o,r,d), and (a,r,d). With 250 trials per query type, no confidence intervals or significance tests are reported. Please provide per-query-type errors/significance and state explicitly that the headline is a full-system comparison.","section":"Sec. 4.3.2, Table 5"},{"comment":"Room-level semantic segmentation is evaluated on only two MP3D scenes, with no per-scene breakdown or variance. The claimed improvements (mAcc 0.72→0.86, mIoU 0.53→0.75) rest on very thin evidence, especially since the paper itself identifies a failure on open-plan scenes (Fig. 6b). Please evaluate on the same scene sets as the mapping experiments (at least the full Replica/MP3D sets) and report per-scene results with standard errors, or substantially temper the claim.","section":"Sec. 4.2, Table 4"},{"comment":"The pipeline assumes accurate and globally consistent camera poses (Sec. 3.1), and all experiments use simulator poses. The conclusion asserts that the secondary fusion pass 'already tolerates moderate pose drift,' but no experiment with realistic SLAM noise is provided. Given the association thresholds δgeo=0.4 and δsem=0.85 (Sec. 4), pose drift beyond these thresholds would fragment or merge instances. If the robustness claim is retained, it must be tested; otherwise the statement should be removed or clearly marked as untested speculation.","section":"Sec. 3.1.3 and Sec. 5"},{"comment":"The expert taxonomy (obj, room, desc, abs) exactly matches the query types used to construct the test set (object, room, caption, mixed, abstract). This alignment is favorable to MTEFR by design and may overstate its advantage on 'diverse' query formulations. It is not circular in the strict sense because the baselines see the same queries, but it should be acknowledged as a favorable protocol, and ideally supplemented with queries that do not fit neatly into a single expert type.","section":"Sec. 3.4 and Sec. B"}],"minor_comments":[{"comment":"The 'over 27%' improvement is actually an average absolute difference in percentage points (0.356−0.093=0.263; 0.371−0.085=0.286). Please reword to '27 percentage points' or report relative improvement to avoid misinterpretation.","section":"Abstract/Conclusion"},{"comment":"'topend' is not defined. Specify whether it denotes the final point before saturation or the full ranking endpoint.","section":"Tables 1/2"},{"comment":"The definition of Instance Diff is given as 'absolute difference between ground-truth and predicted object counts,' but it is unclear whether this is per scene, per category, or aggregated. Please clarify and report per-scene values.","section":"Sec. 4.1.1"},{"comment":"Representative keyframe selection is described as based on 'confidence of the predicted masks and frequency of observations,' but no algorithm or criterion is given. Please include the selection procedure for reproducibility.","section":"Sec. 3.3"},{"comment":"The qualitative statement that the red-boxed 'table' is 'indeed a kitchen countertop' is a useful open-vocabulary discussion, but it also highlights that the ground-truth label and prediction can be both valid. Consider adding a quantitative analysis of such label ambiguity.","section":"Sec. 4.1.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems contribution with transparent failure analysis, but the central claims are not yet supported by controlled comparisons. The main fixes needed are: (1) a matched-backbone HOV-SG comparison or an explicitly restricted claim, (2) error bars/significance tests for the navigation and mapping tables, (3) more room-segmentation scenes, and (4) either testing or removing the pose-drift robustness claim. These are within the scope of a major revision rather than a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is a well-executed systems integration with a genuinely useful representational contribution—the 2.5D grid cell that keeps multiple instance indices per cell, preserving vertical stacking without full 3D storage—and the core approach is sound. The MTEFR gate/expert fusion for query-adaptive retrieval is a plausible addition, and the ~10MB map footprint is a strong practical result. But the abstract and conclusion present the comparison as “2.5D vs 3D” when the experiments actually compare two full pipelines that differ in the segmentation backbone; the 27% prediction-normalized AUC gap is attributable to several simultaneous changes, not just the representation.\n\nWhat the paper does well: it situates itself clearly against HOV-SG and VLMaps, and the ablations are genuinely informative. Feeding HOV-SG the same instance captions without MTEFR barely improves its retrieval, which supports the claim that the fusion mechanism matters. The small-object preservation and the secondary fusion pass are reasonable engineering choices, and the qualitative figures show the system working on challenging abstract queries. The storage comparison is clean: 10MB vs 248MB for HOV-SG, with only a modest loss in representational fidelity.\n\nWhere the soft spots are: the central mapping claim is confounded. Ours uses SEEM (Focal-L); HOV-SG uses SAM+CLIP (ViT-H-14). The paper itself shows SEEM materially improves mapping over LSeg/CLIP in Table 3, so the large AUC_pred gap likely comes substantially from the perception backbone. The matched-backbone comparison—VLMaps+SEEM vs Ours in Table 3—is the right experiment, and it does show the 2.5D representation helps, but that is not the number promoted in the abstract. Room segmentation is evaluated on only two MP3D scenes. There are no error bars or significance tests anywhere. The MTEFR query taxonomy aligns closely with the test query types, so the evaluation protocol is somewhat circular, though the failure-case analysis partially mitigates this. Camera pose robustness is assumed rather than tested; the secondary fusion pass is never stress-tested with realistic SLAM noise, and the paper’s own over-segmentation examples in Fig. 5 show the failure mode. None of this invalidates the work—it is a fair conditional accept with revisions, not a reject.\n\nWho this is for: anyone working on zero-shot VLN or semantic mapping who wants a compact instance-level representation with language-grounded retrieval. It deserves a serious referee; I would take the review. I’d also bring it to a reading group, mainly to discuss how much of the observed gain is representation versus backbone.","headline":"A serious systems paper with a real representational idea—the 2.5D multi-occupancy instance map—but the headline numbers compare full pipelines, not isolated representations, so they overstate what is shown.","tokens_in":27028,"tokens_out":2341,"would_cite":true,"duration_ms":24712,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A compact 2.5D map with instance captions and expert-fusion retrieval can match or beat heavy 3D scene graphs on language-guided navigation.","keywords":["visual language navigation","2.5D semantic mapping","instance-level mapping","open-vocabulary segmentation","large language models","zero-shot navigation","storage-efficient scene representation","expert fusion retrieval"],"falsifier":"Run the same mapping and retrieval pipeline on the same scenes but inject increasingly large pose drift (for instance, 0.1–0.5 m errors) into the camera poses before fusion, and track prediction-normalized AUC and instance-count gap; a sharp collapse at modest drift would show the compact 2.5D map's advantage depends on perfect localization.","tokens_in":26064,"feed_emoji":"🧭","tokens_out":4479,"duration_ms":45912,"temperature":0.7,"pith_summary":"This paper sets out to show that an embodied robot can follow natural-language navigation queries using a compact 2.5D semantic map that keeps individual object instances, their vertical stacking, and rich text captions, instead of a heavy 3D reconstruction. It claims its instance-enriched map improves object-level mapping accuracy by over 27% in prediction-normalized AUC over a 3D scene-graph baseline, while cutting storage by about 96% (roughly 10 MB versus 248 MB). For navigation, it reports over 17% higher object retrieval and 23% higher navigation success across diverse query formulations. The central bet is that storing instance captions plus a query-adaptive expert-fusion retrieval module matters more than geometric fidelity for grounding language in indoor spaces.","feed_headline":"A 10 MB map beats a 248 MB 3D scene graph for robot navigation","feed_subtitle":"Instance captions plus expert-fusion retrieval lift object retrieval by 17% and navigation success by 23%.","key_machinery":"The central object is the 2.5D instance-enriched map: a top-down grid where each cell can list several vertically stacked instance indices, paired with an instance dictionary where each entry is an open-vocabulary embedding, room label, structured tags, and a compact caption. The query-side mechanism is MTEFR: a gate language model assigns weights to four expert rankers (object, room, description, abstract), each produces a top-κ list, and score fusion combines a weighted sum with a peak score to pick the goal instance. The map does the storage work; MTEFR does the interpretation work.","core_discovery":"The paper argues that the limiting factor in language-guided navigation is not geometric fidelity but the richness of per-instance semantics and the way queries are matched to them. It builds a 2.5D map in which each grid cell can hold multiple vertically stacked instance IDs, so objects above one another are not blended. Each instance stores an open-vocabulary feature, a room label, structured attributes such as color and material, and a compressed natural-language caption distilled from multiple views. On top of that, the Multi-Type Expert Fusion Retrieval (MTEFR) module routes each user query through a gate and four type-specialized experts—object category, room context, description, and","pith_inferences":["A direct test of the paper's core bet would be to replace simulator poses with noisy SLAM trajectories and re-measure the same AUC and success metrics; if fragmentation appears at drift below the stated association thresholds, the 10 MB claim may only hold under perfect localization.","Since captions are the main semantic carrier, the approach invites a fully asynchronous or distilled captioning pipeline; caption latency, not map size, would likely bound real-time use.","The abstract-query success rate (about 14%) suggests the next bottleneck is functional knowledge rather than perception; a hierarchy of affordances or functional categories could tighten the abstract expert's ranking.","The single-floor evaluation leaves multi-floor maps untested; extending the 2.5D representation to floors connected by stairs or elevators is a natural, concrete next experiment."],"forward_implications":["A robot can carry a semantically useful map of an indoor floor in about 10 MB, making long-term and embedded deployment feasible without losing instance-level grounding.","Abstract queries that name no category, such as 'the most comfortable object to rest on for three people,' become addressable because captions and room context are searchable, not just labels.","Vertical stacking is resolved without a voxel or mesh reconstruction, so multi-shelf and cluttered scenes avoid semantic blending in the grid.","Providing richer captions alone does not close the gap; the query-adaptive fusion of expert rankings is what delivers the reported gains.","Room segmentation from structural boundaries works best in partitioned spaces; open-plan areas remain a recognized failure point that would need a different boundary source."],"fun_headline_variants":["Instance-enriched 2.5D maps outperform 3D scene graphs in VLN by 27%","Slash map storage 96% and boost robot navigation 23% with instance-aware 2.5D","Language-guided robots navigate better with instance-level semantic maps","Richer instance detail, not geometry, drives VLN success"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole map rests on the assumption of accurate, globally consistent camera poses, and every experiment gets those poses from the simulator rather than from a real SLAM estimate.","fun_headline_variants_meta":{"raw":{"variants":["Instance-enriched 2.5D maps outperform 3D scene graphs in VLN by 27%","Slash map storage 96% and boost robot navigation 23% with instance-aware 2.5D","Language-guided robots navigate better with instance-level semantic maps","Richer instance detail, not geometry, drives VLN success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000847,"raw_usage":{"total_tokens":3556,"prompt_tokens":809,"completion_tokens":2747,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":2658}},"tokens_in":553,"tokens_out":2747,"duration_ms":19302,"temperature":1.0,"reasoning_tokens":2658,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T06:22:29.702071+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same mapping and retrieval pipeline on the same scenes but inject increasingly large pose drift (for instance, 0.1–0.5 m errors) into the camera poses before fusion, and track prediction-normalized AUC and instance-count gap; a sharp collapse at modest drift would show the compact 2.5D map's advantage depends on perfect localization.","supporting_citations":[],"review_version":2}