{"id":"4ccb7615-0a13-44a1-a39a-698a0f025555","arxiv_id":"2509.00210","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"VEME, a dual-memory cross-modal alignment framework built on Qwen-2.5-VL, reports modest gains on VLN-CE and VSI-Bench that are contradicted by its own internal numbers.","lead":"VEME is a new AI system that adds a learnable 'world memory' and trajectory memory to a vision-language model to help robots navigate rooms and answer spatial questions. The paper claims small accuracy gains, but its tables, abstract, and statistics contradict each other, so the result is not supported as written.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main results are internally inconsistent: full-model SPL is 51.0 in Table 1 but 65.1 in Table 3, and reported Wilcoxon p-values are impossible for n=5; the central improvement claim is therefore not reliably supported.","rationale":"The strongest claim is an empirical one: VEME improves over existing methods on VLN-CE and VSI-Bench. The condition that would have to hold for this claim is that the reported numbers are internally consistent and reproducible. That condition is the least secure. The same full model is reported as SPL 51.0 in Table 1 and 65.1 in Table 3; the text around Table 1 contradicts the table's baseline numbers and even its own VEME SPL; Table 2's Avg cannot be recovered from the listed scores; and the ablation significance tests are impossible for n=5. None of these require a particular theoretical commitment to evaluate; they are mechanical checks that any reader can run. They undercut the exact evidence on which the paper's central claim rests. The reader's static-environment objection is real and appears in Appendix E, but it is a scope limitation, not a flaw in the reported numbers. The numerical inconsistencies are more load-bearing because if they stand, the headline results are undefined. I still read the paper in good faith: the architecture may have merit, and releasing the checkpoint and evaluation scripts could resolve the discrepancies. But as written, the central claim is not supported. Verdict remains REJECT; this stress-test does not change the reader's verdict.","tokens_in":17970,"tokens_out":8998,"duration_ms":101163,"concrete_test":"Run the official VLN-CE R2R Val-Unseen evaluation on the released VEME checkpoint and compare the resulting SPL to Table 1's 51.0 and Table 3's 65.1. If the measured SPL matches neither, or if no checkpoint/code is released for this check, the central navigation result is not reproducible and the inconsistency stands. As a secondary arithmetic check, recompute Table 2's Avg as the unweighted mean of the eight listed category scores; if it does not equal 49.3 (and Spatial-MLLM's does not equal 48.4), the VSI-Bench superiority claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that VEME yields measurable, sometimes considerable, gains on VLN-CE and VSI-Bench—depends on internal consistency of the reported tables. That condition fails. Table 1 reports VEME R2R Val-Unseen SPL 51.0, while the main text states 46.7 and describes ETPNav as 42.0/36.5; Table 1 itself lists ETPNav as 57.0/49.0, so VEME only ties SR and the textual comparison is wrong. Table 3 gives the same Full Model on VLN-CE an SPL of 65.1, 14.1 points above Table 1's 51.0, with no stated difference in split, metric, or checkpoint. On VSI-Bench, the Avg column of Table 2 is not the simple mean of the eight listed per-task scores (VEME: 46.6 vs 49.3; Spatial-MLLM: 47.0 vs 48.4), so the 'surpassed' claim is not reproducible from the table. Appendix C.4 reports Wilcoxon signed-rank p-values (0.0003, 0.0005, 0.0007, 0.0011) for n=5; the minimum possible two-sided p for n=5 is 1/16=0.0625, so the significance-test results as reported cannot be correct. These are internal mechanical inconsistencies, not disagreements about baseline choices, and they directly affect the headline comparisons. The static-environment limitation is a real scope restriction, but it is secondary: the numerical contradictions make the paper's own evidence unreliable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VEME, a VLM extension for embodied navigation and spatial reasoning. VEME augments Qwen-2.5-VL-7B with (i) a learnable world embedding serving as spatial semantic memory, (ii) a cross-modal alignment module that grounds 2D visual semantics in geometric features via cross-attention and a contrastive loss, and (iii) an episodic memory module that encodes trajectory and point-cloud history into a query vector attended against the world embedding. The model is trained end-to-end with a weighted combination of task loss, spatial loss, and episodic loss, and is evaluated on VLN-CE (R2R, RxR) and VSI-Bench. The paper claims considerable improvements over specialized baselines such as ETPNav, NaVILA, and Spatial-MLLM.","tokens_in":18476,"tokens_out":4142,"duration_ms":46391,"significance":"If the headline results were reliable, VEME would provide a concrete, modular recipe for injecting geometric and episodic priors into VLMs, with a clearly specified architecture and training setup. The work is not circular in the derivation sense: the contrastive losses are learned supervision, not fitted parameters that analytically reduce the benchmark results. The paper also ships a high level of implementation detail (encoders, LoRA, hyperparameters, data recipe). However, the empirical claims are the central contribution, and the reported evidence contains internal numerical inconsistencies that undermine those claims. The static-environment limitation stated in Appendix E also directly contradicts the abstract's dynamic-environment claim.","major_comments":[{"comment":"The headline VLN-CE results are internally inconsistent. Section 4.2 states VEME achieves SR 57.0 and SPL 46.7 on R2R Val-Unseen, but Table 1 lists SPL 51.0. The same paragraph says ETPNav scores 42.0/36.5, while Table 1 lists ETPNav as 57.0/49.0. Table 3 reports the Full Model SPL as 65.1 on VLN-CE, 14.1 points above Table 1's 51.0, with no stated difference in split, metric, or checkpoint. These are not baseline-choice disagreements; the paper's own tables do not support a stable comparison.","section":"§4.2, Table 1, Table 3"},{"comment":"The reported Wilcoxon signed-rank p-values are impossible for n=5. With five paired observations, the smallest achievable two-sided p-value is 2/2^5 = 0.0625 (and 1/32 = 0.03125 one-sided), assuming no ties. The values p=0.0003, 0.0005, 0.0007, and 0.0011 cannot be obtained from five samples. The statistical significance claims for the ablations are therefore unsupported and the reported numbers cannot be correct.","section":"Appendix C.4"},{"comment":"The VSI-Bench 'Avg' column is not reproducible from the per-task scores. For VEME, the eight listed task accuracies average to 372.9/8 = 46.6, not the reported 49.3; for Spatial-MLLM they average to 375.6/8 = 47.0, not 48.4. No alternative weighting formula is given. As a result, the claim that VEME 'surpassed' Spatial-MLLM on average cannot be verified from the table.","section":"Table 2"},{"comment":"The abstract claims improvements in 'dynamic environments' and 'dynamic, open-set tasks', and the introduction repeats this framing. Appendix E explicitly states that 'our current framework operates under a static environment assumption. It is not designed to handle dynamic scenes with moving objects, interacting agents, or significant changes in layout.' Since the method relies on a complete global point cloud Pt at decision time (Section 3.1) and on geometric/episodic memory built from that static input, the dynamic-environment claim is overstated. This is a scope mismatch, not a minor wording issue.","section":"Abstract and Appendix E"}],"minor_comments":[{"comment":"Heading contains a typo: 'Premilinaries' should be 'Preliminaries'.","section":"§3.1"},{"comment":"The reference 'brian ichter et al. 2022' is not properly capitalized; the author name should be formatted consistently with other references.","section":"References"},{"comment":"LoRA rank is given as r=8 in the main text and Section A.2, but Table 5 lists LoRA Rank = 16. These need to be reconciled.","section":"A.3 / Table 5"},{"comment":"The 'Computational Cost and Efficiency' section appears to be empty in the submitted manuscript, immediately followed by C.3. Either content is missing or the heading is stray.","section":"Appendix C.2"},{"comment":"The 'Rank' column is internally inconsistent: Gemini-1.5 Pro and Spatial-MLLM both receive rank 2, while the subsequent rank labels (3, 4, 5, ...) do not match a strict rank ordering. This makes the rank column confusing.","section":"Table 2"},{"comment":"The heading 'Computational Cost and Efficiency' appears placed within the visual navigation results paragraph, and there is no corresponding content there. This is likely a formatting artifact but should be fixed.","section":"§4.2"}],"recommendation":"reject","confidential_remarks":"The internal contradictions are at the level of the paper's central evidence: the same model has three different SPL values across tables/text, the VSI-Bench average is not derivable from the listed scores, and the reported significance tests are numerically impossible. I do not see how these can be resolved by a revision without the authors rerunning experiments and providing raw per-run results. If a corrected manuscript with consistent numbers were submitted, the core idea might be worth another review; as it stands, the claims are not reliable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is a concrete recipe: take Qwen-2.5-VL, feed it a learnable world embedding plus spatially and episodically contrastive losses that align DINOv2 semantics with VGGT geometry and Sonata point clouds, and show some navigation and spatial-QA gains. That integration is new and sensible, and the qualitative results suggest the episodic memory is doing something real. I also want to credit the authors for stating the static-environment assumption in the appendix; that is an honest limit.\n\nThe problem is the paper's own evidence. Table 1 lists VEME R2R Val-Unseen SPL 51.0, while the main text says 46.7 and describes ETPNav as 42.0/36.5 even though the same table lists ETPNav at 57.0/49.0. Table 3 gives the full model an SPL of 65.1, 14 points higher than Table 1 with no explanation. On VSI-Bench, the Avg column (49.3 for VEME, 48.4 for Spatial-MLLM) is not the simple mean of the eight per-task scores listed (46.6 and 47.0 respectively). And the Wilcoxon results in Appendix C.4 report p=0.0003 for n=5; the smallest possible two-sided p with five paired samples is 1/16. These are not preference disagreements; they are mechanical errors in the core reporting. Until they are fixed, the headline claim of 'considerable improvements' is not supported by the paper as written.\n\nThe architecture may still have merit, but the empirical demonstration is unreliable. The static-environment caveat is real but secondary here. The citation pattern is fine; the self-citations are not an issue.\n\nWho is this for? People working on memory-augmented VLMs for embodied tasks will find the design worth reading, but they should not trust the numbers without checking raw results. It deserves a serious referee—the topic matters and the method is plausible—but the current version should be returned for major revision, not accepted.","headline":"A plausible dual-memory architecture for video-based embodied models, but the reported numbers contradict each other and the claimed gains are unverified.","tokens_in":18913,"tokens_out":2321,"would_cite":false,"duration_ms":23926,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that a video-based vision-language model can navigate and reason about space far better if it is given two memory systems — a learnable geometric-semantic world embedding that acts as a cognitive map, and an episodic memor","keywords":["embodied AI","vision-language navigation","world model","episodic memory","spatial reasoning","multimodal alignment","point cloud","video question answering"],"falsifier":"Run VEME in the Habitat simulator but alter the scene after the agent builds its point cloud — move a chair or open a door mid-episode — and measure SR/SPL; if navigation success does not degrade, the static-scene assumption is not limiting. Separately, corrupt the point cloud (drop 30% of points or add Gaussian noise) and check whether the 15.3-point episodic-memory advantage survives; if it vanishes, the headline gains depend on map quality rather than on the memory design.","tokens_in":17936,"feed_emoji":"🧭","tokens_out":6772,"duration_ms":69520,"temperature":0.7,"pith_summary":"VEME is a training-time wrapper for a 7B vision-language model that tries to close the embodied reasoning gap: standard VLMs see 2D frames but have no persistent sense of 3D layout or of where the agent has been. The paper claims that adding a world embedding (spatial semantic memory) and a trajectory-conditioned episodic memory, trained with two contrastive losses, makes the same VLM generalize to unseen scenes. On VLN-CE R2R Val-Unseen it reports SR 57.0 and SPL 51.0, and on VSI-Bench an average of 49.3, ahead of the listed baselines. Ablations suggest episodic memory is the largest single contributor on navigation. If true, the practical upshot is that spatial intelligence for embodied agents can be injected into an off-the-shelf VLM without building an explicit map.","feed_headline":"Dual memory pushes a 7B VLM to 57.0 navigation success","feed_subtitle":"Adding a world embedding and trajectory memory to Qwen-2.5-VL beats specialized baselines on VLN-CE and VSI-Bench, the paper reports.","key_machinery":"The load-bearing object is the world embedding Eworld, a learnable matrix meant to serve as a geometric-semantic cognitive map: current-frame visual semantics are bound to image-derived geometry via cross-attention and a spatial contrastive loss, while a query made from the global point cloud and action history attends to Eworld to form an episodic trace under an episodic contrastive loss. All of these token streams are then concatenated into the VLM's input, so the model's own attention decides what to retrieve.","core_discovery":"The paper's central claim is that the reason embodied VLMs fail is not lack of visual semantics but spatial naivety and statelessness, and both can be fixed by a dual-memory architecture. A learnable world embedding Eworld is grounded to geometry by cross-attending visual features to depth/normal-aware features and pulling matched pairs together with a spatial contrastive loss. A separate episodic query, built by concatenating global point-cloud features and the action trajectory, attends to Eworld to produce a per-episode fingerprint, trained with an episodic contrastive loss. These memory tokens are concatenated into the VLM's input sequence, letting Qwen-2.5-VL reason over instruction, cu","pith_inferences":["Because the inputs include a complete global point cloud at decision time and the appendix states the framework assumes static scenes, the abstract's claim of gains in 'dynamic environments' is untested; the real-world benefit depends on whether noisy, incomplete, or changing SLAM maps can be tolerated.","The episodic-memory mechanism could plausibly extend to other spatially grounded VLM tasks — 3D visual grounding, object-state tracking, or manipulation affordance prediction — but the paper only demonstrates navigation and QA, so that transfer is an open question rather than a result.","The RxR numbers in Table 1 tell a more modest story than the R2R numbers: VEME trails ETPNav on RxR Val-Unseen (50.7 vs 54.7 SR), so the claimed superiority is specific to the R2R benchmark and to SPL.","A direct stress test of the framework's robustness would be to feed it point clouds with simulated sensor noise or occlusion, since the geometric and episodic modules are both built on that input; the ablation table suggests VGGT removal collapses MRA from 55.7 to 19.2, so the system's spatial QA is sharply dependent on geometry-feature quality."],"forward_implications":["Episodic memory, not just scene geometry, is what the ablations identify as the biggest lever: removing it costs 15.3 SPL points, suggesting trajectory history matters more than spatial grounding for navigation success.","The same architecture, without task-specific heads, transfers across navigation (VLN-CE) and video spatial QA (VSI-Bench), implying a single world-embedding mechanism can serve both planning and question answering.","Because the memory tokens are fed directly into the VLM's context, the approach is compatible with any instruction-tuned VLM, so gains should compound as base models improve.","The spatial contrastive loss forces a unique correspondence between 2D semantics and geometric features, which is a direct, testable training signal for object-to-geometry binding.","On R2R Val-Unseen the method ties ETPNav's SR (57.0) while improving SPL (51.0 vs 49.0), so the claimed edge on that benchmark is primarily path efficiency rather than raw success."],"supporting_citations":[{"why":"The base VLM (Qwen-2.5-VL-7B) that VEME wraps; all memory tokens are fed into its input sequence.","marker":"Bai et al. 2025a"},{"why":"VGGT supplies the image-based geometric encoder whose depth/normal-aware features ground visual semantics; ablating it drops VSI-Bench MRA sharply.","marker":"Wang et al. 2025"},{"why":"Sonata is the point-cloud backbone that produces global 3D features for the cognitive map and the episodic query.","marker":"Wu et al. 2025b"},{"why":"VSI-Bench is one of the two evaluation benchmarks, providing the spatial-QA tasks and baseline comparison numbers.","marker":"Yang et al. 2024"},{"why":"ETPNav is the strongest VLN-CE baseline whose R2R Val-Unseen and RxR numbers define the comparison the paper claims to match or beat.","marker":"An et al. 2024"},{"why":"NaVILA is a VLM-based navigation baseline that VEME is compared against on both VLN-CE splits.","marker":"Cheng et al. 2025b"},{"why":"Spatial-MLLM is the prior state of the art on VSI-Bench whose 48.4 average VEME claims to surpass with 49.3.","marker":"Wu et al. 2025a"},{"why":"DINOv2 supplies the visual semantic encoder for RGB frames, the 'what' stream that gets geometrically grounded.","marker":"Oquab et al. 2023"}],"fun_headline_variants":["Spatial priors ground VLM memory for embodied navigation","Dual memory boosts VLM navigation on unseen scenes","VEME: embedding spatial priors into VLM context","World priors give VLMs spatial memory for tasks"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole apparatus assumes a clean, complete, static 3D point cloud of the scene is available at decision time; if real maps are noisy or the scene changes, the memory modules' claimed benefits are not shown to transfer.","fun_headline_variants_meta":{"raw":{"variants":["Spatial priors ground VLM memory for embodied navigation","Dual memory boosts VLM navigation on unseen scenes","VEME: embedding spatial priors into VLM context","World priors give VLMs spatial memory for tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000855,"raw_usage":{"total_tokens":3567,"prompt_tokens":776,"completion_tokens":2791,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":2733}},"tokens_in":520,"tokens_out":2791,"duration_ms":20949,"temperature":1.0,"reasoning_tokens":2733,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:49:36.253048+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run VEME in the Habitat simulator but alter the scene after the agent builds its point cloud — move a chair or open a door mid-episode — and measure SR/SPL; if navigation success does not degrade, the static-scene assumption is not limiting. Separately, corrupt the point cloud (drop 30% of points or add Gaussian noise) and check whether the 15.3-point episodic-memory advantage survives; if it vanishes, the headline gains depend on map quality rather than on the memory design.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"VGGT supplies the image-based geometric encoder whose depth/normal-aware features ground visual semantics; ablating it drops VSI-Bench MRA sharply."}],"review_version":1}