{"id":"61941c9d-f3d3-4f77-ac40-4c5494d88111","arxiv_id":"2608.01899","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SpatioLM is a parameter-efficient side module for frozen VLMs, trained with pseudo depth and camera supervision, that achieves state-of-the-art spatial perception and understanding while preserving general capabilities.","lead":"SpatioLM adds a small, plug-in side module to a frozen vision-language model, training it on pseudo depth maps and camera ray information to improve spatial reasoning. The method reports state-of-the-art results on spatial benchmarks, including the first VSI-Bench average above 70, while keeping the base model's general abilities largely intact.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unverified scene-level disjointness between VSI-590K and VSI-Bench (and between self-built MD-S/MD-M eval and training sources) is the load-bearing risk; an overlap audit would settle it.","rationale":"The reader's weakest assumption is exactly the condition on which the headline result depends: genuine scene-level separation between the spatial-understanding training corpus and the evaluation benchmarks. The paper's only defense is a one-sentence claim in Sec. 4.1 and App. B.1, with no released data artifacts, no scene-ID comparison, and no near-duplicate analysis. VSI-590K is a constructed corpus rather than an official training split, so the claim cannot be verified from the paper alone. The same structural gap affects the self-built MD-S/MD-M benchmarks, where the training corpus and evaluation sets draw from the same source datasets (Waymo, ScanNet), making split-level leakage a real possibility even if the official splits are nominally disjoint. I do not regard this as evidence of misconduct; rather, it is the single most load-bearing unverified empirical condition. The paper has independent strengths: code is released, ablations are thorough, loss-weight sensitivity is small, DA-2K is an external benchmark, and LIBERO results are internally consistent with the base-model comparisons. Those strengths prevent me from moving to REJECT, but they do not remove the need for an overlap audit. Since the reader already reached CONDITIONAL, my read does not change the verdict; it sharpens the reason for it and specifies the test that would resolve it.","tokens_in":31254,"tokens_out":7756,"duration_ms":74411,"concrete_test":"Run an overlap audit before accepting the headline: (1) compare scene IDs, video names, and frame-level perceptual hashes (e.g., pHash or DINOv2 feature similarity with a fixed threshold) between every VSI-590K training sample and every VSI-Bench video; (2) compare ScanQA and SQA3D training scene IDs against their validation/test scene IDs; (3) compare MD-S/MD-M evaluation samples from Waymo, ScanNet, and KITTI against the corresponding training splits used in Table 9, including a check for whether Waymo training and evaluation logs share geographic areas or timestamps. If any near-duplicate is found, re-run Tables 1-3 on the overlap-free subset and report the delta. If the audit is clean, the concern is refuted and the conditional can be lifted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim, 'first to exceed 70 on VSI-Bench' (71.6, Table 2), rests on the assertion in Sec. 4.1 and App. B.1 that all training samples are drawn strictly from official training splits, with validation/test splits reserved for evaluation. This assertion is not backed by any reported overlap analysis, and it is not self-evident: VSI-590K is a re-packaged multi-source corpus (Yang et al. 2025c) rather than an official split of VSI-Bench, and the two resources share lineage. If any VSI-590K video or scene also appears in VSI-Bench, the +2.9 over the 68.7 base could reflect memorization rather than spatial reasoning, invalidating the headline. The same gap applies to the self-constructed MD-S/MD-M benchmarks (App. B.2): Waymo and ScanNet appear in both the Table 9 training corpus and the MD-S/MD-M evaluation sets, and the paper only asserts split-level separation without checking whether Waymo logs from the same physical intersections or ScanNet frames from the same rooms leak across splits. This is an unverified condition, not an observed failure; the ablations, sensitivity analyses, and public DA-2K/LIBERO results provide independent support. But because the strongest claim is precisely the one most exposed to overlap, the conclusion should remain conditional until the audit is released.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes SpatioLM, a parameter-efficient framework that augments a frozen vision-language model with a plug-and-play Spatio-Vision Module (SV-Module) to improve spatial perception and understanding without extra 3D priors or third-party spatial encoders at inference. The module is trained with pseudo depth and camera-ray supervision, a distillation loss from a 3D foundation model, and a language modeling loss, while the VLM backbone remains frozen. The authors report state-of-the-art results on VSI-Bench (Avg 71.6, described as the first to exceed 70), ScanQA, SQA3D, and their self-constructed metric-depth benchmarks (MD-S, MD-M, DR), competitive results on DA-2K, limited degradation on general-purpose benchmarks, and improved performance on LIBERO manipulation tasks when adapted to a vision-language-action model.","tokens_in":31700,"tokens_out":4698,"duration_ms":42057,"significance":"The paper has several genuine strengths. The central design is clean: a frozen backbone with a compact side module (about 0.3B trainable parameters) that can be activated or deactivated at inference, and the ablations cover the main components (SV-Module, Vision Token Supervision, Dense Geometric Supervision), loss-weight sensitivity, attention mechanism, and number of blocks. The public-benchmark results on DA-2K, ScanQA, SQA3D, and LIBERO are external and not produced by the authors, which reduces the risk of circular evaluation. The paper also explicitly states limitations, including pseudo-label noise, a motion-dynamics trade-off, and safety concerns, which is commendable. If the data-overlap concern is resolved, the reported VSI-Bench result would be a notable advance in spatial VLM capabilities without external geometric inputs. The main uncertainty is whether the training and evaluation corpora are truly scene-disjoint; this is an unverified condition rather than an observed failure, but it is load-bearing for the headline claims.","major_comments":[{"comment":"The assertion that 'all training samples are strictly sourced from the official training splits' is not sufficient to rule out train/evaluation overlap for the central claims. VSI-590K is a re-packaged multi-source corpus rather than a dataset with a canonical official split, and the self-constructed MD-S, MD-M, and DR benchmarks use sources (e.g., Waymo, ScanNet, KITTI) that also appear in the training corpus. No scene-level or frame-level overlap audit is reported. Because the headline result (71.6 on VSI-Bench, 'the first to exceed 70') is the most exposed to memorization if any training video or scene also appears in evaluation, please provide a quantitative overlap analysis (e.g., scene-ID matching or frame hashing) across all training and evaluation sets, and report the fraction of overlapping samples if any are found.","section":"Sec. 4.1, App. B.1"},{"comment":"Three of the four perception benchmarks (MD-S, MD-M, DR) are constructed by the authors, yet the paper reports no error bars, confidence intervals, or statistical significance tests. For close comparisons, such as the DA-2K scores of 83.8 for both SpatioLM and VST-7B-SFT, the absence of variance makes the claimed superiority difficult to assess. In addition, the benchmark construction is described at a high level (e.g., 'randomly select 10 spatial locations' and '500 evaluation samples per dataset') without releasing the exact evaluation files. Please release the constructed benchmarks and report standard errors or repeated-run variability so that the perception results are independently verifiable.","section":"Sec. 4.2.1, App. B.2"},{"comment":"The comparison of general-capability degradation against Spatial-MLLM is not controlled. Figure 6 compares the relative drops of SpatioLM built on SenseNovaSI-8B or InternVL3.5-8B with the relative drop of Spatial-MLLM built on Qwen2.5-VL-3B; because the base models differ in size and architecture, the large relative drop of Spatial-MLLM cannot be attributed solely to the added spatial module. The own-base comparisons in Table 14 (drops of about 3-4%) are more meaningful and do support the claim that SpatioLM mildly degrades general capability, but the comparative statement against Spatial-MLLM should either be re-run on the same base models or softened to avoid an apples-to-oranges comparison.","section":"Sec. 4.3, Fig. 6"}],"minor_comments":[{"comment":"There are typographical inconsistencies in model names, e.g., 'LLaV A-NeXT' instead of 'LLaVA-NeXT', and 'InternVL3.5' versus 'InternVL3.5-VL' is used inconsistently throughout the tables and text.","section":"Tables 10-11"},{"comment":"The joint distribution in Eq. (3) factorizes as P(Y_g | X_v, X_t) followed by text generation conditioned on Y_g, but the paper later describes the auxiliary geometric outputs as supervision rather than as inputs to generation. Please clarify whether Y_g is used as a conditioning variable in the language modeling loss or is only a training-side auxiliary output.","section":"Eq. (3)"},{"comment":"The choice of the intermediate ViT layer is described as 'guided by empirical analysis' and 'layers around 16/24,' but no ablation or validation-set result is shown for this selection. Please specify the exact layer index and provide the evidence for this choice.","section":"Sec. 3.2"},{"comment":"The citation for VSI-590K is given as (Yang et al., 2025c), but the associated reference is the Cambrian-S paper; please ensure that the attribution is correct and that the reader can locate the original VSI-590K source.","section":"App. B.1"},{"comment":"The meaning of the 'P.T.' (pretraining on large-scale robot manipulation data) column is not defined in the table caption or the text. Since the paper emphasizes that SpatioLM does not use such pretraining, please define the column and clarify which baselines use it.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is well-written and the proposed method is interesting. The primary risk is data leakage between the training corpus and the evaluation benchmarks, particularly VSI-Bench and the self-constructed MD-S/MD-M/DR sets; the authors should be required to release an overlap audit and the evaluation benchmarks as part of the revision. Also, the acknowledgment thanking reviewers, area chairs, and program chairs appears unusual for a submitted manuscript and should be removed in the final version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a well-built system paper that deserves to be in the record, but the headline 71.6 on VSI-Bench is conditional on a data-hygiene claim the authors assert but do not actually demonstrate. If you read it, read Sec. 4.1 and App. B.1 carefully.\n\nWhat is genuinely new is the recipe: a frozen VLM with a ControlNet-style zero-initialized side branch, intermediate ViT features, VGGT-style alternating attention, a dual DPT head, and pseudo-depth/ray supervision. None of the pieces are new individually, but the combination is, and the evaluation is more thorough than most. The ablations cover the main components (SV-Module, VTS, DGS, attention type, number of blocks, loss weights), and the general-capability preservation is real: SpatioLM drops only 3-4% on MMMU/VideoMME/etc. while Spatial-MLLM drops over 40%. The LIBERO transfer is a nice extra, though the gains over the strong OFT baseline are modest (+1.0).\n\nThe soft spots are real but mostly addressable. The big one is the overlap risk. VSI-590K is a repackaged multi-source corpus from Yang et al. 2025c, and VSI-Bench comes from Yang et al. 2025a; the two share lineage. The paper says all training samples come from official training splits, but it never reports an overlap audit, and the same issue applies to the self-built MD-S/MD-M/DR benchmarks: Waymo and ScanNet appear in both the Table 9 training corpus and the evaluation sets. This is an unverified condition, not an observed failure. The public benchmarks (DA-2K, ScanQA, SQA3D, LIBERO) provide independent support, and the qualitative depth outputs look plausible. But the strongest claim — first model above 70 on VSI-Bench — is exactly the one most exposed to memorization, so the conclusion should stay conditional until the authors release the audit.\n\nTwo smaller issues. First, there are no error bars anywhere; single runs, and the loss-weight sensitivity table gives no variance, so we cannot tell if the 71.6 vs. 70.4 attention difference is noise. Second, the word \"elicit\" oversells things: the supervision is largely distillation from Depth Anything V3 pseudo-labels and token-level teacher features, so this is teacher injection as much as elicitation. Not a fatal problem, but the framing should be toned down.\n\nThe limitations section is honest about pseudo-label dependence, the MVBench motion trade-off, and safety-critical deployment, which I appreciate. The citation pattern looks fair; I do not see obvious suppression of competing work.\n\nWho this is for: anyone working on spatial reasoning in VLMs or embodied perception. It is worth a serious referee. My recommendation: send it to review, and ask the authors to supply the train/eval overlap analysis for VSI-590K/VSI-Bench and the self-built benchmarks, plus error bars for the central results. If the audit is clean, this is a strong accept.","headline":"Solid spatial-VLM systems result; headline benchmark number rests on an unverified train/eval overlap claim.","tokens_in":32106,"tokens_out":1894,"would_cite":true,"duration_ms":18744,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SpatioLM claims a 0.3B add-on module can give a frozen VLM state-of-the-art spatial reasoning without 3D inputs, reporting 71.6 on VSI-Bench.","keywords":["spatial intelligence","vision-language models","metric depth estimation","spatial understanding","parameter-efficient side module","pseudo-depth supervision","embodied manipulation"],"falsifier":"Check the scene identifiers: list every scene and video source in the VSI-590K, ScanQA, and SQA3D training splits and intersect them with the scene identifiers of VSI-Bench, ScanQA val, and SQA3D test; any shared scene invalidates the headline 71.6, and re-running the evaluation with all overlapping scenes removed gives the honest number.","tokens_in":31072,"feed_emoji":"📏","tokens_out":6687,"duration_ms":54562,"temperature":0.7,"pith_summary":"The paper tries to establish that a frozen vision-language model (VLM) can acquire strong spatial perception and understanding by adding a small, plug-and-play side module, with no 3D sensor inputs, no external spatial encoder, and no retraining of the backbone. If true, this resolves a practical dilemma: previous spatial-VLM methods either depend on depth maps, point clouds, or tool APIs at inference, or fine-tune the backbone and lose general-purpose ability. SpatioLM reports the first average score above 70 on the VSI-Bench spatial-understanding benchmark (71.6), top metric-depth results (83.5 on single-image, 69.0 on multi-image), and 83.8 on the relative-depth DA-2K benchmark, while keeping degradation on four general benchmarks within roughly 4%. The same side module transfers to embodied manipulation, raising the LIBERO robot-manipulation benchmark success rate from 79.2% to 91.0% in a discrete-action setting.","feed_headline":"SpatioLM is the first VLM to top 70 on VSI-Bench","feed_subtitle":"A 0.3B plug-and-play module adds metric depth and spatial reasoning to a frozen VLM, with general skills intact.","key_machinery":"The load-bearing mechanism is the Spatio-Vision Module (SV-Module): about 0.3B parameters of stacked Spatio-Vision Blocks placed alongside a frozen VLM, with each block alternating one frame-attention pass (intra-frame geometry) and one global-attention pass (cross-frame consistency), a pattern adapted from multi-view geometry transformers. Its outputs are fused into frozen language-block hidden states through zero-initialized projections, so the pretrained distribution is perturbed only gradually. The module is steered by three supervisions: a language-modeling loss, a Gram-matrix token-distillation loss toward a pretrained depth teacher, and dense geometric losses on pseudo depth and camera ray maps from a dual Dense Prediction Transformer (DPT) head. The DPT head is discarded at inference, and the whole module is a side path that can be toggled on or off.","core_discovery":"SpatioLM's central claim is that the geometric knowledge needed for spatial reasoning already exists, latent, in a pretrained VLM's intermediate visual tokens, and can be elicited on demand without changing the VLM. The authors freeze both the vision encoder and the language model, and train only a Spatio-Vision Module built from alternating frame-level and global attention blocks plus zero-initialized projection layers that inject geometry-aware features into selected frozen language-model layers. During training, a dual DPT head predicts pseudo depth maps and camera ray maps, and a token-distillation loss aligns the module's features with a pretrained 3D foundation model; at inference the head is removed, the module runs once during prefilling, and ordinary text decoding follows. On this recipe the paper reports state-of-the-art numbers across spatial perception and understanding benchmarks, including an average of 71.6 on VSI-Bench, the first reported result above 70, while the frozen backbone limits general-capability degradation to about 3-4%.","pith_inferences":["A natural next stress test is to detach the SV-Module from its training teacher: if the module is truly eliciting latent geometry rather than memorizing teacher outputs, performance should survive on scenes and depth distributions the teacher never saw.","Because the paper itself reports motion-heavy MVBench subtasks as the main casualty, supplementing the geometric losses with an explicit temporal-dynamics signal such as optical flow is a direct, testable extension that could recover action counting and moving-direction accuracy while keeping the spatial gains.","The scene-disjoint split is the crux to check: recomputing VSI-Bench, ScanQA, and SQA3D numbers after filtering any scene overlap between training and evaluation would settle whether the 71.6 reflects spatial reasoning or memorization.","If the side-module recipe generalizes across base VLMs, it offers a cheap way to retrofit existing deployments with spatial ability without re-certifying the whole model."],"forward_implications":["A frozen VLM can gain metric depth, relative depth, and spatial question-answering ability by training only a roughly 0.3B side module, so spatial enhancement no longer forces full-model fine-tuning.","Because the VLM backbone stays frozen, the same base model can serve both ordinary and spatially augmented modes, with the SV-Module adding no per-token decoding overhead after a single prefill pass.","If the reported results replicate, SpatioLM is the strongest reported RGB-only spatial VLM without external geometry at inference, on both perception and understanding benchmarks.","The same side module transfers to embodied manipulation, improving LIBERO average success from 79.2% to 91.0% under discrete actions and from 95.3% to 96.3% under continuous actions on a 2B base model."],"supporting_citations":[{"why":"Supplies the QA-style metric-depth task formulation and the DepthLM baseline that SpatioLM extends to multi-image and raw-RGB settings.","marker":"Cai et al., 2025c"},{"why":"Provides the DPT head design and the Depth Anything V3 teacher whose pseudo depth and ray labels supervise the SV-Module.","marker":"Lin et al., 2025"},{"why":"The alternating frame/global attention pattern is adapted from this multi-view geometry transformer.","marker":"Wang et al., 2025a"},{"why":"Defines VSI-Bench, the spatial-understanding benchmark where the paper reports 71.6, the first score above 70.","marker":"Yang et al., 2025a"},{"why":"Supplies SenseNovaSI-8B, the base VLM used for the best-performing SpatioLM variant.","marker":"Cai et al., 2025b"},{"why":"Supplies InternVL3.5-8B, the second frozen backbone used for SpatioLM.","marker":"Wang et al., 2025b"},{"why":"Contributes VSI-590K, the large spatial instruction-tuning corpus used for spatial-understanding training.","marker":"Yang et al., 2025c"},{"why":"Justifies selecting intermediate ViT tokens as the module's input for stronger spatial embeddings.","marker":"Bolya et al., 2026"},{"why":"Provides Spatial-MLLM, the external-encoder baseline whose large general-capability degradation motivates the non-invasive design.","marker":"Wu et al., 2025"}],"fun_headline_variants":["First VLM to top 70 on VSI-Bench with frozen backbone","SpatioLM: latent spatial knowledge elicited, no 3D priors","Spatio-Vision Module: 0.3B boost to VLM spatial IQ","71.6 on VSI-Bench: SpatioLM cracks spatial reasoning","SpatioLM: Freeze VLM, add module, beat spatial benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central numbers depend on the claim that no scene appears in both the training corpus and the VSI-Bench, ScanQA-val, and SQA3D-test evaluation sets; if any source scene leaks across the split, the reported gains could come from memorization rather than spatial reasoning.","fun_headline_variants_meta":{"raw":{"variants":["First VLM to top 70 on VSI-Bench with frozen backbone","SpatioLM: latent spatial knowledge elicited, no 3D priors","Spatio-Vision Module: 0.3B boost to VLM spatial IQ","71.6 on VSI-Bench: SpatioLM cracks spatial reasoning","SpatioLM: Freeze VLM, add module, beat spatial benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000677,"raw_usage":{"total_tokens":3103,"prompt_tokens":993,"completion_tokens":2110,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":609,"completion_tokens_details":{"reasoning_tokens":2004}},"tokens_in":609,"tokens_out":2110,"duration_ms":14063,"temperature":1.0,"reasoning_tokens":2004,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:03:36.083566+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check the scene identifiers: list every scene and video source in the VSI-590K, ScanQA, and SQA3D training splits and intersect them with the scene identifiers of VSI-Bench, ScanQA val, and SQA3D test; any shared scene invalidates the headline 71.6, and re-running the evaluation with all overlapping scenes removed gives the honest number.","supporting_citations":[],"review_version":2}