{"id":"69f1a2e2-ee95-4938-8814-0087eede5cf6","arxiv_id":"2608.10278","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Space Tokens distill 3D reconstruction and object bounding box knowledge into continuous latent tokens that a VLM can reason over, improving VSI-Bench scores, though the RL stage's evaluation independence is not established.","lead":"The paper introduces 'Space Tokens', continuous latent tokens that carry 3D scene geometry and object bounding box information inside a vision-language model's own chain of thought. It reports state-of-the-art object size and room size estimates on VSI-Bench, but the reinforcement learning stage appears to optimize the benchmark metric directly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 8,000 stage-3 GRPO samples are unnamed; if they come from VSI-Bench, the headline gains and SOTA object/room-size numbers reflect reward overfitting rather than general spatial reasoning.","rationale":"My stress-test pass agrees with the Reader's verdict. The central empirical claim rests on the final stage-3 models, yet the paper never discloses the source of the 8,000 GRPO samples while admitting the reward is VSI-Bench-specific (Appendix A.3) and excluding the stage-3 model from OOD evaluation. If those samples are from VSI-Bench, the reported +4.3/+1.3 improvements and the SOTA object-size/room-size numbers are the product of direct reward optimization, not of the spatial-token mechanism. This would not invalidate the stage 1&2 results (68.6 vs 67.6/67.1 controls), but the headline claims as stated would be unsupported. I also examined the TRT/attention-masking tension: the negligible TRT effect could indicate the tokens are attention anchors rather than information carriers, but the attention masking experiment provides direct causal evidence that the VLM uses the token representations, and the decoding visualizations show geometric content; this is a secondary mechanistic question, not the primary threat to the empirical claim. The frame-selection procedure is also test-set-informed, but it is of smaller magnitude than a potential contamination of the RL stage. Therefore the single check that would settle the concern is disclosure and, if necessary, re-evaluation on a held-out split. This does not change the Reader's CONDITIONAL verdict.","tokens_in":16329,"tokens_out":7109,"duration_ms":66903,"concrete_test":"Inspect the training configuration and any released checkpoints to identify the dataset and split for the 8,000 stage-3 GRPO samples, and check for overlap with the VSI-Bench evaluation set. If the samples are from VSI-Bench or overlap with it, re-run stage 3 using a disjoint held-out split of VSI-Bench or a different spatial benchmark, then report the resulting VSI-Bench scores; if the 79.2/75.7 SOTA numbers shift or the +1.3 SenseNova gain disappears, the headline claim is invalid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's headline numbers come from the final stage-3 model: Table 1 shows Qwen3-VL-8B +4.3 and SenseNova-SI-1.3 +1.3, with SOTA object size (79.2) and room size (75.7). Appendix A.3 states the GRPO reward 'target[s] the VSI-Bench metric' (fuzzy matching for MCA, mean relative accuracy with confidence intervals for NA). The Data paragraph says stage 3 'exposes the model to an additional 8,000 samples' but does not name the dataset or a train/eval split. If those 8,000 samples are drawn from VSI-Bench, the RL stage directly optimizes the evaluation metric on the evaluation distribution, so the reported gains would be expected even if the spatial tokens contributed nothing. The paper itself excludes the stage-3 model from OOD benchmarks with the note that 'the reward functions are designed for VSI-Bench and are therefore not appropriate for evaluating generalization' — an explicit admission that the final model is specialized to VSI-Bench. The stage 1&2 model (68.6) does improve over both the base (67.6) and the VICA-only control (67.1) in Table 3, so the core token mechanism may survive, but the headline 4.3/1.3 improvements and the SOTA object-size/room-size claims are not interpretable without a disclosed held-out evaluation. This is the load-bearing weakness for the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Space Tokens, a training-only framework that augments vision-language models with continuous latent tokens encoding scene-level 3D geometry (distilled from VGGT-Omega) and object-centric 3D bounding boxes. The tokens are placed in the model's vocabulary and generated during chain-of-thought reasoning, so no additional modules are needed at inference. Training proceeds in three stages: representation learning with reconstruction losses, supervised fine-tuning that teaches the model to reason with the tokens, and a GRPO refinement stage with a reward derived from the VSI-Bench metric. The authors report improvements on VSI-Bench for Qwen3-VL-8B (+4.3) and SenseNova-SI-1.3 (+1.3), state-of-the-art object-size (79.2%) and room-size (75.7%) scores, and modest or positive out-of-distribution results for the Stage 1&2 model.","tokens_in":16639,"tokens_out":5338,"duration_ms":49376,"significance":"If the reported results are valid, Space Tokens would be a valuable contribution: it provides an architecture-agnostic, inference-efficient route for injecting explicit spatial knowledge into VLMs, and the attention-masking experiment (Table S5) gives causal evidence that the learned latent representations are actually used during answer generation. The authors also provide a thoughtful discussion of why the Token Replacement Test is insufficient in multi-image settings, and they include a data-scaling ablation that helps separate the effect of the tokens from the effect of additional training data. The main significance hinges, however, on whether the headline VSI-Bench gains reflect genuine spatial-reasoning improvement or optimization against the evaluation metric; as written, the empirical evidence for that distinction is incomplete.","major_comments":[{"comment":"The Stage 3 GRPO reward is defined directly from the VSI-Bench evaluation metric (Appendix A.3: fuzzy matching for MCA, mean relative accuracy for NA), and the 8,000 Stage 3 training samples are never named or split-specified. The paper itself excludes the Stage 3 model from OOD benchmarks with the explanation that the reward functions are designed for VSI-Bench and are therefore not appropriate for evaluating generalization. This means the headline improvements (Qwen3-VL-8B +4.3, SenseNova-SI-1.3 +1.3, and SOTA object-size 79.2 / room-size 75.7 in Table 1) cannot be interpreted as evidence of general spatial reasoning capability; they may reflect reward engineering toward the target benchmark. Please disclose the source and train/eval split of the 8,000 samples, and either re-run Stage 3 on a dataset disjoint from VSI-Bench or present the Stage 1&2 results as the primary evidence for generalization.","section":"Experiments, Data; Appendix A.3"},{"comment":"The six reasoning frames (indices 1, 10, 12, 15, 21, 28) are selected based on 'observ[ing that], when answering questions from our primary benchmark, VSI-Bench, the trained model relied more frequently on certain frames than on others.' This is hyperparameter selection on the evaluation set, and it affects all results from Stages 2 and 3, including the Stage 1&2 score of 68.6 in Table 3. The frame-selection procedure should be justified on a validation split or shown to be stable across held-out settings; otherwise even the non-RL gains are partially confounded by evaluation-set tuning.","section":"Appendix B.2, Table S3"},{"comment":"The OOD evaluation is performed only for the Stage 1&2 SenseNova-SI-1.3 model, and Table 3 shows the Stage 3 gain over Stage 1&2 is just 0.3 points on average (68.6 to 68.9). The magnitude of the Stage 3 improvement is small relative to the task-specific jumps (e.g., room size +3.2 points, object size +1.4 points). Because Stage 3 is the source of the SOTA room-size claim, any unaddressed leakage in Stage 3 directly undermines the paper's central stated contribution.","section":"Table 2 and Table 3"}],"minor_comments":[{"comment":"The term 'modality-agnostic' is used although the paper demonstrates only two spatial modalities (3D reconstruction and 3D bounding boxes). The unified token interface may be extensible, but 'modality-agnostic' should either be defined more precisely or softened to 'modality-extensible' to avoid overclaiming.","section":"Abstract and Section 1"},{"comment":"The caption says 'after stage 1, all spatial reasoning is performed using the learned latent tokens without requiring additional model parameters,' but Stage 1 itself trains projection and decoder layers that are additional parameters, and these components appear necessary for the representation-alignment losses. Please clarify which parameters remain at inference time.","section":"Figure 2 caption"},{"comment":"The row label 'Qwen3-VL-8B + SenseNova-SI' is confusing because the baseline is the SenseNova-SI-1.3 checkpoint built on Qwen3-VL-8B; consider renaming the rows to identify the exact checkpoint and training stage.","section":"Table 2"},{"comment":"There are two entries for 'Zhang et al. 2026' with different papers (Ablate-to-Validate and SSR); the in-text citations need to be disambiguated (e.g., Zhang, Bigverdi, and Krishna 2026 vs. Zhang, Xia, Wang, et al. 2026).","section":"References"},{"comment":"Equation (S16) defines the MRA reward with confidence intervals C={0.5,0.55,...,0.95}, but the text says 'for a set of confidence intervals C={0.5,0.55,...,0.95}' and the denominator 10 assumes exactly 10 thresholds; this is consistent, but the notation could be made explicit that the average is over 10 values.","section":"Appendix A.3"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claim is compromised by the undisclosed Stage 3 training data combined with a VSI-Bench-defined reward and VSI-Bench-informed frame selection. I would expect the editor to insist on full data/split disclosure and a re-analysis that separates benchmark-optimization from generalization before this paper can be accepted. The Stage 1&2 results and the attention-masking evidence suggest the core idea is worth salvaging, which is why I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper is worth engaging with, but the headline numbers should not be believed until the authors disclose the stage-3 data split. The core idea—continuous latent tokens that carry 3D scene geometry and object bounding boxes into a VLM's chain of thought—is a genuine extension of the CoVT/Mull-Tokens line, and the stage 1&2 evidence suggests it does something real. But the final stage optimizes the VSI-Bench metric directly, and the paper never names the training distribution for the 8,000 RL samples. That makes the +4.3/+1.3 improvements and the SOTA object/room-size claims uninterpretable as held-out results.\n\nWhat is new and good: the three-stage recipe is clean, with representation learning, then SFT, then RL. The attention-masking experiment is the best causal evidence—blocking attention to the spatial tokens drops object size by 15.3 points—and the authors' argument for why the token-replacement test is inconclusive in multi-image settings is fair. The data-scaling ablation is the right control: fine-tuning on VICA-322K alone hurts VSI-Bench, while the token pipeline improves it, so the gains are not simply extra data. The decoded 3D visualizations also show the tokens carry geometric structure, which is more than most latent-token papers do.\n\nWhere it gets soft: the stage-3 reward is the VSI-Bench metric (Appendix A.3), the 8,000 samples are unnamed, and the authors themselves exclude the stage-3 model from OOD benchmarks because the reward is VSI-Bench-specific. That is an explicit admission that the final model is specialized to VSI-Bench. The reasoning-frame indices were selected after looking at VSI-Bench performance, and there are no error bars anywhere. The stage 1&2 improvements are modest (+1.0 on SenseNova, mixed on OOD with a 1.5-point drop on CV-Bench), so the strong empirical claims rest almost entirely on the stage-3 numbers. The stress-test note is right on the central point: without a disclosed train/eval split, the RL gains could reflect direct reward chasing rather than a general spatial reasoning ability.\n\nWho this is for: researchers working on spatial grounding in VLMs, and anyone designing benchmark evaluations for RL-tuned reasoning. The stage 1&2 mechanism and the attention-masking diagnostic are worth building on. But I would not cite the headline results in my own work until the split is clarified.\n\nRecommendation: send to peer review—a serious referee can resolve this—but the paper should not be accepted in current form. The authors need to disclose the 8,000-sample source, evaluate the stage-3 model on a held-out split, and ideally replace the direct benchmark-metric reward with something more general.","headline":"Plausible extension of latent-token distillation to 3D spatial reasoning, but the headline stage-3 RL numbers are not interpretable until the authors disclose where the 8,000 training samples come from.","tokens_in":17231,"tokens_out":2687,"would_cite":false,"duration_ms":25898,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Space Tokens claim that continuous latent tokens distilled from scene geometry and 3D object boxes improve VLM spatial reasoning without inference-time modules.","keywords":["Space Tokens","continuous latent tokens","spatial reasoning","vision-language models","3D scene reconstruction","3D bounding boxes","chain-of-thought","GRPO"],"falsifier":"Check the 8,000 reinforcement-learning training prompts for overlap with VSI-Bench questions; if any overlap exists, or if the stage-3 gain disappears on a held-out spatial benchmark whose questions never entered the reward design, then the reported 1.3-point improvement reflects reward optimization rather than a general spatial-reasoning gain.","tokens_in":16056,"feed_emoji":"📐","tokens_out":12328,"duration_ms":104400,"temperature":0.7,"pith_summary":"Space Tokens is a training-only method for giving vision-language models explicit spatial knowledge. The paper reserves a small set of vocabulary tokens, trains their hidden states to encode scene-level 3D geometry distilled from VGGT-$\\Omega$ and object-centric 3D bounding boxes, and has the model emit these tokens inside a chain-of-thought trace before answering. No extra spatial encoder or architectural change is needed at inference. On VSI-Bench the method raises Qwen3-VL-8B by 4.3 points and SenseNova-SI-1.3 by 1.3 points, reaching 79.2 on object size and 75.7 on room size, and the tokens can be decoded into explicit 3D reconstructions for verification. The case matters because spatial reasoning is a bottleneck for embodied agents, and most strong alternatives pay for it with larger models or external geometry modules.","feed_headline":"15.8-point gain: spatial tokens improve VLM room-size estimates","feed_subtitle":"No new encoder at inference: the model itself carries 3D geometry through its reasoning chain.","key_machinery":"The carrying object is the Space Token: a reserved vocabulary token whose final-layer hidden state is treated as a continuous spatial representation rather than decoded as discrete text. Scene-level 3D tokens are trained with a cosine-similarity loss against VGGT-$\\Omega$'s latent features and with reconstruction losses over camera parameters, depth maps, and point maps; object-level tokens predict 12D 3D bounding boxes (center, dimensions, 6D rotation) through Hungarian matching and Smooth L1 loss. The three-stage pipeline first learns these representations, then teaches the VLM to reason with them under teacher forcing, then refines the reasoning with GRPO. The same tokens are emitted in a structured <think> block that ties each image to its spatial token blocks.","core_discovery":"The paper's central claim is that a VLM's spatial reasoning can be improved by making continuous spatial information part of the model's own autoregressive output, rather than by adding geometry modules at inference. During generation, reserved 'spatial token' positions are read from the final-layer hidden states and supervised in three stages: first to align with VGGT-$\\Omega$'s latent scene representation and to reconstruct cameras, depth, and point maps, plus 3D bounding boxes for objects; then via supervised fine-tuning in which the model answers questions while the tokens appear in a per-view chain-of-thought; finally via GRPO reinforcement learning with a reward built from the VSI-Bench metric. The paper reports consistent gains on VSI-Bench, with the largest improvements on room-size and absolute-distance tasks, and shows that both scene-level and object-level tokens contribute. It also reports that masking attention to the spatial tokens sharply hurts performance, which the authors take as evidence that the latent representations are actually used.","pith_inferences":["A matched baseline that uses the same chain-of-thought prompt and SFT/RL data but replaces the spatial-token slots with ordinary text descriptions of geometry would isolate the token representation; the paper's no-token baseline changes both the representation and the prompt.","Because the token interface is modality-agnostic, the same three-stage recipe could be applied to other continuous sensors, such as depth from stereo, optical flow, audio localization, or tactile maps, and it remains open whether each modality needs its own teacher model.","The paper's teacher-forced chain-of-thought means the model never decides when to emit the spatial-token trace; letting the model generate that trace freely at inference could either amplify the gains or reveal that the trace is only useful when forced."],"forward_implications":["Any autoregressive VLM can use the same recipe: reserve vocabulary tokens, train their hidden states on geometry objectives, and keep the architecture untouched.","Tasks that need global scene geometry benefit most: on VSI-Bench, room-size accuracy rises by 15.8 points on Qwen3-VL-8B and 9.5 points on SenseNova-SI-1.3.","The tokens are causally used, not decorative: masking attention to them drops object-size accuracy from 79.0 to 63.7 on VSI-Bench.","The representation is verifiable: the learned token states can be decoded into 3D point maps and boxes, allowing inspection of what geometry the model encodes.","The gains are not simply data scaling: adding the same VICA-322K data without the spatial-token losses decreases VSI-Bench score from 67.6 to 67.1, while the full method raises it."],"supporting_citations":[{"why":"Supplies VSI-Bench, the benchmark whose tasks and metric define both evaluation and the stage-3 reward.","marker":"Yang et al. 2025"},{"why":"Supplies VGGT-Omega, the teacher model whose latent features and reconstruction targets the 3D tokens are aligned to.","marker":"Wang et al. 2025a"},{"why":"Establishes the continuous latent-token training recipe that Space Tokens extends to 3D geometry.","marker":"Qin et al. 2025"},{"why":"Provides the GRPO algorithm used in the third-stage refinement.","marker":"Shao et al. 2024"},{"why":"Provides the VICA-322K video instruction dataset used in stages 1 and 2.","marker":"Feng 2025"},{"why":"Provides the SenseNova-SI baseline checkpoint that the method is applied to.","marker":"Cai et al. 2026"},{"why":"Provides the LoRA parameter-efficient fine-tuning method used throughout training.","marker":"Hu et al. 2021"}],"fun_headline_variants":["Spatial tokens: VLM geometry without extra encoders","Teach VLMs to think in 3D with latent tokens","Space Tokens: 3D geometry inside the VLM's chain","No new modules: VLM learns 3D via its own tokens","VLM spatial reasoning boosted by hidden spatial tokens"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 8,000 samples used in the third reinforcement-learning stage are independent of the VSI-Bench evaluation, even though the reward is designed from the VSI-Bench metric and the dataset is not named.","fun_headline_variants_meta":{"raw":{"variants":["Spatial tokens: VLM geometry without extra encoders","Teach VLMs to think in 3D with latent tokens","Space Tokens: 3D geometry inside the VLM's chain","No new modules: VLM learns 3D via its own tokens","VLM spatial reasoning boosted by hidden spatial tokens"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000291,"raw_usage":{"total_tokens":1718,"prompt_tokens":982,"completion_tokens":736,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":649}},"tokens_in":598,"tokens_out":736,"duration_ms":6777,"temperature":1.0,"reasoning_tokens":649,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:10:54.720267+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check the 8,000 reinforcement-learning training prompts for overlap with VSI-Bench questions; if any overlap exists, or if the stage-3 gain disappears on a held-out spatial benchmark whose questions never entered the reward design, then the reported 1.3-point improvement reflects reward optimization rather than a general spatial-reasoning gain.","supporting_citations":[],"review_version":1}