{"id":"2b1f4dcb-24e7-4590-9f47-13edc406c4cd","arxiv_id":"2507.20529","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A two-stage training pipeline that adds region location tokens and step-by-step text reasoning improves vision-language model performance on spatial reasoning benchmarks.","lead":"This paper trains a vision-language model to first mark useful objects in an image, then reason out loud, before answering spatial questions about distance and direction. The method improves average accuracy on spatial reasoning benchmarks without depth sensors or segmentation masks, which could help robots and assistants handle space-aware questions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparability under the LLM-judge protocol is the decisive uncertainty; Table 1 does not state whether prior-model numbers were re-scored with the same DeepSeek judge and 0.5 threshold.","rationale":"The reader's weakest assumption is exactly the protocol comparability of baseline numbers. My analysis confirms this is the most load-bearing concern because the paper's metric is an LLM judge with a chosen threshold, and Table 1 does not state protocol provenance. A judge inconsistency can change several points, comparable to the reported margins over baselines. The paper does offer some independent support: ablations are internally consistent, and the method description is plausible; there is no internal contradiction. I do not see evidence that the method is fraudulent or that the claimed mechanism cannot work. The concern is empirical verification, not soundness. The verdict should remain CONDITIONAL: the authors should disclose protocol provenance, release outputs/code, and ideally run a second judge. This is not a rejection because the concern is addressable and the method may well hold up. I agree with the reader's assessment, though I would emphasize that the judge-based metric, not just dataset overlap, is the primary risk.","tokens_in":10820,"tokens_out":1517,"duration_ms":13894,"concrete_test":"Re-run all baselines in Table 1 using the exact stated protocol: feed each model's raw outputs to DeepSeek-V3-0324 with the 0.5 threshold and the same prompt, and compare against VSR, SpatialRGPT-QUAL, WhatsUP, BLINK-spatial ground truths. If the re-scored baseline averages shift by more than ~2 points per benchmark or change the rank ordering, the reported advantage is partly evaluator artifact. A second check would be to use a different judge (e.g., GPT-4o or exact-match) on both SpatialVTS and baselines to determine whether the margins persist.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that SpatialVTS significantly improves average spatial reasoning without extra inputs. Table 1 reports accuracy for Qwen2-VL, Qwen2.5-VL, VPT, SpatialRGPT, and SpaceThinker, but the paper never states whether these numbers were re-run under the paper's evaluation protocol (DeepSeek-V3-0324 judge, score > 0.5; Section 4.1 Metrics), or quoted from their source papers. The metric is not simple string match: it is an LLM-based judge assigning a score in [0,1] with a 0.5 acceptance threshold. If the baselines were scored by different judges, thresholds, or exact-match criteria, the reported deltas in Table 1 (e.g., SpatialVTS averages 82.03 vs. SpaceThinker's 57.55) may largely reflect evaluator differences, not spatial reasoning gains. The paper's own ablations (Table 2) show the text-thinking component is worth only 3-7% on quantitative benchmarks, so a judge-protocol gap of several points would undermine the headline 'significantly improved.' Additional risks: no error bars, no code/data release (stated in Section 4.1), and no indication whether own-answer advantage (model trained on the same judge style) biases the LLM judge. These are resolvable gaps, but they sit directly on the central claim, so the paper should be CONDITIONAL until clarified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SpatialVTS, a two-stage training pipeline for enhancing spatial reasoning in vision-language models (VLMs). In the spatial visual thinking stage, the model is trained to emit discrete region-location tokens identifying both explicitly mentioned and potentially informative objects. In the spatial textual thinking stage, the model receives the cropped candidate regions as visual cues and is trained to produce step-by-step rationales before the final answer. The authors reconstruct a training set from the SpatialRGPT and VPT datasets by manually correcting labels, replacing mask/depth placeholders with region labels, and generating CoT rationales via a 'seeking the cause by grasping the result' strategy that takes the ground-truth answer as input. They fine-tune Qwen2-VL-7B and report results on VSR, WhatsUP, BLINK-spatial, SRGPT-QUAL, and quantitative benchmarks, claiming a significant average improvement over existing open VLMs and spatial-reasoning-specific models without using masks or depth.","tokens_in":11104,"tokens_out":3681,"duration_ms":41096,"significance":"If the empirical results hold, the paper provides a practical and scalable recipe for improving spatial reasoning in open VLMs using only RGB images and text. The combination of discrete region tokens with CoT rationales is a sensible synthesis of existing ideas, and the manually corrected dataset with rationales is a potentially valuable community resource. The paper's explicit focus on 'potential objects' beyond those named in the question is a useful insight, and the ablations on text-thinking and data scale help isolate the contributions. The design also avoids the need for depth sensors or segmentation masks, which broadens applicability. However, the central claim of significance depends on the comparability of the baseline numbers and on the statistical reliability of the reported advantages, both of which are currently not established.","major_comments":[{"comment":"The evaluation protocol uses an LLM judge (DeepSeek-V3-0324) that scores each answer in [0,1] with acceptance above 0.5. The manuscript never states whether the baseline numbers in Table 1 and Fig. 6 were obtained by re-running all baselines under this exact protocol or are quoted from the original publications. This is load-bearing: different judges, different thresholds, or exact-match criteria can shift scores by several points, and the reported gaps (e.g., 82.03 vs. 57.55 average) are much larger than the ablation gains in Table 2. The authors must re-evaluate every baseline with the same judge, the same prompt template, and the same threshold, and disclose the judge prompt and version. They should also report a small human-validation subset to show the judge is not systematically biased toward SpatialVTS's answer style, which is a real risk because the model was trained on rationales that resemble the judge's expected output format.","section":"Section 4.1, Metrics; Table 1"},{"comment":"No error bars, confidence intervals, or significance tests are reported anywhere. The ablation study in Table 2 shows text thinking contributes about 3-7 points on the quantitative benchmarks, and Table 3 shows an 8-point difference from data scale on VSR. Without variance estimates, statements such as 'significantly improved' and 'almost on par' are not supported. Please report standard errors or bootstrap intervals across benchmark items, and if the LLM judge is stochastic, repeat scoring multiple times and report the judge's variance. This is necessary to assess whether the reported margins exceed noise, especially given the modest sizes of some of the qualitative benchmarks.","section":"Tables 1-3 and Fig. 6"},{"comment":"The rationale generation strategy 'seeking the cause by grasping the result' feeds the ground-truth answer into the large model when constructing the reasoning process. This means the rationales are post-hoc justifications constrained to end at the correct answer, not independent derivations. The paper should acknowledge this limitation explicitly and provide evidence about the faithfulness of the rationales, for example a human evaluation of whether the rationales correspond to the visual evidence actually used by the model. Without this, it is unclear whether the improved accuracy comes from genuine spatial reasoning or from the model learning to produce confident-sounding text that the LLM judge accepts.","section":"Section 3.2, Fig. 4"},{"comment":"The manuscript states 'We promise to publish our model and datasets after the review is completed.' The manually corrected dataset and the region/CoT annotations are central to the method and to verifying the claimed improvements. Since the paper is currently under review, this is acceptable as a condition, but the final version should commit to a concrete release plan (e.g., model weights, dataset, and evaluation scripts) and should additionally specify which public VLM was used to generate the reasoning processes and which manual annotation guidelines were followed.","section":"Section 4.1 and full paper"}],"minor_comments":[{"comment":"The table formatting is corrupted: entries such as '79.0058.45' and '69.1785.3175.96' should be split into separate numbers (e.g., '79.00 58.45' and '69.17 85.31 75.96'). Please also verify the row/column alignment for VPT and SpatialVTS.","section":"Table 1"},{"comment":"The heading contains a typo: 'Benckmarks' should be 'Benchmarks'.","section":"Section 2.1 heading"},{"comment":"The corrected answer example contains a typo: 'Inorrect' should be 'Incorrect'.","section":"Fig. 5"},{"comment":"'SRPGTBench' appears to be a typo for 'SRGPT-Bench'.","section":"Section 4.2"},{"comment":"The conclusion refers to the 'Visual Textual Thinking' phase, whereas the body consistently uses 'Spatial Textual Thinking'; please align the terminology.","section":"Section 5"},{"comment":"The reference [43] for SpaceThinker is listed as 'Vqasynth' by Remyx AI, which does not appear to be the correct source for SpaceThinker. Please cite the actual SpaceThinker publication.","section":"Section 3.3 and References"},{"comment":"The grid size k is a free parameter set to k=8 without sensitivity analysis. A brief ablation or justification for this choice would strengthen the paper.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The core concern is baseline comparability: if the numbers in Table 1 and Fig. 6 are not produced under the same LLM-judge protocol, the reported average improvements could be partly an artifact of the evaluator. The authors should be asked to provide the judge prompt, re-run all baselines, and release the model and dataset as promised. The incorrect SpaceThinker reference should also be fixed; it suggests a lack of copyediting on a submission that otherwise makes a credible contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The method is a sensible combination of VPT-style region tokens and CoT reasoning, and the data-cleaning work is real. What's actually new: most region-token methods focus on objects named in the question, while SpatialVTS adds annotations of 'potential targets' that provide scale or reference cues, and pairs that with a textual reasoning stage. The dataset correction looks legitimate—Figure 5 shows obvious annotation errors in SpatialRGPT's templated data. The ablation studies are informative: text thinking adds 3–7%, and more data helps (Table 3). Those internal comparisons don't depend on the external judge and they support the method's components.\n\nThe soft spots sit directly on the central claim. Table 1 does not state whether Qwen2-VL, VPT, SpatialRGPT, and SpaceThinker numbers were re-scored with the paper's DeepSeek judge at the 0.5 threshold or quoted from prior papers. The huge jump on SRGPT-QUAL (95.38 vs 39.32 for SpatialRGPT) smells like a protocol mismatch or a judge biased toward the model's own answer style. If the baselines were scored by different judges or exact-match criteria, the 'significant improvement' could be an artifact of the evaluator, not genuine spatial reasoning gains. This is load-bearing and must be clarified. Also missing: error bars, code/data release, and a clear statement on whether the judge itself was used in training. The promise to publish after review is honest but leaves the claim unverifiable now.\n\nMinor issues: several references are wrong ([38] is a graph networks paper, not the SpatialRGPT dataset; [43] is VQASynth, not SpaceThinker). And the 'seeking the cause by grasping the result' strategy means rationales are generated with the ground-truth answer as input. That's not circular for external benchmarks, but it's worth a sentence acknowledging that the CoT is not an independent derivation.\n\nThis paper deserves a serious referee—the method is well-motivated and the ablations are solid. I'd ask the authors to release evaluation code and re-score baselines under the same judge protocol before acceptance. If they can do that, this is a useful practical contribution; if not, the headline is unverified. I'd bring it to a reading group as a case study in VLM evaluation pitfalls.","headline":"A plausible training recipe and a real data-cleaning effort, but the headline gains over baselines are unverified because the paper never documents whether baselines were scored under the same LLM-judge protocol.","tokens_in":11621,"tokens_out":2735,"would_cite":false,"duration_ms":29786,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SpatialVTS trains a vision-language model to first locate helpful regions, then reason over them in text, lifting average spatial-benchmark accuracy to 82.03 percent without depth or mask inputs.","keywords":["spatial reasoning","vision language models","region tokens","chain-of-thought","dataset reconstruction","visual thinking","textual thinking","spatial VQA"],"falsifier":"Re-run all baselines under the exact protocol, using the same judge, same threshold, same answer prompt, and same $\\delta \\le 2$ acceptance, then compare the averages; if the gap shrinks or reverses, the reported advantage is an artifact of scoring rather than spatial reasoning. A judge-free check would replace the LLM grader with exact-match or human scoring on a random subset.","tokens_in":1567,"feed_emoji":"🧭","tokens_out":1904,"duration_ms":68530,"temperature":0.7,"pith_summary":"The paper claims that spatial reasoning in vision-language models can be substantially improved without feeding in depth maps or segmentation masks: instead, the model is trained to think twice, first by emitting location tokens for both obvious and hidden reference targets, then by producing step-by-step textual reasoning before the answer. The authors build this capability by reconstructing the SpatialRGPT and VPT training data, replacing mask and depth placeholders with plain region labels, manually fixing incorrect labels, and generating rationales. On five qualitative benchmarks the trained model averages 82.03 percent, above general VLMs and dedicated spatial models, and on quantitative distance benchmarks it lands near SpatialRGPT, which trains with extra depth and mask inputs. A sympathetic reading is that explicit visual cue discovery plus textual reasoning over those cues is what carries the gain.","feed_headline":"Two-stage thinking lifts spatial reasoning accuracy to 82%","feed_subtitle":"Training a VLM to name helpful regions, then reason over them, beats depth-free spatial baselines.","key_machinery":"The load-bearing mechanism is a two-phase generation loop. First, spatial visual thinking: the model divides the image into an 8x8 grid and outputs special tokens such as <x_3><y_6> for the top-left and bottom-right cells of each target region, covering both objects named in the question and potential reference objects. Second, spatial textual thinking: the selected regions are cropped, passed through the same vision encoder, and the model must reason step by step from those visual cues before giving the final answer. Around this loop, the paper reconstructs the training data so that region labels replace mask and depth placeholders, incorrect labels are manually corrected, and a long reasoning process is generated by a 'seeking the cause by grasping the result' strategy, which feeds the problem, visual cues, and answer into a public VLM to produce a correct rationale.","core_discovery":"The central claim is that a VLM trained with SpatialVTS reaches better spatial understanding than existing open models using only RGB images and text. During spatial visual thinking, the model emits discrete region tokens (cell indices on an 8x8 grid) for the objects in the question and for additional potential reference objects; during spatial textual thinking, those crops are re-encoded and the model must produce a rationale and then the answer. The paper reports an average score of 82.03 across CoT, VSR, SRGPT-QUAL, WhatsUP, and BLINK-Spatial, compared with 74.47 for VPT, 67.26 for Qwen2.5-VL, 59.00 for SpatialRGPT, and 57.55 for SpaceThinker, and near-parity with SpatialRGPT on quantitative benchmarks. It further claims the reconstructed dataset and the chain-of-thought component are responsible for the gains: removing the textual thinking drops accuracy by 3 to 7 percentage points, and a 250k-sample subset already rivals VPT.","pith_inferences":["If the gain really comes from naming potential reference objects, then a testable extension is to evaluate whether the model's region tokens are themselves interpretable: removing the 'potential' targets and keeping only question-mentioned objects should close most of the gap.","The reported numbers may depend on the DeepSeek-V3-0324 judge accepting verbose, well-structured answers; a judge-free or human-scored replication would tell whether the CoT format inflates qualitative scores.","The near-parity with SpatialRGPT on quantitative tasks, despite no depth, suggests that reference-object scales can substitute for metric depth in many indoor scenes; applying the method to outdoor or aerial images, where reference scales are rarer, would stress-test the mechanism.","The manual dataset correction step is expensive; a follow-up could automate label correction and rationale generation while keeping the same input format, testing whether the gains survive without human cleaning."],"forward_implications":["Spatial reasoning can be improved with only RGB input, so depth sensors or mask annotations are not prerequisites for VLM spatial competence.","The explicit emission of potential-reference-target tokens gives a mechanism by which a model can use everyday reference scales, such as a building's height or a TV's width, to answer distance questions.","Training with textual rationales is directly measurable: adding CoT raises performance by about 3 points on SRGPT-QUAN at $\\delta \\le 1.25$ and 7 points on Q-Spatial++ at $\\delta \\le 2$.","A much smaller reconstructed dataset of 250k samples is enough to match VPT's level, suggesting that data quality and input format may matter more than raw training scale.","The same two-phase recipe could transfer to other perception tasks that need measuring or comparing objects, not just spatial VQA."],"supporting_citations":[{"why":"Supplies the region-based spatial reasoning framework and the SpatialRGPT-Bench baseline that SpatialVTS must beat.","marker":"[1]"},{"why":"Supplies the region token representation and the 340k-sample training data drawn from VPT.","marker":"[31]"},{"why":"Cited source of the 100k sampled SpatialRGPT training data.","marker":"[38]"},{"why":"The Qwen2-VL base model that all SpatialVTS training starts from.","marker":"[9]"},{"why":"The DeepSeek-V3-0324 judge used to score qualitative answers at the greater-than-0.5 threshold.","marker":"[42]"},{"why":"Defines the delta<=2 metric and the quantitative benchmark used for distance estimation.","marker":"[41]"},{"why":"The VSR benchmark covering 66 spatial relations, one of the five qualitative evaluation sets.","marker":"[18]"},{"why":"SpaceThinker, an open-source spatial reasoning baseline compared in the main results.","marker":"[43]"}],"fun_headline_variants":["SpatialVTS: name regions, then think textually to hit 82%","RGB-only VLM spatial reasoning reaches 82% via two-phase thinking","Teaching VLMs to name regions and reason lifts spatial accuracy to 82%","SpatialVTS: visual+textual thinking lifts spatial accuracy to 82%"],"cache_read_input_tokens":13824,"weakest_assumption_plain":"The load-bearing premise is that the baseline scores in Table 1 and Fig. 6 were obtained with the same evaluation protocol as SpatialVTS, namely the DeepSeek-V3-0324 judge at a 0.5 threshold for qualitative tasks and the $\\delta \\le 2$ rule for quantitative ones; the paper does not state that the baselines were re-run under this protocol.","fun_headline_variants_meta":{"raw":{"variants":["SpatialVTS: name regions, then think textually to hit 82%","RGB-only VLM spatial reasoning reaches 82% via two-phase thinking","Teaching VLMs to name regions and reason lifts spatial accuracy to 82%","SpatialVTS: visual+textual thinking lifts spatial accuracy to 82%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001851,"raw_usage":{"total_tokens":7283,"prompt_tokens":967,"completion_tokens":6316,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":6228}},"tokens_in":583,"tokens_out":6316,"duration_ms":43635,"temperature":1.0,"reasoning_tokens":6228,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:42:02.144073+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run all baselines under the exact protocol, using the same judge, same threshold, same answer prompt, and same $\\delta \\le 2$ acceptance, then compare the averages; if the gap shrinks or reverses, the reported advantage is an artifact of scoring rather than spatial reasoning. A judge-free check would replace the LLM grader with exact-match or human scoring on a random subset.","supporting_citations":[{"cited_title":"Reasoning paths with reference objects elicit quantitative spatial reasoning in large vision-language models, 2024","cited_arxiv_id":null,"evidence_quote":"Defines the delta<=2 metric and the quantitative benchmark used for distance estimation."},{"cited_title":"Vqasynth, 2023","cited_arxiv_id":null,"evidence_quote":"SpaceThinker, an open-source spatial reasoning baseline compared in the main results."}],"review_version":2}