Artificial Intelligence · arXiv:2610.28354 v1 · submitted 20 Oct 2026 · 61 references
S-Agent 2.0: Spatial Reasoning with a General Coding Agent
The paper says
“Spatial reasoning asks a model to work out where objects are in 3D space, how they relate to one another, and how they move.”
more of the abstract
Agentic VLMs have recently brought large improvements on such tasks by letting the model actively gather evidence with 3D perception tools instead of answering in a single pass. Even so, on many questions the tools return the right evidence and the model still does not arrive at the correct answer: the gap between model and tools, where raw readings become usable conclusions, is left unfilled. We present S-Agent 2.0, a spatial reasoning agent that equips an off-the-shelf general coding agent with a command-line perception toolkit, and we design the interface around one principle: help the VLM close the evidence loop: acquire scene evidence, remember it in a persistent scene memory, and interpret each reading in plain spatial language. This design keeps the full flexibility of a coding agent, and every reading it acts on stays grounded in the tools' geometry. On MMSI-Bench and ReVSI, the toolkit improves a strong open-source model, Qwen3.8-27B, by +13.9 and +20.5 points over the same agent without tools, and a strong closed-source model, Gemini-3.8-Flash, by +11.2 and +9.6 points. With the open backbone S-Agent 2.0 reaches 66.5 on MMSI-Bench and 82.3 on ReVSI, ahead of the specialized system SpatialClaw on the same backbone (62.6 and 74.5), and with Gemini-3.8-Flash it reaches 74.1 and 90.5 against its 67.0 and 79.7. Both tool gains shrink as the backbone grows more capable, but SpatialClaw's shrinks far more on ReVSI, from +13.2 to +2.7, while ours stays at +9.6. S-Agent 2.0 also works across different general coding agent harnesses, such as Codex and Claude Code.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 21 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A solid interface-design result for agentic spatial reasoning; the unvalidated fused reconstruction is the gap that needs fixing, not the relative-reasoning results. This line is wrong
2 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 61 references say.
The paper says, at §2.2-2.3, Eq. (2); Table 2; Appendix B; Figures 5-10
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
No accuracy evaluation of the fused reconstruction P is reported anywhere in the paper, although every readout (distance, side, elevation, motion, compass) is computed in closed form from P, and the interface instructs the model to copy readout conclusions verbatim ('copy the conclusion, do not re-derive it from the pictures', Figures 5-10).
more of objection 1
Appendix B does disclose that reply units are 'metres on the reconstruction's own metric scale (uncalibrated)' unless the agent passes --known D, but the paper never reports how often that flag is used and never analyzes the consequences of the disclosure. In the weak reading ('derived from the tools' geometry') the abstract's claim that every reading 'stays grounded in the tools' geometry' is true by construction; in the reading the surrounding text invites, namely that the measured numbers are trustworthy, the claim is unsupported. The paper's own data provide a probe: on ReVSI (Table 2), absolute distance is the category where S-Agent 2.0 trails SpatialClaw on both backbones (-2.8 Qwen, -1.7 Gemini), and Gemini object size is also behind (-0.5); since §3.4 and Appendix E state that both systems use the same perception backends, the deficit is attributable to how the interface handles metric questions rather than to backend choice. I ask the authors to add a reconstruction-accuracy evaluation (for example, comparing toolkit distance, size, and elevation readouts against ReVSI's metric ground truth, or on a small set of scenes with known distances) and to run the scale-calibration check with the already-implemented --known D flag, to determine whether the absolute-distance deficit is a global scale bias, which calibration would fix, or geometric distortion and registration error, which it would not.
The paper says, at §3.1-3.2; Tables 1-6, 9
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The paper reports no evaluation protocol details: sampling temperature, number of runs per condition, step budgets, and compute cost are absent, and no variance or repeated-run information accompanies any quantitative claim. The headline margins are large (+3.9 to +10.8 points), but several statements rest on small differences whose stability the reader cannot assess: the MMSI O-R margin is +1.2 (Table …
more of objection 2
… 1), the ReVSI object-count margin is +0.4 (Table 2), the Gemini object-size margin is -0.5 (Table 2), and the SpatialClaw-plus-readouts transplant is +1.0/+0.3 (Table 9). I ask that the protocol be stated explicitly and that repeated-run variance (or a determinism argument) be reported for at least the main comparisons, so that category-level statements such as 'the one deficit is absolute distance' can be checked against run-to-run noise.
6 minor objections
The paper says, at §3.2, Table 2
Pith says: minor objection 1 of 6
The sentence 'the one deficit is absolute distance' is inaccurate for the Gemini-3.8-Flash backbone, where object size is also behind SpatialClaw (86.1 vs 86.6); please correct the statement and indicate whether differences of this size are stable across runs.
The paper says, at Abstract; §3.3, Table 5
Pith says: minor objection 2 of 6
The claim that on ReVSI SpatialClaw's tool gain 'shrinks far more' (from +13.2 to +2.7) while 'ours stays at +9.6' is metric-sensitive: the absolute drops in tool gain from Qwen to Gemini are nearly identical (SpatialClaw -10.5, S-Agent 2.0 -10.9), whereas the relative drops differ sharply (about 80% vs 53%); please report both measures so the comparison is unambiguous.
show 4 more
The paper says, at §3.4; Appendix E
Pith says: minor objection 3 of 6
The main text says 'Seven further items ... are in Appendix E', but Appendix E presents six items (Figures 5-10); the counts should be aligned.
The paper says, at Appendix D, Table 9
Pith says: minor objection 4 of 6
The readout-transplant experiment is one of the paper's cleanest decomposition results, since interpreted readouts alone add only +1.0 ReVSI / +0.3 MMSI to SpatialClaw; it deserves discussion in the main text next to Table 6, because it shows that the full evidence loop, rather than the readout format alone, generates the gains.
The paper says, at §3.4, §1
Pith says: minor objection 5 of 6
The failure analysis of SpatialClaw trajectories that motivates the design ('acquisition is rarely what breaks') is reported without methodology; a sentence on the corpus size, sampling procedure, and how failures were coded would let the reader gauge the analysis's scope.
The paper says, at Table 4
Pith says: minor objection 6 of 6
The cross-harness result is noisy on ReVSI (OpenHands 82.3, Codex 76.6, Claude Code 79.1); please state whether these differences are consistent across items, which is relevant to the claim that the method works across coding-agent harnesses.
What would settle it
Re-run the recon pipeline on ~50 sampled MMSI-Bench and ReVSI scenes, annotate ground-truth distances between several segmentable object pairs per scene (ReVSI items with metric answers already provide a subset), and compute the error distribution of the distance/depth/side readouts against this ground truth. Then apply a single per-scene scale calibration (--known D) and recompute Table 2's absolute-distance and object-size columns. If calibration closes the gap to SpatialClaw, the deficit is the disclosed uncalibrated scale and is routine to fix; if the gap persists, P carries structural distortion and the unverified reconstruction propagates confidently wrong metric answers, undercutting the groundedness claim.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 21 Oct 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Read this one. It is a solid systems paper making a real argument: for agentic spatial reasoning, the interface between model and tools matters more than the perception stack. The evidence: same-backbone comparisons where S-Agent 2.0 beats the specialized SpatialClaw on five benchmarks, clean component ablations, and a genuinely useful control (Appendix D) where transplanting only the readouts into SpatialClaw's kernel recovers just +1.0/+0.3 of the gap, showing the scene memory and model-perception route carry most of the effect. The authors also report honestly: two remaining failure classes documented, the absolute-distance deficit visible in their own Table 2, uncalibrated scale disclosed in Appendix B.
The real novelty is design, not math: persistent scene memory held by the environment, conclusion-attached readouts, and the add_instance/drop route that puts the model's own perception on the same geometric footing. The ablations show all three matter, with the model-perception interface the most expensive to remove (-8.8/-5.9).
Soft spots, in proportion. The biggest: the fused point cloud P is never validated. Every readout is computed closed-form from P, which fuses three frozen models, and there is no reconstruction-error measurement anywhere. The readout notes instruct the agent to copy conclusions rather than re-derive, a sensible fix for SpatialClaw's sign-flipping failure mode, but it means any bias inside P propagates silently into answers. The paper's own data are the probe: absolute distance is the one category where S-Agent 2.0 trails SpatialClaw, by exactly the kind of margin a scale error would produce. That does not undermine the relative-direction and route-planning results, which are scale-invariant, but the claim that every reading is grounded in the tools' geometry is asserted, not measured, for the metric tier. A calibration check using the existing --known D flag on scenes with known ground-truth distances would settle it; I would want that before trusting the metric readouts.
Second, there is no run-to-run variance anywhere. Agent trajectories are stochastic, and while the headline margins are large, the +0.3 point Appendix D control and the ReVSI per-category numbers need error bars. Third, the paper never states whether the SpatialClaw and S-Agent baseline numbers were reproduced locally or quoted; the Appendix D experiment implies they ran SpatialClaw, so this is probably fine, but it should be explicit. Minor: the system was designed from failure analysis on the evaluation benchmarks themselves, so the reported margins may be mildly optimistic; common practice, but undisclosed.
The stress-test concern about P is the right one to push on. It lands on the metric tier, not on the relative-reasoning results, and the reader's conditional verdict is about right.
Who gets value: anyone building tool-augmented agents or spatial-reasoning benchmarks. Send it to serious peer review; ask for variance reporting and a reconstruction-validation experiment.
The strongest objection
Every interpreted readout (distance, depth, side, elevation, motion, compass) is computed closed-form from the fused, gravity-aligned point cloud P (Eq. 2, §2.2), and P is produced by the recon command fusing three frozen perception models — Depth Anything 3, GeoCalib, π3mos (§2.3, Table 7). The paper never measures the error of this fused product on any scene: no reconstruction-error numbers, no check that P's scale or registration supports the metre-valued readouts it feeds. Appendix B discloses that the scale is 'metres on the reconstruction's own metric scale (uncalibrated)', and the verbatim transcripts show the interface deliberately suppresses verification — replies carry notes like 'copy the conclusion, do not re-derive it from the pictures' (Figures 5–10). The inspect-and-revise habit (§3.4) edits the instance registry (drop, add_instance) but cannot catch metric errors inside P, because every position, including model-registered instances whose pixels are lifted onto P, inherits P's errors. The one place the paper's own data probe this is Table 2: on both backbones, absolute distance is the single category where S-Agent 2.0 falls behind SpatialClaw (−2.8 Qwen, −1.7 Gemini), and Gemini object size is slightly behind (−0.5). These are exactly the questions that isolate metric quality. The abstract's claim that 'every reading it acts on stays grounded in the tools' geometry' is therefore asserted rather than measured, and §3.4's failure analysis audits question-grounding but never the geometry tier. The concern targets the paper's own unvalidated fusion, not the cited perception models [20, 32, 37]: if P carries structural (non-homogeneous) distortion, the evidence loop closes around confidently wrong metric answers, weakening the method's correctness and its transfer to calibrated-scale settings.
The paper is an empirical systems paper whose central claim is comparative (S-Agent 2.0 leads SpatialClaw on every benchmark and backbone) plus a transfer claim (the evidence-loop interface, not a purpose-built harness, carries the gain). The condition that must hold for the central claim is that the readouts the agent is told to trust are accurate interpretations of correct geometry, and that the comparisons are apples-to-apples. I find the geometry condition the least secure. The readouts are closed-form functions of P, P is a fusion of frozen models whose error is never measured, and the interface instructs the agent to copy conclusions without re-deriving them, so metric errors in P cannot be caught by the inspect-and-revise reflex, which only edits the instance registry. Table 2 supplies the in-paper signal: on the benchmark category that most isolates metric quality (absolute distance), S-Agent 2.0 loses to SpatialClaw on both backbones, consistent with the uncalibrated scale that Appendix B discloses. I steelmanned the paper on all three fronts. The tool gains (+13.9/+20.5 on Qwen) are internally controlled against the same agent without tools and survive any concern about SpatialClaw bookkeeping. If SpatialClaw shares the perception backends, geometry errors partially cancel in the comparative margins, though the paper never states whether the Table 1–3 SpatialClaw numbers were reproduced locally with identical backends — a genuine gap the reader correctly flagged, which I treat as secondary because the tool-gain pillar stands independently. Test-informed design (failure analysis on the evaluation benchmarks, §1 and §3.4) is disclosed through the paper's own narrative and threatens external validity more than internal correctness; the reader's accounting of it is fair. Variance reporting is absent, but the headline margins (+3.9 to +10.8) are too large for run-to-run noise to plausibly flip them, so I do not treat that as the load-bearing point. Because the reader's CONDITIONAL verdict already conditions acceptance on exactly this class of unresolved verification, my pass does not change the verdict category; I record UNCHANGED. I partially agree with the reader's weakest_assumption: it names the same reconstruction foundation, and I add the mechanism (verbatim-trust instructions, registry-only cross-checks) and the in-paper evidentiary signal (the absolute-distance deficit) that make the concern concrete and testable.
The referee's report
The paper's strengths are its clean design hypothesis and its systematic evidence: same-backbone comparisons to SpatialClaw on five benchmarks (Tables 1-3), tool/no-tool and interface ablations (Tables 5-6), a cross-harness study (Table 4), a readout-transplant decomposition showing that interpreted readouts alone add little to SpatialClaw (Table 9), released code and verbatim trajectories, and an honest two-class failure analysis (Figure 4). If the results hold, the design result is significant: an evidence-loop interface (environment-held scene memory, closed-form readouts in spatial language, model-registered instances on the same geometric footing) layered on an unmodified coding agent outperforms a purpose-built spatial harness on the same backbone, and the principle transfers across three coding-agent harnesses.
Pith's reading of the paper (a machine's summary, not the authors')
S-Agent 2.0 argues that agentic spatial reasoning fails not because perception tools measure the scene wrongly, but because the model never closes the loop between a measurement and the answer: correct readings get misread, forgotten, or ignored. The fix is to keep the coding agent stock and move memory and interpretation into the tool interface — a persistent scene memory that later calls query by name, and replies that arrive already phrased in the spatial language the question is asked in. With no training, the toolkit lifts Qwen3.8-27B by +13.9 and +20.5 points on MMSI-Bench and ReVSI and Gemini-3.8-Flash by +11.2 and +9.6, ahead of the purpose-built SpatialClaw on every benchmark and backbone tested and under several coding-agent harnesses. If the claim holds, spatial competence can be bought at inference time by interface design rather than by training or custom controllers, and the next bottleneck is grounding the question's meaning into a measurement.
What carries the argument. The carrying mechanism is the evidence loop, materialized as a persistent scene memory. Each episode holds $\mathcal{S} = \{I, P, \{(\nu_i, g_i)\}_i\}$: the input frames, a fused metric point cloud in a gravity-aligned world frame, and a registry of named instances, each with an $xyz$ position, an extent, and a 6-DoF pose. Two write routes fill the memory — tool perception (reconstruction, segmentation, motion, and gravity from off-the-shelf models) and model perception (the agent adds imagined observers or removes wrong masks through `add_instance` and `drop`) — so the model's own reading of the scene and the tools' measurements land on one geometric footing. Readout commands such as `side`, `distance`, `compass`, and `motion` then compute answers in closed form from the stored geometry and return a structured report with typed numeric fields plus a one-sentence conclusion in plain spatial language, so the model never re-derives a reading from the pictures. A fixed instruction gives the agent a tutorial over these commands and a recommended workflow: decide the evidence the question needs, acquire it, check the outputs, and answer from the readings.
Core claim. The paper's central claim is that the bottleneck in agentic spatial reasoning is the evidence loop, not evidence acquisition. Analyzing the specialized baseline's trajectories, the authors find that the tools rarely fail: the model flips the sign of a rotation it measured correctly, reads a correct number in the wrong coordinate frame, or collects readings and then answers from its own impression of the pictures. S-Agent 2.0 inverts the usual allocation of effort — a stock general coding agent stays the planner and executor, and the design effort goes into a command-line perception toolkit that lets the model register what it perceives or imagines as named instances in the same geometric scene the tools build, keeps every measurement in a persistent per-episode scene memory queried by instance name, and returns every reading as an interpreted conclusion sentence with the supporting numbers attached. On MMSI-Bench, MindCube, SPAR-Bench, MMSI-Video-Bench, and ReVSI it outperforms the specialized agentic baseline on the same backbones; with Gemini-3.8-Flash it posts the highest overall scores among all compared models, GPT-6 included, though GPT-6 keeps the lead on region-to-region relations, camera motion, and multi-step reasoning. Its tool gain on ReVSI stays at +9.6 on the stronger backbone, where the baseline's shrinks from +13.2 to +2.7.
The premise it rests on. The load-bearing premise is that the fused metric point cloud from the frozen, un-audited perception models is accurate enough for every closed-form reading to be trustworthy, and the paper never measures or bounds that reconstruction, so biased geometry would make every interpreted reading confidently wrong.
If the paper is right (Pith's inferences, not the authors')
No fine-tuning is involved: a 27B open backbone with the toolkit surpasses every agentic baseline on all five benchmarks, so spatial competence can be delivered at inference time by tool design.
The same toolkit transfers across general coding-agent harnesses — OpenHands, Codex, and Claude Code drive the identical workspaces to MMSI-Bench scores of 66.5, 67.0, and 63.8 — so the method is not tied to one agent loop.
The ablations rank where the gain lives: removing the model-perception interface costs the most (−8.8 on MMSI-Bench, −5.9 on ReVSI), removing the scene memory costs less (−3.5, −2.5), and removing the interpreted readout costs least (−1.3, −1.2), matching the diagnosis that acquired-but-unused evidence is the core failure.
The interface, not just the reply format, carries the gain: transplanting only the interpreted readouts into the specialized baseline's kernel recovers just +1.0 on ReVSI and +0.3 on MMSI-Bench, far short of S-Agent 2.0's full margins, so scene memory and the model-perception route do most of the work.
Errors that remain concentrate where the question first enters the loop — a wrongly grounded imagined viewpoint or a quantity measured in the wrong sense — which identifies the next target for interface design.
If the evidence-loop diagnosis generalizes, the same failure pattern — correct tool output that the model misreads or discards — should appear in other tool-augmented settings such as audio, robotic state estimation, or scientific data, and moving memory and interpretation into those interfaces should reproduce the effect; this is testable but the paper does not test it.
The consistent deficit on absolute-distance questions (−2.8 for Qwen3.8-27B, −1.7 for Gemini-3.8-Flash on ReVSI) sits exactly where Appendix B says the scale is uncalibrated unless the agent injects a known reference; a plausible extension is that forcing scale calibration on every episode would close or reverse that gap.
All readouts are computed closed-form from one fused point cloud produced by frozen, un-audited perception models, so the whole approach inherits their error; measuring reconstruction accuracy against ground-truth scene geometry would show how much of the reported gain is the interface and how much is the geometry.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Spatial reasoning asks a model to work out where objects are in 3D space, how they relate to one another, and how they move. Agentic VLMs have recently brought large improvements on such tasks by letting the model actively gather evidence with 3D perception tools instead of answering in a single pass. Even so, on many questions the tools return the right evidence and the model still does not arrive at the correct answer: the gap between model and tools, where raw readings become usable conclusions, is left unfilled. We present S-Agent 2.0, a spatial reasoning agent that equips an off-the-shelf general coding agent with a command-line perception toolkit, and we design the interface around one principle: help the VLM close the evidence loop: acquire scene evidence, remember it in a persistent scene memory, and interpret each reading in plain spatial language. This design keeps the full flexibility of a coding agent, and every reading it acts on stays grounded in the tools' geometry. On MMSI-Bench and ReVSI, the toolkit improves a strong open-source model, Qwen3.8-27B, by +13.9 and +20.5 points over the same agent without tools, and a strong closed-source model, Gemini-3.8-Flash, by +11.2 and +9.6 points. With the open backbone S-Agent 2.0 reaches 66.5 on MMSI-Bench and 82.3 on ReVSI, ahead of the specialized system SpatialClaw on the same backbone (62.6 and 74.5), and with Gemini-3.8-Flash it reaches 74.1 and 90.5 against its 67.0 and 79.7. Both tool gains shrink as the backbone grows more capable, but SpatialClaw's shrinks far more on ReVSI, from +13.2 to +2.7, while ours stays at +9.6. S-Agent 2.0 also works across different general coding agent harnesses, such as Codex and Claude Code.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-21
The authors' main claim, as the classifier picked it out 0.56
“With the open backbone S-Agent 2.0 reaches 66.5 on MMSI-Bench and 82.3 on ReVSI, ahead of the specialized system SpatialClaw on the same backbone (62.6 and 74.5), and with Gemini-3.8-Flash it reaches 74.1 and 90.5 against its 67.0 and 79.7.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.91 · human participants 0.03 · controlled comparison 0.96 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.05
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Spatial reasoning asks a model to work out where objects are in 3D space, how they relate to one another, and how they move. | background 1.00 | 0.04 | 0.02 | asserted 0.99 |
| Agentic VLMs have recently brought large improvements on such tasks by letting the model actively gather evidence with 3D perception tools instead of answering in a single pass. | background 1.00 | 0.04 | 0.03 | asserted 1.00 |
| Even so, on many questions the tools return the right evidence and the model still does not arrive at the correct answer: the gap between model and tools, where raw readings become usable conclusions, is left unfilled. | gap 1.00 | 0.18 | 0.06 | asserted 0.99 |
| We present S-Agent 2.0, a spatial reasoning agent that equips an off-the-shelf general coding agent with a command-line perception toolkit, and we design the interface around one principle: help the VLM close the evidence loop: acquire scene evidence, remember it in a persistent scene memory, and interpret each reading in plain spatial language. | contribution 0.88 | 0.94 | 0.26 | asserted 1.00 |
| This design keeps the full flexibility of a coding agent, and every reading it acts on stays grounded in the tools' geometry. | method 0.59 | 0.63 | 0.07 | asserted 1.00 |
| On MMSI-Bench and ReVSI, the toolkit improves a strong open-source model, Qwen3.8-27B, by +13.9 and +20.5 points over the same agent without tools, and a strong closed-source model, Gemini-3.8-Flash, by +11.2 and +9.6 points. | result 1.00 | 0.95 | 0.25 | asserted 1.00 |
| With the open backbone S-Agent 2.0 reaches 66.5 on MMSI-Bench and 82.3 on ReVSI, ahead of the specialized system SpatialClaw on the same backbone (62.6 and 74.5), and with Gemini-3.8-Flash it reaches 74.1 and 90.5 against its 67.0 and 79.7. | result 1.00 | 0.95 | 0.22 | asserted 1.00 |
| Both tool gains shrink as the backbone grows more capable, but SpatialClaw's shrinks far more on ReVSI, from +13.2 to +2.7, while ours stays at +9.6. | result 0.96 | 0.93 | 0.11 | asserted 1.00 |
| S-Agent 2.0 also works across different general coding agent harnesses, such as Codex and Claude Code. | result 0.83 | 0.85 | 0.04 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-21: 1 request, 7,415 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (10)
Figures are the authors', reproduced from the paper.
References (61)
- [1] Anthropic. 2025. Claude 3.7 sonnet and claude code. https://www.anthropic.com/news/ claude-3-7-sonnet. Accessed: 2026-09-23 on Pith
- [2] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasm... on PithPith's reviewarXiv
- [3] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. 2020. nuscenes: A multimodal dataset for autonomous driving. In2020 IEEE/CVF conference on computer vision and pattern recognition... on Pith
- [4] Zhongang Cai, Ruisi Wang, Chenyang Gu, Fanyi Pu, Junxiang Xu, Yubo Wang, Wanqi Yin, Zhitao Yang, Chen Wei, Qingping Sun, Tongxi Zhou, Jiaqi Li, Hui En Pang, Oscar Qian, Yukun Wei, Zhiqian Lin, Xuanke Shi, Kewang Deng, Xiaoyang Han, and 10 others. 2026. Scaling spatial intellig... on Pith
- [5] Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris Coll- Vinent, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman Rä... on Pith
- [6] Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14455–14465 on Pith
- [7] Siyi Chen, Mikaela Angelina Uy, Chan Hee Song, Faisal Ladhak, Adithyavairavan Murali, Qing Qu, Stan Birchfield, Valts Blukis, and Jonathan Tremblay. 2026. Spacetools: Tool-augmented spatial reasoning via double interactive rl. InProceedings of the IEEE/CVF Conference on Comput... on Pith
- [8] Zeren Chen, Xiaoya Lu, Zhijie Zheng, Pengrui Li, Lehan He, Yijin Zhou, Jing Shao, Bohan Zhuang, and Lu Sheng. 2026. Geometrically-constrained agent for spatial reasoning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 38689–38699 on Pith
- [9] An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. 2024. Spatialrgpt: Grounded spatial reasoning in vision-language models.Advances in Neural Information Processing Systems, 37:135062–135093 on Pith
- [10] Seokju Cho, Ryo Hachiuma, Abhishek Badki, Hang Su, Byung-Kwan Lee, Chan Hee Song, Sifei Liu, Subhashree Radhakrishnan, Seungryong Kim, Yu-Chiang Frank Wang, and Min-Hung Chen. 2026. Spatialclaw: Rethinking action interface for agentic spatial reasoning.CoRR, abs/2606.13673 on PithPith's reviewarXiv
- [11] Yalun Dai, Hao Li, Shulin Tian, Runmao Yao, Yuhao Dong, Fangzhou Hong, Zhaoxi Chen, Fangfu Liu, Baoliang Tian, Dingwen Zhang, Tao Wang, Kim-Hui Yap, and Ziwei Liu. 2026. S-agent: Spatial tool-use elicits reasoning for spatial intelligence.CoRR, abs/2606.20515 on PithPith's reviewarXiv
- [12] Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Tous- sai... on PithPith's reviewarXiv
- [13] Mengfei Du, Binhao Wu, Zejun Li, Xuan-Jing Huang, and Zhongyu Wei. 2024. Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volum... on Pith
- [14] Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. 2024. Blink: Multimodal large language models can see but not perceive. InEuropean Conference on Computer Vision, pages 148–166. Springer on Pith
- [15] Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for autonomous driving? the kitti vision benchmark suite. In2012 IEEE conference on computer vision and pattern recognition, pages 3354–3361. IEEE on Pith
- [16] Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Nan Duan, and Weizhu Chen on Pith
- [17] Tanmay Gupta and Aniruddha Kembhavi. 2023. Visual programming: Compositional visual reason- ing without training. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14953–14962. IEEE on Pith
- [18] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. Swe-bench: Can language models resolve real-world github issues? InInternational Conference on Learning Representations, volume 2024, pages 54107–54157. 12 on Pith
- [19] Hongxing Li, Dingming Li, Zixuan Wang, Yuchen Yan, Hang Wu, Wenqi Zhang, Yongliang Shen, Weiming Lu, Jun Xiao, and Yueting Zhuang. 2026. Spatialladder: Progressive training for spatial reasoning in vision-language models. InInternational Conference on Learning Representations,... on Pith
- [20] Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Yang Zhao, Sida Peng, Hengkai Guo, Xiaowei Zhou, Guang Shi, Jiashi Feng, and Bingyi Kang. 2026. Depth anything 3: Recovering the visual space from any views. InThe Fourteenth International Conference on Learning R... on Pith
show 41 more
- [21] Jingli Lin, Runsen Xu, Shaohao Zhu, Sihan Yang, Peizhou Cao, Yunlong Ran, Miao Hu, Chenming Zhu, Yiman Xie, Yilin Long, Wenbo Hu, Dahua Lin, Tai Wang, and Jiangmiao Pang. 2025. Mmsi-video- bench: A holistic benchmark for video-based spatial intelligence.CoRR, abs/2512.10863 on PitharXiv
- [22] Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, and James Zou. 2025. Octotools: An agentic framework with extensible tools for complex reasoning.arXiv preprint arXiv:2502.11271 on PithPith's reviewarXiv
- [23] Zhanpeng Luo, Ce Zhang, Silong Yong, Cunxi Dai, Qianwei Wang, Haoxi Ran, Guanya Shi, Katia Sycara, and Yaqi Xie. 2026. pyspatial: Generating 3d visual programs for zero-shot spatial reasoning. InThe Fourteenth International Conference on Learning Representations on Pith
- [24] Damiano Marsili, Rohun Agrawal, Yisong Yue, and Georgia Gkioxari. 2025. Visual agentic ai for spatial reasoning with a dynamic api. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19446–19455. IEEE on Pith
- [25] OpenAI. 2025. Introducing codex. https://openai.com/index/introducing-codex/. Accessed: 2026-09-23 on Pith
- [26] Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun on Pith
- [27] Fernando Ropero, Erkin Turkoz, Daniel Matos, Junqing Du, Antonio Ruiz, Yanfeng Zhang, Lu Liu, Mingwei Sun, and Yongliang Wang. 2026. Riemind: Geometry-grounded spatial agent for scene understanding.arXiv preprint arXiv:2603.15386 on PitharXiv
- [28] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.Advances in Neural Information Processing Systems, 36:38154–38180 on Pith
- [29] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Re- flexion: Language agents with verbal reinforcement learning.Advances in neural information processing systems, 36:8634–8652 on Pith
- [30] Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. 2025. Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15... on Pith
- [31] Dídac Surís, Sachit Menon, and Carl Vondrick. 2023. Vipergpt: Visual inference via python execution for reasoning. In2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11854–11864. IEEE on Pith
- [32] Alexander Veicht, Paul-Edouard Sarlin, Philipp Lindenberger, and Marc Pollefeys. 2024. Geocalib: Learning single-image calibration with geometric optimization. InEuropean Conference on Computer Vision, pages 1–20. Springer on Pith
- [33] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. 2025. Vggt: Visual geometry grounded transformer. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5294–5306. IEEE on Pith
- [34] Qineng Wang, Baiqiao Yin, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, jiajun Wu, Li Fei-Fei, and Manling Li. 2025. Spatial mental modeling from limited views. InThe Fourteenth Internation... on Pith
- [35] Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. Executable code actions elicit better llm agents. InInternational Conference on Machine Learning (ICML). 13 on Pith
- [36] Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Daniel Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, and 5 others. 2025. Openhands:... on Pith
- [37] Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. 2026. $\pi^3$: Permutation-equivariant visual geometry learning. InThe Fourteenth International Conference on Learning Representations on Pith
- [38] Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. 2026. Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence.Advances in neural information processing systems, 38:13569–13597 on Pith
- [39] Junfei Wu, Jian Guan, Kaituo Feng, Qiang Liu, Shu Wu, Liang Wang, Wei Wu, and Tieniu Tan. 2026. Reinforcing spatial reasoning in vision-language models with interwoven thinking and visual drawing. Advances in Neural Information Processing Systems, 38:143297–143330 on Pith
- [40] Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. 2025. Thinking in space: How multimodal large language models see, remember, and recall spaces. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10632–10643. IEEE on Pith
- [41] John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. Swe-agent: Agent-computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems, 37:50528–50652 on Pith
- [42] Rui Yang, Ziyu Zhu, Yanwei Li, Jingjia Huang, Shen Yan, Siyuan Zhou, Zhe Liu, Xiangtai Li, Shuangye Li, Wenqian Wang, Yi Lin, and Hengshuang Zhao. 2026. Visual spatial tuning. InEuropean Conference on Computer Vision, pages 192–211. Springer on Pith
- [43] Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis L Brown II, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, Rob Fergus, Yann LeCun, Li Fei-Fei, and Saining Xie. 2026. Cambrian-s: Towards spatial supersensing in video. InThe Fourteenth International Confer... on Pith
- [44] Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, Dahua Lin, Tai Wang, and Jiangmiao Pang. 2026. Mmsi-bench: A benchmark for multi-image spatial intelligence. InInternational Conference on Learning Represen... on Pith
- [45] Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. 2023. Mm-react: Prompting chatgpt for multimodal reasoning and action.arXiv preprint arXiv:2303.11381 on PithPith's reviewarXiv
- [46] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao on Pith
- [47] Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyza- guirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, and 1 others. 2025. Re-thinking temporal search for long-form video understanding. In2025 IEEE/CVF Conference on Comp... on Pith
- [48] Hang Zhang, Xin Li, and Lidong Bing. 2023. Video-llama: An instruction-tuned audio-visual language model for video understanding. InProceedings of the 2023 conference on empirical methods in natural language processing: system demonstrations, pages 543–553 on Pith
- [49] Jiahui Zhang, Yurui Chen, Yueming Xu, Ze Huang, Jilin Mei, Chunhui Chen, Yanpeng Zhou, Yu-Jie Yuan, Xinyue Cai, Guowei Huang, Xingyue Quan, Hang Xu, and Li Zhang. 2026. From flatland to space: Teaching vision-language models to perceive and reason in 3d.Advances in Neural Info... on Pith
- [50] Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. 2025. Long context transfer from language to vision. Transactions on Machine Learning Research. 14 on Pith
- [51] Yiming Zhang, Jiacheng Chen, Jiaqi Tan, Yongsen Mao, Wenhu Chen, and Angel X Chang. 2026. Revsi: Rebuilding visual spatial intelligence evaluation for accurate assessment of vlm 3d reasoning. InForty-third International Conference on Machine Learning on Pith
- [52] Zaibin Zhang, Yuhan Wu, Lianjie Jia, Yifan Wang, Zhongbo Zhang, Yijiang Li, Binghao Ran, Fuxi Zhang, Zhuohan Sun, Yizhuang Peng, Zhenfei Yin, Lijun Wang, and Huchuan Lu. 2026. Think3d: Thinking with space for spatial reasoning.Preprint, arXiv:2601.13029 on PithPith's reviewarXiv
- [53] Shitian Zhao, Haoquan Zhang, Shaoheng Lin, Ming Li, Qilong Wu, Kaipeng Zhang, and Chen Wei on Pith
- [54] Xingguang Zhong, Liren Jin, Marija Popovic, Jens Behley, and Cyrill Stachniss. 2026. Dynamic visual slam using a general 3d prior. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21805–21815. 15 A. Related Work Spatial reasoning in vi... on Pith
- [57] Pyvision: Agentic vision with dynamic tooling.arXiv preprint arXiv:2507.07998 on PithPith's reviewarXiv
- [60] -> this is RIGHT! right_correct = np.array([fwd_2d[1], -fwd_2d[0]]) # = up x forward f x u: [-1 0 0] u x f: [1 0 0] Correct dot_fwd: -1.970 (positive = front, negative = back) Correct dot_right: -1.596 (positive = right, negative = left) Final Answer: A S-Agent 2.015 perceptio... on Pith
- [61] **Ground Crack and Jack Handle:** ... remain in nearly the same relative lateral position ... confirming little to no lateral translation. Therefore, the person simply moved straight ahead. **Correct Answer:** **D: Forward** # step 3, model text: In egocentric action / navigat... on Pith
- [223] Each figure shows the question, the intermediate images of the two agents (segmentation masks, top-down plots, our bev maps), and the command-line record on both sides, quoted verbatim from the trajectories and trimmed to the lines that decide the answer; images are downsample... on Pith
- [2023] React: Synergizing reasoning and acting in language models. InThe Eleventh International Conference on Learning Representations on Pith
- [2024] Critic: Large language models can self-correct with tool-interactive critiquing. InInternational Conference on Learning Representations, volume 2024, pages 57734–57811 on Pith
- [2025] Spacer: Reinforcing mllms in video spatial reasoning.arXiv preprint arXiv:2504.01805 on PithPith's reviewarXiv
Cited by 1 Pith paper
Reviewed papers on Pith that cite this one. The full citation record
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment