{"id":"10727fce-2b41-426e-9910-0f72c26b7181","arxiv_id":"2603.12939","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Training-free STF-Tokens plus a Causal Spatio-Temporal Graph let VLMs keep object permanence and action history, raising long-horizon robotic manipulation success far above reactive baselines.","lead":"RoboStream is a training-free add-on that gives vision-language robot planners persistent 3D object tokens and a causal memory graph so they stop forgetting occluded objects and compounding spatial errors over long tasks. It lifts long-horizon success from ~11% to 44% on hard real block-building and to 90% on RLBench, showing memory and geometry anchors matter more than model size alone.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Ablation confounds memory with VLM reasoning depth; CSTG gains may not isolate causal memory as claimed.","rationale":"The reader correctly flags perception fragility (20.7% parsing failures) and modest absolute real-world numbers, which already justify CONDITIONAL. The more load-bearing issue for the paper’s strongest claim, however, is that the CSTG ablation does not cleanly separate causal structure from prompt richness. The reader’s weakest_assumption (identity-preserving masks) is real but secondary: even perfect perception would leave the causal-memory claim under-isolated. A non-causal history control would settle the matter with one experiment and either strengthen or temper the abstract’s assertion that the two modules are the critical missing components. No change of verdict category is required; the existing CONDITIONAL already accommodates this refinement.","tokens_in":28909,"tokens_out":559,"duration_ms":6946,"concrete_test":"Re-run the eight RLBench long-horizon tasks with a non-causal memory control: keep the same sliding-window length K and object list, but replace CSTG event captions/sources with a flat chronological list of previous action strings (no occlusion flags, no causal source tags). If success remains near the full 90.5% (or within ~10 points), the causal structure itself is not load-bearing; if it collapses toward the 14.5% w/o-CSTG number, the original claim is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that STF-Tokens + CSTG are the critical missing components rests on Table 5 (and Appendix Table 8) ablations: removing CSTG collapses average success from 90.5% to 14.5% on RoboStream-235B. However, the w/o-CSTG condition still feeds the same large VLM a prompt that lacks the structured causal log and sliding-window history (Sec. 3.4–3.5). The observed collapse therefore conflates two effects: (1) absence of persistent causal state, and (2) a substantially weaker prompt that no longer supplies the chain-of-thought scaffolding the VLM was using for multi-step verification. Because the paper never reports a control that retains an unstructured or non-causal history of comparable length (e.g., raw previous action captions or a flat object list without event sources), it is impossible to know how much of the 76-point drop is truly attributable to causal structure versus simply giving the VLM less context. The same confound appears at 8B/32B scales. Thus the strongest empirical support for “causal memory is critical” is only partially isolated.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"RoboStream is a training-free planning framework that injects two structured representations into a VLM loop for long-horizon robotic manipulation: Spatio-Temporal Fusion Tokens (STF-Tokens), which bind per-object visual patches to 3D centroids and Gaussian shape parameters (Eqs. 1, 5; §3.3), and a Causal Spatio-Temporal Graph (CSTG) that maintains sliding-window object identities and action-triggered event logs (Eqs. 2, 6; §3.4). Semantic action directives from the VLM are then deterministically grounded to 6-DoF poses via stored geometry (Eqs. 3–4). Evaluated zero-shot across SIMPLER, eight long-horizon RLBench tasks, 6-DoF SpatialBench, Open6DOR V2, and 21 real Franka tasks, the method reports 90.5% average success on long-horizon RLBench (RoboStream-235B) and 44.4% on hard real block-building, versus 11.1% for SoFar and VoxPoser, with ablations (Table 5; Appendix Table 8) attributing large gains to CSTG and secondary gains to STF-Tokens.","tokens_in":29234,"tokens_out":992,"duration_ms":17707,"significance":"If the results hold under tighter controls, the paper makes a clear empirical case that persistent geometric anchoring and structured cross-step memory—not model scale alone—are decisive for long-horizon VLM planners. Strengths include: (i) a fully training-free design that transfers across embodiments (Google Robot, WidowX, Franka) without fine-tuning; (ii) multi-benchmark coverage spanning short-horizon spatial grounding, long-horizon occlusion/restoration, and real hardware; (iii) scale sweeps (8B/32B/235B) and component ablations that cleanly show CSTG is necessary for multi-step consistency; and (iv) an honest real-world failure breakdown (Appendix C.2). These are concrete, falsifiable contributions to embodied VLM planning and would be of interest to the robotics and VLM communities.","major_comments":[{"comment":"Table 5 / Appendix Table 8 and §4.6: the claim that “causal memory” is the critical missing component rests on the w/o-CSTG collapse (90.5% → 14.5% at 235B). The ablated condition removes both the causal event structure and essentially all multi-step history/scaffolding from the prompt (Sec. 3.4–3.5). There is no control that retains a non-causal history of comparable length (e.g., raw prior action captions, a flat object list without event sources, or an unstructured sliding window). Without that control, the 76-point drop confounds causal structure with simply giving the VLM more context for chain-of-thought verification. The same pattern appears at 8B/32B. Either add such a control or soften the abstract/title/§5 language from “causal memory” to “structured persistent memory,” and state the isolation limit explicitly.","section":null},{"comment":"§3.2–3.3 and Appendix C.2: identity-preserving object parsing is a load-bearing assumption. Descriptors and SAM3 masks are conditioned on M_<t, yet 20.7% of real-world failures are already Object Parsing & Detection errors under color-similar distractors and stacking occlusion. The paper does not quantify identity-switch or re-ID rates over long horizons, nor does it report sensitivity to the free parameters I_th (patch IoU filter) and sliding-window length K. Because STF-Tokens and CSTG nodes inherit these identities, a short quantitative study of re-ID stability (and of K / I_th) is needed to support the claim of “persistent object grounding” under occlusion.","section":null},{"comment":"§4.2 / Tables 6–7 and §4.4 / Table 2: success rates are reported without uncertainty. Real-world tasks use only three trials each (so 44.4% is 4/9), and RLBench uses 25 episodes per task with no confidence intervals or binomial error bars. The headline comparisons (e.g., 44.4% vs 11.1%; 90.5% vs 28.0%) are directionally convincing but not statistically characterized. Please report intervals (or at least raw counts consistently) and, where feasible, increase real-world trial counts for the hard settings that drive the abstract claim.","section":null}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: if you give a frozen VLM identity-persistent object tokens that already carry 3D centroid + Gaussian shape, plus a sliding-window causal log of what each action did, long-horizon success jumps hard without any fine-tuning. On eight RLBench long-horizon tasks RoboStream-235B hits 90.5% average; SoFar and VoxPoser sit at 28% and 26.5%. On the hard real Franka block-building suite they report 44.4% versus 11.1% for both baselines, and on hide-and-restore under full occlusion the gap is 88.9% vs 0%. That is the result that matters.\n\nWhat is actually new is the packaging, not the individual ideas. Scene graphs, object permanence, and VLM planners already exist. The concrete engineering is STF-Tokens (visual patches filtered by IoU, serialized with median centroid and per-axis Gaussian extents into the same embedding space) plus a CSTG that keeps a K-step buffer of those tokens and an event log with causal source. The design is training-free, works zero-shot across Google Robot / WidowX / Franka, and the ablations (main Table 5 + Appendix Table 8) cleanly show both pieces matter: drop CSTG and average collapses to ~14%; drop only STF-Tokens and you lose ~11 points of physical precision. Failure modes are quantified (43% VLM planning, 21% SAM3 parsing). That is honest systems work.\n\nThe soft spot the stress-test flags is real but not fatal. The w/o-CSTG condition also removes the structured history the VLM was using for chain-of-thought verification, so the 76-point drop confounds “causal structure” with “less context.” They never ran the obvious control (flat previous-action captions or a non-causal object list of similar length). So the claim that causal memory is the critical missing component is only partially isolated. Perception dependence is also still load-bearing: 20% of real failures are already open-vocab segmentation. Absolute hard-task numbers remain modest, and no code is released.\n\nStill, the empirical pattern is consistent across five benchmarks and three model scales. Anyone building long-horizon VLM or VLA planners should read the method and the ablations. I would send it to referees; the contribution is real enough to deserve the scrutiny, and the confound is fixable with one extra control.","headline":"Training-free VLM planner that binds visual patches to 3D Gaussians and logs action-triggered state transitions; large, ablated gains on long-horizon RLBench and real Franka, with one real confound in the memory ablation.","tokens_in":29879,"tokens_out":618,"would_cite":true,"duration_ms":7461,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Vision-language robot planners fail on long tasks because they rebuild geometry from pixels and forget action history; RoboStream fixes both with persistent 3D object tokens and a causal memory graph, without training.","keywords":["robot manipulation","vision-language models","long-horizon planning","spatio-temporal reasoning","causal memory","object permanence","training-free"],"falsifier":"Run the same long-horizon hide-and-restore block tasks while deliberately corrupting or swapping segmentation masks and object IDs mid-episode; if success collapses while the causal graph still receives those broken identities, the claim that the memory structures themselves are doing the decisive work is falsified.","tokens_in":29792,"feed_emoji":"🤖","tokens_out":1012,"duration_ms":24455,"temperature":0.7,"pith_summary":"Vision-language models that plan robot actions usually treat every step as a fresh look at the camera: they re-guess where objects are from raw pixels and keep no record of what earlier moves did to the scene. That works for short pick-and-place work but collapses on long sequences—small spatial errors compound, temporarily hidden objects are forgotten, and later steps fail because the assumed preconditions no longer hold. RoboStream is a training-free method that binds each object's visual evidence to its measured 3D center and shape (Spatio-Temporal Fusion Tokens) and logs every action-triggered state change in a Causal Spatio-Temporal Graph. The same off-the-shelf models then reach 90.5% average success on eight long-horizon simulation tasks and 44.4% on hard real block-building, while prior systems score near 11%. The paper's claim is that geometric anchoring and causal memory, not merely larger models, are the missing pieces for reliable multi-step manipulation.","feed_headline":"Persistent 3D memory lifts robot VLMs to 90% long-horizon success","feed_subtitle":"Geometry-bound tokens and a causal graph stop planners from forgetting occluded objects.","key_machinery":"Spatio-Temporal Fusion Tokens (STF-Tokens) compress each object's filtered visual patches together with its 3D centroid and Gaussian shape into a single identity-stable reference; the Causal Spatio-Temporal Graph (CSTG) stores those tokens across a sliding window plus action-triggered events, so the planner can query last-known pose and causal history even when an object is fully hidden.","core_discovery":"The dominant failure of VLM-based robot planners on long-horizon work is the lack of two representations: identity-persistent geometric anchors that stop the model from re-inferring 3D geometry from pixels at every step, and an explicit causal log of how actions and occlusions rewrite object states. Injecting Spatio-Temporal Fusion Tokens and a Causal Spatio-Temporal Graph into an otherwise unchanged VLM recovers object permanence under occlusion and prevents cascading precondition violations, producing large gains on both simulated multi-step benchmarks and real Franka block tasks with no fine-tuning.","pith_inferences":["The same token-plus-graph pattern could be internalized inside continuous vision-language-action controllers so causal memory guides low-level motion, not only high-level plans.","Because parsing and detection already dominate real-world failures, further gains may come more from identity-stable open-vocabulary tracking than from larger planners.","If the causal graph is the main driver of long-horizon gains, lighter discrete geometric state machines might deliver similar reliability at lower latency.","Contact-rich or deformable tasks would test whether Gaussian shape slots are enough or need richer physical attributes."],"forward_implications":["Long multi-step stacking, covering, and restoration tasks become reliable once geometric identity and causal history are supplied, even for 8B-scale models.","Zero-shot transfer across robot embodiments improves because geometry is expressed in metric 3D rather than image-specific features.","Model scale helps more once the memory structures exist; without them, larger VLMs still fail on occlusion and multi-step consistency.","Reactive planners that rebuild the scene from pixels each step remain limited on any task that requires object permanence or reverse-order restoration.","Structured memory can be injected without fine-tuning and still close much of the gap previously expected only from end-to-end action models."],"fun_headline_variants":["STF tokens bind VLMs to persistent 3D geometry across robot steps","Causal graph tracks action states so VLMs stop forgetting occlusions","RoboStream adds memory of state transitions for 90% long-horizon gains","Geometry anchors plus causal log lift VLM robots without fine-tuning","Training-free spatio-temporal memory ends cascading planner failures"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The system assumes open-vocabulary segmentation and language parsing will keep assigning the same object the same identity mask across many steps even under stacking, full occlusion, and color-similar distractors.","fun_headline_variants_meta":{"raw":{"variants":["STF tokens bind VLMs to persistent 3D geometry across robot steps","Causal graph tracks action states so VLMs stop forgetting occlusions","RoboStream adds memory of state transitions for 90% long-horizon gains","Geometry anchors plus causal log lift VLM robots without fine-tuning","Training-free spatio-temporal memory ends cascading planner failures"]},"model":"grok-4.5","effort":"low","cost_usd":0.004026,"raw_usage":{"total_tokens":1307,"prompt_tokens":906,"num_sources_used":0,"completion_tokens":77,"cost_in_usd_ticks":40260000,"prompt_tokens_details":{"text_tokens":906,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":324,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":906,"tokens_out":77,"duration_ms":3589,"temperature":1.0,"reasoning_tokens":324,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T21:58:21.229815+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same long-horizon hide-and-restore block tasks while deliberately corrupting or swapping segmentation masks and object IDs mid-episode; if success collapses while the causal graph still receives those broken identities, the claim that the memory structures themselves are doing the decisive work is falsified.","supporting_citations":[],"review_version":1}