{"id":"aa2097a7-8768-4eb5-aa46-ed5f03434b98","arxiv_id":"2602.06033","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Training VLMs through interaction (GRPO) does not yield generalizable physical intuitions beyond within-task performance, matching—not exceeding—supervised fine-tuning.","lead":"This paper tests whether vision-language models (VLMs) can learn generalizable intuitive physics by interacting with a simulated block-tower environment through reinforcement learning. It finds that both interactive (GRPO) and non-interactive (SFT) training reach ceiling on their trained task but neither generalizes to related physical tasks or real images.","discovery_kind":"replication","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'interaction' manipulation is one-step GRPO with no environment state feedback, so the broad claim that interaction fails to produce physical intuition is not actually tested.","rationale":"The reader's weakest assumption is the same one I identify: the operationalization of 'interaction' as one-step GRPO is the load-bearing premise. Both the reader and I see that this scope limitation is acknowledged in the Discussion but not reflected in the Abstract/Conclusion. Because the paper's own Conclusion narrows the claim to 'one-step reinforcement learning with GRPO,' the empirical core is sound within that scope, and the reader's CONDITIONAL verdict already accounts for this. I do not think the verdict should move: the concern is real but does not overturn the paper's carefully stated narrow result. I agree with the reader's assessment about the second issue (shared task properties asserted, not measured) but regard the one-step interaction issue as the primary load-bearing concern. The proposed concrete test — running an actual multi-step interaction loop — would directly settle whether the broad interaction hypothesis survives, and would also clarify what the paper's negative result does and does not establish.","tokens_in":20405,"tokens_out":5527,"duration_ms":65871,"concrete_test":"Train Qwen3-VL-8B on the x-only top block task with a genuine multi-step interaction loop: let the model propose an integer action, render the resulting tower in ThreeDWorld, feed that new image back to the model, and allow up to 3 correction steps before the episode ends. Train with GRPO over the full trajectory (or a trajectory-level policy-gradient variant). Then evaluate zero-shot on x-only side block and binary stability. If multi-step interaction improves transfer beyond the one-step GRPO result, the paper's broad claim is falsified; if it also fails to transfer, the concern is resolved and the conclusion can be extended to a wider class of interaction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the Abstract and Title — 'learning from interaction fails to produce models with generalizable physical intuitions' — is broader than what the experiments manipulate. In Section 3.3–3.4, GRPO is applied as a one-shot bandit: the model receives one image, emits an integer or two integers, and gets a scalar reward computed from a static distance-to-optimal function. There is no state transition: the model never observes the tower that would result from its proposed action, there are no multi-step episodes, and no opportunity to correct or explore consequences. This is not the interactive, action-outcome learning loop that motivates the cognitive-science hypothesis (Gibson, Piaget) and that the Introduction invokes. The Discussion explicitly acknowledges this limitation ('We also only investigated 1-step interactions with the environment. It remains possible that advantages of interaction only surface when models are able to interact with their environment over long state-action sequences.'), but the Abstract and Conclusion do not carry this qualifier. Consequently, the negative result is fully compatible with the possibility that multi-step or embodied interaction does yield transferable physical knowledge; that hypothesis has not been tested. A secondary but related weakness is that the four tasks are asserted to share 'visual statistics and physical principles' without measuring those quantities, so the generalization failures could partly reflect task mismatch (different displacements, layouts, and action spaces) rather than the absence of physical intuition. The within-task results are clear and the paper is honest about its scope, but the headline claim outruns the operationalization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether vision-language models can acquire generalizable intuitive physics from interaction, hypothesizing that an interactive training condition (GRPO) should outperform a non-interactive one (SFT). The authors construct four block-tower tasks from ThreeDWorld, train Qwen3-VL-8B (and ablate on Qwen2.5-VL-7B and Qwen3-VL-32B) with GRPO, GSPO, or SFT, and evaluate within-task performance, cross-task transfer, transfer to real images from Lerer et al. (2016), and linear decodability of physical quantities. They find that both GRPO and SFT reach near-ceiling performance on the training task, that neither method yields reliable generalization across the full task set or to real images, and that physical quantities are highly decodable from activations in base and post-trained models. The paper concludes that one-step interaction does not confer a general advantage for learning physical intuitions and that the models learn task-specific shortcuts.","tokens_in":20619,"tokens_out":4663,"duration_ms":56581,"significance":"If the claim is restricted to the operationalization studied, the paper provides a useful, clearly described negative result: a one-step GRPO reward-optimization loop over textual actions does not improve cross-task generalization of physical intuitions in VLMs relative to SFT. Strengths include the breadth of ablations (three model families/sizes, two RL algorithms, multiple ranks, training horizons, joint/blocked training), the use of an established physics engine and real-image benchmark, and the decodability analysis which speaks to a competence/performance distinction. The main limitation is that the abstract and title state a much broader conclusion than the experiments support, because the 'interaction' manipulation contains no state transitions or multi-step episodes. Nevertheless, with appropriate qualification the study is a meaningful contribution to the empirical literature on post-training VLMs for intuitive physics.","major_comments":[{"comment":"The central negative claim is broader than the manipulation. 'Interaction' is implemented as one-step GRPO: the model sees one image, emits an integer or two integers, and receives a scalar reward computed from a static distance-to-optimal function. There is no state transition, no observation of the resulting tower, and no multi-step episode, so this is a bandit-style reward optimization rather than the interactive action-outcome loop motivated in the Introduction. The Discussion concedes this ('We also only investigated 1-step interactions...'), but the Abstract and Conclusion state without qualification that interaction 'fails to produce models with generalizable physical intuitions.' Please qualify the title, abstract, and conclusion to 'one-step interaction of the type studied here' and soften the claim to absence of evidence in this setting.","section":"Abstract / §3.3–3.4, §5"},{"comment":"The null claim 'no model reliably generalizes' rests on point estimates from what appear to be single runs, with no confidence intervals, significance tests, or seed variation reported for the main evaluations. For a negative result, uncertainty quantification is needed; otherwise the word 'reliably' is not supported. In addition, the premise that the four tasks 'share visual statistics and physical principles' is asserted rather than measured. Without a demonstration that the transfer tasks are close in the relevant feature space, the generalization failures could partly reflect task dissimilarity. I suggest adding multiple seeds with CIs and a positive control or a quantitative similarity measure between the task distributions.","section":"§4.2, Fig. 2, Table A.2"},{"comment":"The conclusion that the models learn 'non-general shortcuts' is not directly established. The decodability analysis shows that tower stability and x-offset are linearly decodable in the base model and remain so after training, but it does not show that the post-trained models' output decisions are caused by those representations. The attention analysis is acknowledged to be inconclusive. The shortcut interpretation should be presented as a hypothesis consistent with the negative transfer results, not as a demonstrated mechanism.","section":"§4.4, §6"}],"minor_comments":[{"comment":"For the x-only side block task, the prompt says to move the block 'to the most stable position on the top of the tower', but the action space is a single integer along x. Please clarify how the vertical placement is handled or whether the reward is computed only from the x-distance.","section":"§3.2"},{"comment":"The text reports 'accuracies' of 0.624, 0.503, and −0.264 for a binary task. Since the reward scheme includes −1 for illegal answers, these are mean rewards, not accuracies. Please relabel or explain the scoring in the text.","section":"§4.2"},{"comment":"The first row header 'x-only binary stability' is inconsistent with the task names used elsewhere; it should presumably be 'binary stability top block'.","section":"Table A.2"},{"comment":"Please report missing training details: batch size M, learning rate and schedule, number of seeds, and the parser used to map text completions to legal integers. This is important for reproducing the near-ceiling rewards and for assessing variability.","section":"§3.3 / Appendix A.5"},{"comment":"Typo: 'its'' should be 'its' in 'from its'' fine-tuning task'.","section":"Appendix A.9.1"}],"recommendation":"major_revision","confidential_remarks":"The empirical work is solid and the negative result is worth publishing, but the title/abstract currently overclaim relative to the one-step operationalization of interaction. The revision should be straightforward: constrain the claim, add uncertainty quantification, and perhaps add a positive control for task relatedness. I would be supportive after that revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid empirical contribution, but only for the narrow version of interaction it actually tests. They compare GRPO (one-step RL) against SFT for teaching VLMs to stabilize block towers. Within-task performance reaches ceiling for both, generalization to related tasks is weak, and interaction gives no edge. The comparison is new, the ablations are extensive (multiple models, GSPO, longer horizons, rank, reasoning, joint training), and the decodability analysis is a nice addition: physical quantities are already decodable in the base model, yet that competence never turns into reliable transfer. That supports a shortcut-learning interpretation. The paper is also honest about its limitations in the Discussion, explicitly noting that only 1-step interactions were investigated and that multi-step or embodied interaction might still help.\n\nWhere it gets soft is the gap between the headline and the operationalization. The abstract and title say learning from interaction 'fails to produce generalizable physical intuitions.' But their interaction is a one-step bandit: the model sees an image, outputs an integer or two, and receives a scalar reward computed from a static distance function. There is no state transition, no multi-step episode, no observation of the tower that would result from the proposed action. That is not the feedback loop Gibson and Piaget were talking about. So the negative result rules out one cheap version of interaction, not interaction per se. The authors know this, but the abstract and conclusion should carry the qualifier. A second, more minor issue: the claim that the four tasks share 'visual statistics and physical principles' is asserted rather than measured. Some of the generalization failures could be task mismatch (different displacement ranges, different action spaces) rather than absence of physical intuition. Finally, no code or data are released, and several reporting details (seeds, error bars on the main numbers, exact dataset splits) are missing, so independent replication is currently not practical.\n\nWho is this for? People working on VLM post-training, intuitive physics, or RL for perception. They will get a clean negative baseline and a useful warning: don't expect one-shot GRPO to cure shortcut learning on simple physics tasks. The empirical core is sound for the narrow operationalization. It deserves a serious referee, mainly to push the authors to align the claims with the evidence and to release artifacts. I'd recommend engaging with it.","headline":"Clean, controlled negative result for one-step RL on VLM physics tasks, but the title and abstract claim about 'interaction' is wider than the one-shot bandit they actually trained.","tokens_in":21264,"tokens_out":2012,"would_cite":true,"duration_ms":29953,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Learning from interaction lets a vision language model master the exact task it trains on, but the paper finds that one-step reinforcement learning fails to produce physical intuitions that transfer to related tasks or real block-tower imag","keywords":["intuitive physics","vision language models","reinforcement learning","GRPO","generalization","shortcut learning","tower stability","decodability"],"falsifier":"The cheapest decisive check is same-task, different-dataset transfer: train on x-only top block and test on x-only side block, where the action (one integer, move the block to center) and objective are identical. The paper reports essentially no transfer (mean reward −0.152, below the null baseline), so any interaction-based training that produces substantial positive reward on this pair — or a multi-step variant that lifts accuracy on the real wooden-tower images to near the human mean of about 0.8 — would overturn the central negative claim.","tokens_in":20180,"feed_emoji":"🧱","tokens_out":13810,"duration_ms":128786,"temperature":0.7,"pith_summary":"The paper tests a hypothesis from developmental psychology: that a model would acquire genuinely generalizable physical intuitions by interacting with a physics environment, the way children learn by playing with objects. To test it, the authors post-train an 8-billion-parameter vision language model on block-tower tasks using one-step reinforcement learning (Group-Relative Policy Optimization, GRPO), where the model proposes an action and receives a reward computed by a physics engine, and compare it with supervised fine-tuning on optimal examples. Both methods drive models to near-ceiling performance on the exact task they train on — mean rewards around 20 out of 20 and accuracy near 0.97 on the binary stability judgment. But neither method yields models that reliably generalize to related tasks, even when those tasks share the same visual statistics and physical principles, and every model lands below the human average on real photos of wooden block towers. The paper's conclusion is that one-step interaction, as implemented here, does not give vision language models generalizable physical intuitions: the models learn task-specific shortcuts, and the physical quantities they need are already present in their internal representations but not used outside the trained distribution.","feed_headline":"One-step RL teaches vision models the task, not the physics","feed_subtitle":"Trial-and-error models master tower-stability tasks yet transfer to new setups and real photos no better than supervised ones.","key_machinery":"The carrying mechanism is one-step reinforcement learning with Group-Relative Policy Optimization (GRPO) over the model's textual answer. The model sees a rendered tower image and outputs one or two integers; a physics engine simulates the placement and returns a Gaussian reward peaking at 20 for a stable tower, negative for unstable or illegal answers. The interaction is single-shot: the model never observes the resulting state. Supervised fine-tuning on optimal examples is the non-interactive comparison. The four tasks (binary stability and x-only on top-block data; x-only and x-y on side-block data) are built to share visual statistics and physical principles, so transfer failure is meant","core_discovery":"The paper's discovery is a null result with a specific shape. GRPO-trained models reach near-ceiling performance on their training task (reward 19.999 of 20; accuracy 0.969 on binary), matching supervised fine-tuning. Yet no model reliably generalizes: the model trained on x-only top block scores 0.624 on binary-stability top block and −0.152 on the same-action x-only side block; real-photo transfer tops out at 0.6, below the human mean of about 0.8. Stability and x-offset are decodable from activations at all layers before and after training, so the bottleneck is unused rather than missing representation — the authors say shortcut learning. The pattern holds across 7B, 8B, and 32B models, a","pith_inferences":["The paper's own Discussion limits the scope: only one-step interactions were tested. A multi-step regime in which the model acts, observes the new tower state, and acts again over a long horizon remains the untested version of the interaction hypothesis — the negative result does not rule it out.","The decodability finding invites a targeted intervention the paper does not run: if the x-offset is linearly present in activations, a probe-guided reward that forces the policy to be consistent with the decoded quantity might convert competence into transfer, distinguishing 'represented but unused' from 'rigidly task-bound.'","The multi-task results hint at a different lever: GRPO trained sequentially on two tasks kept both skills, while blocked SFT forgot the first task and joint SFT preserved both. This suggests task diversity — not interaction per se — may be what preserves or builds transferable skills, a claim the paper does not make."],"forward_implications":["If the result holds, one-step online reinforcement learning offers no generalization advantage over supervised fine-tuning for visual physical reasoning: both ceiling in-distribution, both fail out-of-distribution.","Failure to transfer between tasks that share the load-bearing variable (notably x-only top block to binary-stability top block, both determined by the top block's x-offset) implies the trained models rely on task-specific output shortcuts rather than the underlying physical quantity.","Since stability and offset are decodable from activations at all layers in the base and post-trained models, representation is not the bottleneck; performance under distribution shift is — so additional fine-tuning data or steps on the same task will not unlock generalization.","The null result is stable across model sizes (7B, 8B, 32B), across GRPO and an alternative RL algorithm, and across longer training (up to 48,000 steps, which only overfits), so it is not specific to one architecture or one training recipe.","For real-world use, post-trained models remain below human-level judgment of tower stability on real images (best accuracy 0.6 vs. human mean ~0.8), so synthetic physics fine-tuning does not yet transfer to natural visual physics."],"fun_headline_variants":["Trial-and-error AI learns the task, not physics","RL mastery doesn't teach vision models physics","No physical intuition from interaction alone","Why RL-trained models still fail physics transfer","Interaction fails to build transferable physics in AI"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that a single text answer scored by a physics engine counts as the kind of interaction that should build intuitive physics — the model never sees the tower after its action, so the negative result is established only for one-shot textual actions, the scope the authors themselves flag at the end of the paper.","fun_headline_variants_meta":{"raw":{"variants":["Trial-and-error AI learns the task, not physics","RL mastery doesn't teach vision models physics","No physical intuition from interaction alone","Why RL-trained models still fail physics transfer","Interaction fails to build transferable physics in AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000143,"raw_usage":{"total_tokens":976,"prompt_tokens":681,"completion_tokens":295,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":425,"completion_tokens_details":{"reasoning_tokens":227}},"tokens_in":425,"tokens_out":295,"duration_ms":3940,"temperature":1.0,"reasoning_tokens":227,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T03:59:44.046887+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The cheapest decisive check is same-task, different-dataset transfer: train on x-only top block and test on x-only side block, where the action (one integer, move the block to center) and objective are identical. The paper reports essentially no transfer (mean reward −0.152, below the null baseline), so any interaction-based training that produces substantial positive reward on this pair — or a multi-step variant that lifts accuracy on the real wooden-tower images to near the human mean of about 0.8 — would overturn the central negative claim.","supporting_citations":[],"review_version":1}