Pith. sign in

REVIEW 3 major objections 5 minor 23 references

Learning from interaction lets a vision language model master the exact task it trains on, but the paper finds that one-step reinforcement learning fails to produce physical intuitions that transfer to related tasks or real block-tower imag

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 03:59 UTC pith:LL3SZFVD

load-bearing objection Clean, controlled negative result for one-step RL on VLM physics tasks, but the title and abstract claim about 'interaction' is wider than the one-shot bandit they actually trained. the 3 major comments →

arxiv 2602.06033 v2 pith:LL3SZFVD submitted 2026-02-05 cs.LG

Can Vision Language Models Learn Intuitive Physics from Interaction?

Luca M. Schulze Buschoff , Konstantinos Voudouris , Can Demircan , Eric Schulz This is my paper
classification cs.LG
keywords intuitive physicsvision language modelsreinforcement learningGRPOgeneralizationshortcut learningtower stabilitydecodability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests a hypothesis from developmental psychology: that a model would acquire genuinely generalizable physical intuitions by interacting with a physics environment, the way children learn by playing with objects. To test it, the authors post-train an 8-billion-parameter vision language model on block-tower tasks using one-step reinforcement learning (Group-Relative Policy Optimization, GRPO), where the model proposes an action and receives a reward computed by a physics engine, and compare it with supervised fine-tuning on optimal examples. Both methods drive models to near-ceiling performance on the exact task they train on — mean rewards around 20 out of 20 and accuracy near 0.97 on the binary stability judgment. But neither method yields models that reliably generalize to related tasks, even when those tasks share the same visual statistics and physical principles, and every model lands below the human average on real photos of wooden block towers. The paper's conclusion is that one-step interaction, as implemented here, does not give vision language models generalizable physical intuitions: the models learn task-specific shortcuts, and the physical quantities they need are already present in their internal representations but not used outside the trained distribution.

Core claim

The paper's discovery is a null result with a specific shape. GRPO-trained models reach near-ceiling performance on their training task (reward 19.999 of 20; accuracy 0.969 on binary), matching supervised fine-tuning. Yet no model reliably generalizes: the model trained on x-only top block scores 0.624 on binary-stability top block and −0.152 on the same-action x-only side block; real-photo transfer tops out at 0.6, below the human mean of about 0.8. Stability and x-offset are decodable from activations at all layers before and after training, so the bottleneck is unused rather than missing representation — the authors say shortcut learning. The pattern holds across 7B, 8B, and 32B models, a

What carries the argument

The carrying mechanism is one-step reinforcement learning with Group-Relative Policy Optimization (GRPO) over the model's textual answer. The model sees a rendered tower image and outputs one or two integers; a physics engine simulates the placement and returns a Gaussian reward peaking at 20 for a stable tower, negative for unstable or illegal answers. The interaction is single-shot: the model never observes the resulting state. Supervised fine-tuning on optimal examples is the non-interactive comparison. The four tasks (binary stability and x-only on top-block data; x-only and x-y on side-block data) are built to share visual statistics and physical principles, so transfer failure is meant

Load-bearing premise

The load-bearing premise is that a single text answer scored by a physics engine counts as the kind of interaction that should build intuitive physics — the model never sees the tower after its action, so the negative result is established only for one-shot textual actions, the scope the authors themselves flag at the end of the paper.

What would settle it

The cheapest decisive check is same-task, different-dataset transfer: train on x-only top block and test on x-only side block, where the action (one integer, move the block to center) and objective are identical. The paper reports essentially no transfer (mean reward −0.152, below the null baseline), so any interaction-based training that produces substantial positive reward on this pair — or a multi-step variant that lifts accuracy on the real wooden-tower images to near the human mean of about 0.8 — would overturn the central negative claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the result holds, one-step online reinforcement learning offers no generalization advantage over supervised fine-tuning for visual physical reasoning: both ceiling in-distribution, both fail out-of-distribution.
  • Failure to transfer between tasks that share the load-bearing variable (notably x-only top block to binary-stability top block, both determined by the top block's x-offset) implies the trained models rely on task-specific output shortcuts rather than the underlying physical quantity.
  • Since stability and offset are decodable from activations at all layers in the base and post-trained models, representation is not the bottleneck; performance under distribution shift is — so additional fine-tuning data or steps on the same task will not unlock generalization.
  • The null result is stable across model sizes (7B, 8B, 32B), across GRPO and an alternative RL algorithm, and across longer training (up to 48,000 steps, which only overfits), so it is not specific to one architecture or one training recipe.
  • For real-world use, post-trained models remain below human-level judgment of tower stability on real images (best accuracy 0.6 vs. human mean ~0.8), so synthetic physics fine-tuning does not yet transfer to natural visual physics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own Discussion limits the scope: only one-step interactions were tested. A multi-step regime in which the model acts, observes the new tower state, and acts again over a long horizon remains the untested version of the interaction hypothesis — the negative result does not rule it out.
  • The decodability finding invites a targeted intervention the paper does not run: if the x-offset is linearly present in activations, a probe-guided reward that forces the policy to be consistent with the decoded quantity might convert competence into transfer, distinguishing 'represented but unused' from 'rigidly task-bound.'
  • The multi-task results hint at a different lever: GRPO trained sequentially on two tasks kept both skills, while blocked SFT forgot the first task and joint SFT preserved both. This suggests task diversity — not interaction per se — may be what preserves or builds transferable skills, a claim the paper does not make.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper asks whether vision-language models can acquire generalizable intuitive physics from interaction, hypothesizing that an interactive training condition (GRPO) should outperform a non-interactive one (SFT). The authors construct four block-tower tasks from ThreeDWorld, train Qwen3-VL-8B (and ablate on Qwen2.5-VL-7B and Qwen3-VL-32B) with GRPO, GSPO, or SFT, and evaluate within-task performance, cross-task transfer, transfer to real images from Lerer et al. (2016), and linear decodability of physical quantities. They find that both GRPO and SFT reach near-ceiling performance on the training task, that neither method yields reliable generalization across the full task set or to real images, and that physical quantities are highly decodable from activations in base and post-trained models. The paper concludes that one-step interaction does not confer a general advantage for learning physical intuitions and that the models learn task-specific shortcuts.

Significance. If the claim is restricted to the operationalization studied, the paper provides a useful, clearly described negative result: a one-step GRPO reward-optimization loop over textual actions does not improve cross-task generalization of physical intuitions in VLMs relative to SFT. Strengths include the breadth of ablations (three model families/sizes, two RL algorithms, multiple ranks, training horizons, joint/blocked training), the use of an established physics engine and real-image benchmark, and the decodability analysis which speaks to a competence/performance distinction. The main limitation is that the abstract and title state a much broader conclusion than the experiments support, because the 'interaction' manipulation contains no state transitions or multi-step episodes. Nevertheless, with appropriate qualification the study is a meaningful contribution to the empirical literature on post-training VLMs for intuitive physics.

major comments (3)
  1. [Abstract / §3.3–3.4, §5] The central negative claim is broader than the manipulation. 'Interaction' is implemented as one-step GRPO: the model sees one image, emits an integer or two integers, and receives a scalar reward computed from a static distance-to-optimal function. There is no state transition, no observation of the resulting tower, and no multi-step episode, so this is a bandit-style reward optimization rather than the interactive action-outcome loop motivated in the Introduction. The Discussion concedes this ('We also only investigated 1-step interactions...'), but the Abstract and Conclusion state without qualification that interaction 'fails to produce models with generalizable physical intuitions.' Please qualify the title, abstract, and conclusion to 'one-step interaction of the type studied here' and soften the claim to absence of evidence in this setting.
  2. [§4.2, Fig. 2, Table A.2] The null claim 'no model reliably generalizes' rests on point estimates from what appear to be single runs, with no confidence intervals, significance tests, or seed variation reported for the main evaluations. For a negative result, uncertainty quantification is needed; otherwise the word 'reliably' is not supported. In addition, the premise that the four tasks 'share visual statistics and physical principles' is asserted rather than measured. Without a demonstration that the transfer tasks are close in the relevant feature space, the generalization failures could partly reflect task dissimilarity. I suggest adding multiple seeds with CIs and a positive control or a quantitative similarity measure between the task distributions.
  3. [§4.4, §6] The conclusion that the models learn 'non-general shortcuts' is not directly established. The decodability analysis shows that tower stability and x-offset are linearly decodable in the base model and remain so after training, but it does not show that the post-trained models' output decisions are caused by those representations. The attention analysis is acknowledged to be inconclusive. The shortcut interpretation should be presented as a hypothesis consistent with the negative transfer results, not as a demonstrated mechanism.
minor comments (5)
  1. [§3.2] For the x-only side block task, the prompt says to move the block 'to the most stable position on the top of the tower', but the action space is a single integer along x. Please clarify how the vertical placement is handled or whether the reward is computed only from the x-distance.
  2. [§4.2] The text reports 'accuracies' of 0.624, 0.503, and −0.264 for a binary task. Since the reward scheme includes −1 for illegal answers, these are mean rewards, not accuracies. Please relabel or explain the scoring in the text.
  3. [Table A.2] The first row header 'x-only binary stability' is inconsistent with the task names used elsewhere; it should presumably be 'binary stability top block'.
  4. [§3.3 / Appendix A.5] Please report missing training details: batch size M, learning rate and schedule, number of seeds, and the parser used to map text completions to legal integers. This is important for reproducing the near-ceiling rewards and for assessing variability.
  5. [Appendix A.9.1] Typo: 'its'' should be 'its' in 'from its'' fine-tuning task'.

Circularity Check

0 steps flagged

No significant circularity: empirical generalization study with external benchmarks; one-step interaction is an acknowledged scope limitation, not a derivation loop.

full rationale

The paper's central claim—that one-step interaction (GRPO) improves within-task performance but not cross-task generalization—is an empirical result evaluated on held-out instances and on the external Lerer et al. (2016) real-image benchmark. The reward functions (Section 3.4) are explicit and are the training objectives; reporting that models reach 19.999 mean reward on their training task is a manipulation check, not a prediction of an independently defined quantity. Cross-task generalization is not optimized by the reward, so the failure to transfer is not baked into the training loss. The one-step interaction scope is a genuine limitation and is openly acknowledged in the Discussion ('We also only investigated 1-step interactions with the environment...'), but acknowledging a limitation is not circularity. Self-citations (Schulze Buschoff et al., 2025a,b) are used as related work and to source human comparison data; they are not invoked as a uniqueness theorem, are not used to forbid alternative operationalizations, and the comparison against real images and human means is externally grounded. No fitted parameter is renamed as a prediction, and no equation reduces to its own output. The assertion that the four tasks share 'visual statistics and physical principles' is not measured, but that is a missing-evidence/correctness concern, not a circular reduction. The strongest charge available is over-generalization of the negative result beyond the tested one-step setting, which is a scope issue rather than circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical entities, forces, or conserved quantities. The load-bearing assumptions are domain assumptions about simulator labels, the fairness of the task-sharing claim, and the operationalization of interaction as one-step GRPO. Free parameters are standard post-training hyperparameters and hand-designed reward constants; the main conclusion does not hinge on a single fitted constant because ablations (r, steps, models) are provided.

free parameters (5)
  • LoRA rank r=16 = 16
    Chosen hyperparameter for all adapters. The authors ablate r=1 and r=8 in Appendix A.9.3 and show similar non-generalization, so it does not single-handedly drive the conclusion.
  • GRPO group size N=16 = 16
    Standard GRPO group size; not ablated, but does not appear to break the comparison.
  • Training steps 10,000 = 10000
    Chosen training horizon. Appendix A.9.2 tests up to 48,000 steps and finds only overfitting, so the choice is partially supported.
  • Reward function constants (20, 2, -2, -4) = 20·e^(-d²), 2·e^(-d²)-2, etc.
    Hand-designed reward shaping in Section 3.4. The GRPO within-task ceiling is largely determined by these reward functions, though the generalization comparison is not circular because the same reward is used for evaluation.
  • Camera angle, block size, image resolution 256×256 = fixed
    Fixed perceptual configuration (Section 3.1). This makes the tasks easier and may limit generalization; asserted rather than varied.
axioms (4)
  • domain assumption ThreeDWorld physics engine accurately simulates block tower stability and the 'stable' ground-truth labels are correct.
    The entire supervised and reward-based training depends on the simulator's stability labels as ground truth (Section 3.1). No independent verification against real physics is provided.
  • domain assumption The Lerer et al. (2016) dataset provides valid real-world stability labels and a fair human benchmark.
    Used as external real-image evaluation (Section 4.3). The human mean is taken from the authors' prior work; label validity is not independently audited.
  • domain assumption The four constructed tasks share sufficient 'visual statistics and physical principles' so that transfer failure indicates absent physical intuitions rather than task mismatch.
    Asserted in the Introduction and Section 4.2 (e.g., x-only and binary-stability both require x-offset). But displacement ranges differ between top-block and side-block tasks and image layouts differ, so the shared-variable claim is not quantified.
  • ad hoc to paper The GRPO implementation without KL penalty is a valid instance of 'learning through interaction'.
    The paper defines interaction as one-step GRPO (Section 3.3). This operationalization is a modeling choice, acknowledged as a limitation in Discussion.

pith-pipeline@v1.3.0-alltime-deepseek · 19853 in / 8934 out tokens · 70299 ms · 2026-08-03T03:59:44.046887+00:00 · methodology

0 comments
read the original abstract

Pre-trained vision language models do not have good intuitions about the physical world. Recent work has shown that supervised fine-tuning can improve model performance on simple physical tasks. However, fine-tuned models do not appear to learn robust physical rules that can generalize to new contexts. Based on research in cognitive science, we hypothesize that models need to interact with an environment to properly learn its physical dynamics. We train models that learn through interaction with a simulated environment using reinforcement learning. While learning from interaction allows models to improve their within-task performance, it fails to produce models with generalizable physical intuitions. We find that models trained on one task do not reliably generalize to related tasks, even if the tasks share visual statistics and physical principles, and regardless of whether the models are trained through interaction.

Figures

Figures reproduced from arXiv: 2602.06033 by Can Demircan, Eric Schulz, Konstantinos Voudouris, Luca M. Schulze Buschoff.

Figure 1
Figure 1. Figure 1: Overview of all combinations of datasets and action types. Datasets: We train models on two related datasets, one where the block on top of a tower is displaced, and one where a block is displaced on the ground next to a tower. Action types: For each dataset, we train models on two action types. Binary stability requires models to make a binary judgment on whether a given tower is stable. For the x-only ta… view at source ↗
Figure 2
Figure 2. Figure 2: Performance by test task and training task for Qwen3-VL-8B. Rows show models evaluated on a given task. Columns show models trained on a given task. The blue and orange lines show the performance of the models trained with SFT and GRPO, respectively. The grey dotted line shows the baseline for the evaluation task. Plots on the diagonal show within-task performance, meaning models are evaluated on the same … view at source ↗
Figure 3
Figure 3. Figure 3: Qwen3-VL-8B trained on all four conditions and evaluated on the real images of block towers from Lerer et al. (2016). Crucially, we find some generalization from our synthetic binary-stability top block images to real images of block towers. However, we still find that no model fine-tuned on other tasks generalizes to judging the stability of real block towers. Furthermore, we find that even the models pos… view at source ↗
Figure 4
Figure 4. Figure 4: Example images for the top block dataset. Images feature towers with 2 to 4 blocks with the top block displaced. A.1.2. SIDE BLOCK DATASET [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Example images for the side block dataset. Images feature towers with 1 to 3 blocks with a misplaced block to the side. A.1.3. LERER DATASET [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example images for the Lerer evaluation dataset. Images are real pictures of block towers with 2 to 4 blocks. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Reward function for the x-only task on the top block dataset. The red dot sits at the lower center of the block from which the reward is calculated. The first row shows the non normalized reward values. The second row shows the symmetric-log transformed reward values. The third row shows the log normalized reward values. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Reward function for the x-y task on the side block dataset. The red dot sits at the lower center of the block from which the reward is calculated. The first row shows the non normalized reward values. The second row shows the symlog transformed reward values. The third row shows the log normalized reward values. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Training logs for the SFT (left) and GRPO (right) Qwen3-VL-8B models trained on all datasets. For the SFT models, we show the log loss and a running average with a window of 25. For the GRPO models, we show the mean reward and a running average with a window of 25. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Training logs for the SFT (left) and GRPO (right) Qwen2.5-VL-7B models trained on all datasets. For the SFT models, we show the log loss and a running average with a window of 25. For the GRPO models, we show the mean reward and a running average with a window of 25. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Physical property decodability analysis. (A) shows the decodability of the ground truth binary stability for the base model, as well as models post-trained with GRPO and SFT on x-only top block and binary stability top block. (B) shows the decodability of the x-offset of the uppermost block in the top block dataset for the same set of models. A.7. Attention maps To better understand how finetuned models l… view at source ↗
Figure 12
Figure 12. Figure 12: Attention maps for the base model and the model post-trained with GRPO on x-only top block. The model is asked if a given tower is stable or not. Attention maps over the different heads are averaged in each layer. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qwen3-VL-8B model trained with GRPO, GSPO, and SFT on the x-only top block task. The model also does not generalize from its’ fine-tuning task to other related tasks. Noticeably, the model is above chance for the binary-stability top block from the get-go and improves slightly over the course of training on the related x-only top block task under all post-training regimes. 0 2000 4000 6000 8000 10000 0 0.… view at source ↗
Figure 14
Figure 14. Figure 14: Qwen3-VL-32B model trained with GRPO, GSPO, and SFT. The model is trained on the x-only top block task. Noticeably, the SFT post-trained model improves slightly over the course of training on the related binary-stability top block. In contrast, the GRPO/GSPO post-trained models do not generalize from the x-only top block task to the binary-stability top block task. 20 [PITH_FULL_IMAGE:figures/full_fig_p0… view at source ↗
Figure 15
Figure 15. Figure 15: Qwen2.5-VL-7B performance by test task and training task. Rows show models evaluated on a given task. Columns show models trained on a given task. The blue and orange lines show the performance of the models trained with SFT and GRPO, respectively. The grey dotted line shows the baseline for the evaluation task. Plots on the diagonal show within-task performance, meaning models are evaluated on the same t… view at source ↗
Figure 16
Figure 16. Figure 16: Generalization after additional post-training for Qwen2.5-VL-7B. The blue line on the bottom left shows the test performance of the SFT-trained model and the orange line on the bottom right shows that of the GRPO-trained model. Both models were trained on the x-only top block task. For each checkpoint, we take the model trained on this task up to that checkpoint and train it with SFT on the binary stabili… view at source ↗
Figure 17
Figure 17. Figure 17: Performance for a longer horizon. We show the same plot as 15 but with results for up to 48K training steps for the Qwen2.5- VL-7B GRPO models. Performance is shown for each combination of test task and training task. Rows show models evaluated on a given task. Columns show models trained on a given task. The blue and orange lines show the performance of the models trained with SFT and GRPO, respectively.… view at source ↗
Figure 18
Figure 18. Figure 18: Ablation checks. All models are trained on x-only top block but they have lower ranks (1 and 8 compared to 16) or are trained without fine-tuning of the vision encoder or they are trained with reasoning (a larger generation length). All ablations learn to perform well on the task they are trained on (shown in the second row). However, all models fail to generalize to other related tasks — just as the stan… view at source ↗
Figure 19
Figure 19. Figure 19: Blocked joint training. The model was first trained for 10.000 steps on x-only side block. It is then trained for 10.000 steps on binary-stability top block and performance is shown below over these 10.000 steps. The model forgets the proper formatting of responses for the initial x-only side block task (see continuous line on the left), but legal answers still perfectly solve the task (see dotted line on… view at source ↗
Figure 20
Figure 20. Figure 20: All models trained with GRPO, GSPO, and SFT on the x-only top block task, evaluated on the real block towers from Lerer et al. (2016). While we still found some generalization from post-training on our x-only top block task to our binary-stability top block task, the models do not generalize to this external task. Error bars show 95% confidence intervals. We also test whether any model Qwen2.5-VL-7B model… view at source ↗
Figure 21
Figure 21. Figure 21: Qwen2.5-VL-7B models trained on all four tasks, evaluated on the real block towers from Lerer et al. (2016). We again find that no model performs well on this task — even the model trained on our similar binary-stability top block task. Error bars show 95% confidence intervals. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_21.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 15 linked inside Pith

  1. [2]

    We first test whether the model has learned some generalizable understanding after all, that would lead to faster supervised fine-tuning (see Section A.9.1)

    shows no trace of generalization. We first test whether the model has learned some generalizable understanding after all, that would lead to faster supervised fine-tuning (see Section A.9.1). We then test a number of ablations to test whether other parameter combinations could lead to a better generalizing model (see Section A.9.3). -1 -0.5 0 0.5 1.0 bina...

    2000

  2. [8]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model

    Hu, J., Zhang, Y ., Han, Q., Jiang, D., Zhang, X., and Shum, H.-Y . Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290,

    Pith/arXiv arXiv

  3. [10]

    S., and Lin, M

    Liu, Z., Chen, C., Li, W., Qi, P., Pang, T., Du, C., Lee, W. S., and Lin, M. Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783,

    Pith/arXiv arXiv

  4. [14]

    and Spaulding, S

    Shapiro, L. and Spaulding, S. Embodied Cognition. In Zalta, E. N. and Nodelman, U. (eds.),The Stanford Encyclope- dia of Philosophy. Metaphysics Research Lab, Stanford University, Summer 2025 edition,

    2025

  5. [16]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  6. [17]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y ., Zuo, X., Yue, Y ., Dai, W., Fan, T., Liu, G., Liu, L., et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Pith/arXiv arXiv

  7. [18]

    Group sequence policy optimization.arXiv preprint arXiv:2507.18071,

    Zheng, C., Liu, S., Li, M., Chen, X.-H., Yu, B., Gao, C., Dang, K., Liu, Y ., Men, R., Yang, A., et al. Group sequence policy optimization.arXiv preprint arXiv:2507.18071,

    Pith/arXiv arXiv

  8. [21]

    17 Can vision language models learn intuitive physics from interaction? A.6. Decoding analysis For the top block dataset, we train linear probes on the representation of the model at each layer to predict the binary stability of a tower and the x-offset of the top block from those representations. Since the image tokens appear before the text tokens, the ...

    2000

  9. [23]

    All ablations learn to perform well on the task they are trained on (shown in the second row)

    or are trained without fine-tuning of the vision encoder or they are trained with reasoning (a larger generation length). All ablations learn to perform well on the task they are trained on (shown in the second row). However, all models fail to generalize to other related tasks — just as the standard model we used throughout our experiments (last column)....

    2000

  10. [25]

    16 Can vision language models learn intuitive physics from interaction? A.5.2. QWEN2.5-VL-7B 10 8 6 4 2 0 2 Log train loss SFT binary stability top block Log loss Running average (N=25) 0.5 0.0 0.5 1.0 Train reward GRPO binary stability top block Reward Running average (N=25) 10 8 6 4 2 0 2 Log train loss SFT x only top block 5 0 5 10 15 20Train reward GR...

    2000

  11. [600]

    You can think about the problem for as long as you’d like

    Provide your reasoning between <think> and < /think>. You can think about the problem for as long as you’d like. While thinking, you should robustly verify your solution. Return your final answer between<answer> </answer>. 15 Can vision language models learn intuitive physics from interaction? A.5. Training logs A.5.1. QWEN3-VL-8B 20 15 10 5 0 5 Log train...

    2000

  12. [1945]

    Do generative video models learn physical principles from watching videos?arXiv preprint arXiv:2501.09038,

    Motamed, S., Culp, L., Swersky, K., Jaini, P., and Geirhos, R. Do generative video models learn physical principles from watching videos?arXiv preprint arXiv:2501.09038,

    Pith/arXiv arXiv

  13. [1991]

    On the general- ization of sft: A reinforcement learning perspective with reward rectification.arXiv preprint arXiv:2508.05629,

    Wu, Y ., Zhou, Y ., Ziheng, Z., Peng, Y ., Ye, X., Hu, X., Zhu, W., Qi, L., Yang, M.-H., and Yang, X. On the general- ization of sft: A reinforcement learning perspective with reward rectification.arXiv preprint arXiv:2508.05629,

    arXiv

  14. [1995]

    H., and Krishnan, R

    Balazadeh, V ., Ataei, M., Cheong, H., Khasahmadi, A. H., and Krishnan, R. G. Synthetic vision: Training vision- language models to understand physics.arXiv preprint arXiv:2412.08619,

    arXiv

  15. [1998]

    M., Wong, C., Feng, J., Wei, M., and Tenenbaum, J

    Collins, K. M., Wong, C., Feng, J., Wei, M., and Tenenbaum, J. B. Structured, flexible, and robust: benchmarking and improving large language models towards more human- like behavior in out-of-distribution reasoning tasks.arXiv preprint arXiv:2205.05718,

    Pith/arXiv arXiv

  16. [2007]

    M., Akata, E., Bethge, M., and Schulz, E

    Schulze Buschoff, L. M., Akata, E., Bethge, M., and Schulz, E. Visual cognition in multimodal large language models. Nature Machine Intelligence, pp. 1–11, 2025a. Schulze Buschoff, L. M., V oudouris, K., Akata, E., Bethge, M., Tenenbaum, J. B., and Schulz, E. Testing the limits of fine-tuning for improving visual cognition in vision lan- guage models. InF...

    Pith/arXiv arXiv

  17. [2012]

    K., ´Eltet˝o, N., et al

    Binz, M., Akata, E., Bethge, M., Br¨andle, F., Callaway, F., Coda-Forno, J., Dayan, P., Demircan, C., Eckstein, M. K., ´Eltet˝o, N., et al. Centaur: a foundation model of human cognition.arXiv preprint arXiv:2410.20268,

    Pith/arXiv arXiv

  18. [2016]

    Offline rein- forcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643,

    Levine, S., Kumar, A., Tucker, G., and Fu, J. Offline rein- forcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643,

    Pith/arXiv arXiv 2005

  19. [2020]

    V ., Levine, S., and Ma, Y

    Chu, T., Zhai, Y ., Yang, J., Tong, S., Xie, S., Schuurmans, D., Le, Q. V ., Levine, S., and Ma, Y . Sft memorizes, rl generalizes: A comparative study of foundation model post-training.arXiv preprint arXiv:2501.17161,

    Pith/arXiv arXiv

  20. [2021]

    Threedworld: A platform for inter- active multi-modal physical simulation.arXiv preprint arXiv:2007.04954,

    Gan, C., Schwartz, J., Alter, S., Mrowca, D., Schrimpf, M., Traer, J., De Freitas, J., Kubilius, J., Bhandwaldar, A., Haber, N., et al. Threedworld: A platform for inter- active multi-modal physical simulation.arXiv preprint arXiv:2007.04954,

    Pith/arXiv arXiv 2007

  21. [2023]

    Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S

    URL http://github.com/unslothai/ unsloth. Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q. Parameter-efficient fine-tuning for large models: A com- prehensive survey.arXiv preprint arXiv:2403.14608,

    Pith/arXiv arXiv

  22. [2024]

    J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685,

    Pith/arXiv arXiv

  23. [2025]

    G., Slater, B., Teˇsi´c, M., Prunty, J., V oudouris, K., and Cheke, L

    Mecattaf, M. G., Slater, B., Teˇsi´c, M., Prunty, J., V oudouris, K., and Cheke, L. G. A little less conversation, a little more action, please: Investigating the physical common- sense of llms in a 3d embodied environment.arXiv preprint arXiv:2410.23242,

    Pith/arXiv arXiv