Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Frozen vision-language-action models keep past frames readable everywhere, but mostly as redundant copies of the present, and only push them into actions as a fallback when the current view fails.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 02:56 UTC pith:W6QV53YL

load-bearing objection Solid open-loop mechanistic audit of VLA history: A4≈0 redundancy, L6 cutoff, and architecture-conditional fallback/standing steerability are well supported within scope. the 3 major comments →

arxiv 2607.03372 v1 pith:W6QV53YL submitted 2026-07-03 cs.CV

Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History

Chih-Ting Liao , Xin Cao This is my paper
classification cs.CV
keywords vision-language-action modelstemporal historylinear probingcausal interchangememory deploymentfrozen policiesrobotic manipulationsteerability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks what frozen multi-frame robot policies already do with the visual history they are given, before anyone adds extra memory modules. Using layer-wise linear probes and causal swaps of history activations on three models from two families, it finds a clean split: past-frame content stays decodable at every depth, yet almost none of that content is unique beyond the current frame. History only moves the predicted action when the present frame is nearly obliterated, and that dependence is cut off by mid-network. Encoding of unique history is the same across architectures, but deployment flips: under the same occlusion one family leans harder on history while the other leans less, and only the latter can be steered by re-injecting history content. The practical lesson is that memory work should inject information that cannot be reconstructed from the present, not simply more of the same history.

Core claim

Across three frozen VLAs in two architecture families, past-frame content is linearly decodable throughout the network, but information unique to history after residualizing against the current frame is essentially zero. History is causally deployed into the action only under near-total current-frame loss and is severed by the middle of the network. Encoding is identical, yet deployment and content-steerability flip with architecture: one regime uses history as a fallback, the other as standing input, and only the standing regime can be steered by injection.

What carries the argument

The temporal-deployment audit: a training-free open-loop procedure that reports presence (linear decode of the past frame), uniqueness (residual decode after removing what the current frame already explains), causal deploy gap and mid-network cutoff via interchange and attention knockout, the fallback-versus-standing regime sign, and an injectability gate that tests whether re-injected history content actually steers the action.

Load-bearing premise

The claim rests on residualizing previous-frame content against a linear current-frame probe, plus open-loop activation swaps on a few hundred controlled pairs, being enough to conclude that stored history carries almost no decision-relevant unique information for real closed-loop behavior.

What would settle it

Find large history-unique features (nonlinear or dictionary-learned) that, when injected before the mid-network cutoff, both repair heavy occlusion in open-loop action distance and raise success on genuinely non-Markov closed-loop tasks in both architecture families; that would overturn the redundancy-and-fallback diagnosis.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Lengthening windows or overlaying more history mostly adds redundancy unless the added signal is unique to the past.
  • Memory pathways must land before the mid-network cutoff where native history dependence is severed.
  • Whether injected history can steer actions tracks the fallback-versus-standing deployment regime, not whether history is encoded.
  • Policies trained on near-Markov demonstrations are predicted to keep treating history as a copy of the present unless the objective rewards present-irreducible information.
  • A single training-free audit can classify any frozen multi-frame VLA by regime and predict whether injection will be content-bearing.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Abstract task-progress or intent features that are not linear reads of the previous frame could still matter in closed loop even when frame-level uniqueness is near zero.
  • The reported inverse scaling (larger models rely on history less) implies foundation-scale VLAs may become more present-only and need engineered unique-memory routes.
  • Standing-use history channels may already act as writable control interfaces for temporal behavior without retraining.
  • Because heavy natural occlusion is rare in teleop data, native policies can afford redundant history; curated non-Markov benchmarks will keep overstating the value of naive longer context.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper audits how frozen multi-frame VLAs encode and use visual history along the temporal axis, complementary to prior within-frame modality analyses. Using layer-resolved linear probing (A1 present vs A4 history-unique residual) and noise-corrected interchange interventions, plus an orthogonal attention-knockout and an injectability gate, it reports a three-part dissociation across Octo-Small/Base and CronusVLA: past-frame content is linearly decodable at every depth; unique residual information beyond the current frame is near zero (A4 ceiling ≈0 over an 81-config sweep); and history is causally deployed into the action only under near-total current-frame loss, with dependence severed by mid-network (L6). Encoding is shared across families while deployment flips (fallback vs standing), and content-steerability of injected history tracks that regime. The authors package the measurements as a training-free Temporal-Deployment Audit (Algorithm 1) and argue that memory augmentation should inject present-irreducible information rather than more redundant history.

Significance. If the result holds within its stated open-loop frozen-VLA scope, this is a useful mechanistic baseline that the memory-augmentation literature has largely presupposed rather than measured. Strengths include: (i) a clean A1/A4 split that separates presence from uniqueness; (ii) causal localization via interchange with self-swap null calibration, dose-response ladder, and agreement with attention-knockout on the L6 cutoff; (iii) multi-seed, bootstrap, and permutation hardening (e.g., p=0.0004 at L4); (iv) a falsified sub-hypothesis (standing use need not encode more unique history); and (v) a reusable, training-free audit on public checkpoints with a content-hashed stimulus set and detailed reproducibility appendices. The architecture-conditional encoding-same/deployment-flips/steerability-flips pattern is a concrete, testable contribution beyond a vague “memoryless” slogan.

major comments (3)
  1. [Abstract; §4.1; Eq. (2)] Abstract and §4.1 / Eq. (2): The headline claim that “information unique to history beyond the current frame is nearly absent” is stronger than the operational definition of A4, which residualizes a linear read of the previous-frame scene against a linear current-frame probe. §4.1 correctly notes that abstract non-linear history (task progress, intent) is not fully excluded, and that deploy legs are independent of the decode target. The abstract, title framing, and design lesson should carry the same qualifier so that A4≈0 is not read as a general proof of zero decision-relevant unique memory. A short, consistent scope sentence in the abstract and conclusion would fix this without new experiments.
  2. [§3.1; §4.2; Table 2; Limitations] §3.1, §4.2, Table 2, and Limitations: The causal “fallback only under near-total occlusion” claim rests on open-loop interchange under an artificial occ_black ladder on ~250 Bridge pairs that are themselves near-Markov (natural occlusion 3/250). The L6 cutoff and fallback-vs-standing sign-flip are well supported inside that protocol, but the manuscript sometimes elevates them to a general account of when VLAs “remember” and how memory modules should be built. Please more sharply separate (a) open-loop localization facts from (b) the closed-loop / real-robot design implication, which remains a prediction (§4.5 already lists closed-loop injection as future work). This is framing, not a request to rerun the study closed-loop.
  3. [Table 3; Table 5; §4.5; Appendix G] Table 3, Table 5 (S3), and §4.5: The “architecture-conditional steerability law” is central, but the positive standing-regime evidence is thinner than the Octo fallback negatives: CronusVLA uses a different history locus (per-frame cognition feature), smaller absolute contribution scale, and S3 is reported on a held-out subset with three donors/seeds. The within-architecture sign-flip of deployment is convincing; the content-steerability contrast should be qualified as preliminary for the standing family, or strengthened with the same full-set protocol used for Octo G3/G3b/G4, so the law is not over-anchored on one positive cell.
minor comments (5)
  1. [Figure 1] Figure 1 caption and panel labels are dense; several panels compress multiple claims (e.g., 1e–f). Consider splitting the cross-architecture panels or enlarging CIs so the sign-flip is readable in print.
  2. [§3; Algorithm 1; Table 12] Notation table (Table 12) is helpful but appears late; defining A1/A4/Δ_ℓ/σ once near Algorithm 1 would reduce early-section load.
  3. [Abstract; §4.1] A4/A1 is correctly called a heuristic redundancy ratio (footnote in §4.1); avoid ratio language in the abstract where only A4≈0 is needed.
  4. [Title page; headings] Minor typos and spacing artifacts from the preprint conversion (“PRESENT BUTNOTREMEMBERED”, missing spaces in several headings) should be cleaned for the camera-ready version.
  5. [Table 1; Related Work] Table 1 positioning is useful; ensure concurrent arXiv citations (Grant 2026, Buurmeijer 2026, etc.) remain clearly marked as concurrent mechanistic work rather than established prior art if timelines are close.

Circularity Check

0 steps flagged

No significant circularity: independent empirical measurements (decode, residual A4, interchange gap, knockout, injection) do not reduce by construction to their inputs.

full rationale

The paper's central three-part dissociation is obtained from distinct, non-self-referential measurement legs on frozen open-loop checkpoints: A1 (Eq. 1) is ordinary ridge R^{2} of past-frame content from history-token activations; A4 (Eq. 2) residualizes that target against a current-frame linear probe and reports a ceiling over an 81-configuration sweep (plus shuffled-label control), which is a diagnostic bound rather than a fitted free parameter renamed as a prediction; the deploy gap Δℓ (Eq. 4) is a noise-corrected (self-swap null) interchange difference under controlled occlusion versus markov, hardened by bootstrap CIs, multi-seed sign-consistency and permutation tests, and corroborated by an orthogonal attention-knockout that recovers the same L6 cutoff; the injectability gate further tests content-bearingness and yields architecture-conditional results (inert in fallback, steering in standing). None of these quantities is defined in terms of the claimed dissociation, nor is any uniqueness theorem or ansatz imported via self-citation (authors Liao & Cao cite external VLA and interpretability literature; no load-bearing self-citation appears). The paper itself notes that A4 does not fully exclude abstract non-linear history and leaves closed-loop validation to future work, and it reports falsification of its own sub-hypothesis that standing-use architectures would encode more unique history. The audit (Algorithm 1) simply composes the same independent measurements. The derivation chain is therefore self-contained empirical observation, not circular reduction.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 3 invented entities

The central claim rests on standard mechanistic-interpretability tools plus domain assumptions about near-Markov robot data and open-loop localization. Few numeric free parameters are load-bearing; the A4 ceiling is a maximized empirical upper bound rather than a fitted constant that defines the result. Invented quantities are operational metrics and regime labels, not new physical entities.

free parameters (3)
  • ridge regularization α and 81 probe configurations
    Swept to establish A4 ceiling; best A4 reported as +0.019/+0.015/−0.011. Choice of sweep bounds the uniqueness claim but is not a single fitted constant that forces the result.
  • self-swap null floor (~0.15)
    Measured calibration subtracted from raw interchange shifts; not hand-chosen to create a deploy gap.
  • occlusion ladder thresholds (half/heavy/black/blur)
    Hand-designed degradations that induce non-Markovness; only full black yields significant gap, which is data-driven but stimulus-defined.
axioms (4)
  • domain assumption Linear probes on pooled history-token (or cognition-feature) activations are adequate to measure presence and uniqueness of past-frame content.
    Standard linear-representation hypothesis in interpretability; paper bounds non-linear abstract history but does not measure it (§4.1).
  • domain assumption Behavior-cloned robot demonstrations are near-Markov, so the current frame is nearly sufficient and history is redundant unless the current frame is corrupted.
    Stated in §3.1 Markov vs non-Markov; underpins why A4≈0 is expected and why occ_black is the stress test.
  • domain assumption Open-loop noise-corrected interchange and attention knockout on fixed pairs localize causal history deployment into the action distribution.
    Core method assumption (§3.2 Why open-loop); closed-loop is deferred.
  • standard math Activation patching / interchange and attention knockout are valid causal interventions for measuring contribution to continuous action outputs.
    Standard causal-mediation toolkit cited (Geiger, Meng, Vig, Wang et al.); applied here to diffusion-averaged 7-DoF actions.
invented entities (3)
  • A4 history-unique residual metric independent evidence
    purpose: Separate decodable past content from information not already linearly explained by the current frame.
    Operational R² residual (Eq. 2); independent_evidence true via shuffled-label control and multi-model ceiling.
  • Temporal-Deployment Audit (TDA) / Algorithm 1 independent evidence
    purpose: Reusable training-free profile of presence, uniqueness, deploy cutoff, regime, and injectability for any frozen multi-frame VLA.
    Method contribution; falsifiable by running on new checkpoints.
  • fallback vs standing deployment regimes (σ) independent evidence
    purpose: Label opposite signs of history reliance under the same occlusion and organize steerability.
    Empirical regime labels from sign of Δ under occlusion; not a new physical object.

pith-pipeline@v1.1.0-grok45 · 26048 in / 3405 out tokens · 37012 ms · 2026-07-12T02:56:40.603471+00:00 · methodology

0 comments
read the original abstract

A frozen vision-language-action model (VLA) receives recent observations at every decision step, yet prior work has focused on adding memory rather than asking how existing history is represented and used. We study this temporal axis using layer-resolved linear probing and causal interchange interventions across three VLAs from two architecture families. We find a three-part dissociation. First, past-frame content remains linearly decodable throughout the network. Second, information unique to history beyond the current frame is nearly absent, indicating that stored history is largely a redundant copy of the present. Third, history is causally deployed only when the current frame is heavily degraded, while the action readout progressively loses dependence on history through the network. Although all models encode history similarly, their deployment strategies differ: under the same occlusion, one architecture increasingly relies on history as a fallback, whereas the other relies on it less. We further introduce a training-free temporal deployment audit that distinguishes these regimes. In the fallback regime, re-injecting history neither repairs occlusion nor disambiguates actions, confirming the redundancy of the stored representation. In the other regime, the same intervention reliably steers the predicted action toward the donor history. These results show that steerability depends on how history is deployed rather than whether it is encoded. VLAs do not forget the past; they largely fail to represent it as information distinct from the present. Our findings suggest that future memory augmentation should inject information unique to the past rather than simply more history.

Figures

Figures reproduced from arXiv: 2607.03372 by Chih-Ting Liao, Xin Cao.

Figure 1
Figure 1. Figure 1: The temporal-deployment audit at a glance (Octo-Small unless noted [25], n=200). Rows group the story: encoding and the central dissociation (a,b), fallback deployment (c,d), cross-architecture (e,f). (a) De￾code R 2 (t−1) (teal) stays above 0.30 at every depth while the causal deploy gap (rose, occ black − markov, 95% band) is significant only through L4 and collapses at L6. (b) History is decodable (A1) … view at source ↗
Figure 2
Figure 2. Figure 2: Interchange noise calibration (Octo-Small [25]). The raw donor-swap shift (teal) minus the self-swap [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Supporting panels (Octo-Small unless noted [25]): (a) decode by depth, (b) per-layer interchange contribution, (c) five-seed deploy-gap hardening with a permutation test, (d) inverse scaling across model size. These expand the summary panels of [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Population and behavioral evidence (Octo-Small [25]). (a) Mean per-DoF |∆| under masking vs. swapping t−1: history influences the continuous pose dimensions throughout, not just the gripper. (b) Per-pair lost fraction of the frame across the frozen set; the distribution sits far below the near-total-occlusion band that triggers deployment, so only 3/250 pairs are naturally occluded, the fallback is seldom … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 22 linked inside Pith

  1. [1]

    Understanding intermediate layers using linear classifier probes.International Conference on Learning Representations (Workshop); arXiv:1610.01644, 2017

    Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes.International Conference on Learning Representations (Workshop); arXiv:1610.01644, 2017

    Pith/arXiv arXiv 2017

  2. [2]

    Probing classifiers: Promises, shortcomings, and advances.Computational Linguistics, 48(1), 2022

    Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances.Computational Linguistics, 48(1), 2022

    2022

  3. [3]

    Eliciting latent predictions from transformers with the tuned lens.arXiv preprint arXiv:2303.08112, 2023

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens.arXiv preprint arXiv:2303.08112, 2023. 10 Preprint

    Pith/arXiv arXiv 2023

  4. [4]

    GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Johan Bjorck, Fernando Casta ˜neda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al.π 0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Towards monosemanticity: Decomposing language models with dictionary learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023

    2023

  7. [7]

    RT-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning (CoRL), 2023. arXiv:2307.15818

    Pith/arXiv arXiv 2023

  8. [8]

    Observing and controlling features in vision-language-action models.arXiv preprint arXiv:2603.05487, 2026

    Huub Buurmeijer, Carmen Amo Alonso, Aaron Swann, and Marco Pavone. Observing and controlling features in vision-language-action models.arXiv preprint arXiv:2603.05487, 2026

    arXiv 2026

  9. [9]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. InRobotics: Science and Systems (RSS), 2023

    2023

  10. [10]

    Sparse au- toencoders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse au- toencoders find highly interpretable features in language models. InInternational Conference on Learning Representations (ICLR), 2024. arXiv:2309.08600

    Pith/arXiv arXiv 2024

  11. [11]

    Causal abstractions of neural networks

    Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  12. [12]

    Not all features are created equal: A mechanistic study of vision-language-action models.arXiv preprint arXiv:2603.19233, 2026

    Bryce Grant, Xijia Zhao, and Peng Wang. Not all features are created equal: A mechanistic study of vision-language-action models.arXiv preprint arXiv:2603.19233, 2026

    arXiv 2026

  13. [13]

    Mechanistic interpretability for steering vision-language-action models

    Bear H ¨aon, Kaylene Stocking, Ian Chuang, and Claire Tomlin. Mechanistic interpretability for steering vision-language-action models. InConference on Robot Learning (CoRL), 2025. arXiv:2509.00328

    Pith/arXiv arXiv 2025

  14. [14]

    Camps, John Bruno, Guillaume A

    Chi He, Xin Liu, Gemma M.S. Camps, John Bruno, Guillaume A. Sartoretti, and Mac Schwa- ger. Demystifying robot diffusion policies: Action memorization and a simple lookup table alternative. InInternational Conference on Learning Representations (ICLR), 2026

    2026

  15. [15]

    Con- textVLA: Vision-language-action model with amortized multi-frame context.arXiv preprint arXiv:2510.04246, 2025

    Huiwon Jang, Sihyun Yu, Heeseung Kwon, Hojin Jeon, Younggyo Seo, and Jinwoo Shin. Con- textVLA: Vision-language-action model with amortized multi-frame context.arXiv preprint arXiv:2510.04246, 2025

    arXiv 2025

  16. [16]

    DROID: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, et al. DROID: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. OpenVLA: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024. arXiv:2406.09246

    Pith/arXiv arXiv 2024

  18. [18]

    Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    CronusVLA: Towards efficient and robust manip- ulation via multi-frame vision-language-action modeling

    Hao Li, Shiyu Yang, Yang Chen, Yonghao Tian, Xuanyu Yang, Xifeng Chen, Hanqing Wang, Tai Wang, Feng Zhao, and Dahua Lin. CronusVLA: Towards efficient and robust manip- ulation via multi-frame vision-language-action modeling. InAAAI Conference on Artificial Intelligence, 2026. arXiv:2506.19816. 11 Preprint

    arXiv 2026

  20. [20]

    CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650, 2024

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Evaluating real-world robot manipulation policies in simulation.arXiv preprint arXiv:2405.05941, 2024

    Xuanlin Li, Kyle Hsu, Jiayuan Gu, Oier Mees, Karl Pertsch, Homer Walke, et al. Evaluating real-world robot manipulation policies in simulation.arXiv preprint arXiv:2405.05941, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    LIBERO: Benchmarking knowledge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  23. [23]

    Locating and editing factual associations in GPT

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  24. [24]

    Marco Molinari, Leonardo Nevali, Saharsha Navani, and Omar G. Younis. Emergent world representations in OpenVLA.arXiv preprint arXiv:2509.24559, 2025

    arXiv 2025

  25. [25]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. InRobotics: Science and Systems (RSS), 2024

    2024

  26. [26]

    Open x-embodiment: Robotic learning datasets and RT-X models.arXiv preprint arXiv:2310.08864, 2023

    Open X-Embodiment Collaboration. Open x-embodiment: Robotic learning datasets and RT-X models.arXiv preprint arXiv:2310.08864, 2023

    Pith/arXiv arXiv 2023

  27. [27]

    The linear representation hypothesis and the ge- ometry of large language models

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the ge- ometry of large language models. InInternational Conference on Machine Learning (ICML), 2024

    2024

  28. [28]

    FAST: Efficient action tokenization for vision- language-action models.arXiv preprint arXiv:2501.09747, 2025

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. FAST: Efficient action tokenization for vision- language-action models.arXiv preprint arXiv:2501.09747, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    InConference on Robot Learning (CoRL), 2025

    Physical Intelligence, Kevin Black, Noah Brown, Danny Driess, et al.π0.5: A vision-language- action model with open-world generalization. InConference on Robot Learning (CoRL), 2025. arXiv:2504.16054

    Pith/arXiv arXiv 2025

  30. [30]

    MemoryVLA: Perceptual-cognitive memory in vision- language-action models for robotic manipulation.arXiv preprint arXiv:2508.19236, 2025

    Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. MemoryVLA: Perceptual-cognitive memory in vision- language-action models for robotic manipulation.arXiv preprint arXiv:2508.19236, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    SmolVLA: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. SmolVLA: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    MemER: Scaling up memory for robot control via experience retrieval.arXiv preprint, 2025

    Ajay Sridhar, Jennifer Pan, Satvik Sharma, and Chelsea Finn. MemER: Scaling up memory for robot control via experience retrieval.arXiv preprint, 2025

    2025

  33. [33]

    Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet.Trans- former Circuits Thread, 2024

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, et al. Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet.Trans- former Circuits Thread, 2024

    2024

  34. [34]

    Learning long-context diffusion policies via past-token prediction.arXiv preprint arXiv:2505.09561, 2025

    Marcel Torne, Andy Tang, Yuejiang Liu, and Chelsea Finn. Learning long-context diffusion policies via past-token prediction.arXiv preprint arXiv:2505.09561, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Vazquez, Ulisse Mini, and Monte MacDiarmid

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without opti- mization.arXiv preprint arXiv:2308.10248, 2023

    Pith/arXiv arXiv 2023

  36. [36]

    Investigating gender bias in language models using causal mediation anal- ysis

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation anal- ysis. InAdvances in Neural Information Processing Systems (NeurIPS), 2020. 12 Preprint

    2020

  37. [37]

    BridgeData V2: A dataset for robot learning at scale

    Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen-Estruch, Quan Vuong, Andre He, et al. BridgeData V2: A dataset for robot learning at scale. InConference on Robot Learning (CoRL), 2023

    2023

  38. [38]

    Interpretability in the wild: A circuit for indirect object identification in GPT-2 small

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: A circuit for indirect object identification in GPT-2 small. In International Conference on Learning Representations (ICLR), 2023

    2023

  39. [39]

    4D-VLA: Spatiotemporal vision-language- action pretraining with cross-scene calibration

    Jiahui Zhang, Yurui Chen, Yueming Xu, Ze Huang, Yanpeng Zhou, Yu-Jie Yuan, Xinyue Cai, Guowei Huang, Xingyue Quan, Hang Xu, et al. 4D-VLA: Spatiotemporal vision-language- action pretraining with cross-scene calibration. InAdvances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  40. [40]

    Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained biman- ual manipulation with low-cost hardware. InRobotics: Science and Systems (RSS), 2023

    2023

  41. [41]

    3D-VLA: A 3d vision-language-action generative world model

    Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jianzhe Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3D-VLA: A 3d vision-language-action generative world model. InInternational Conference on Machine Learning (ICML), 2024. arXiv:2403.09631

    Pith/arXiv arXiv 2024

  42. [42]

    X-VLA: Soft-prompted transformer as scalable cross-embodiment vision- language-action model.arXiv preprint arXiv:2510.10274, 2025

    Jinliang Zheng et al. X-VLA: Soft-prompted transformer as scalable cross-embodiment vision- language-action model.arXiv preprint arXiv:2510.10274, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    TraceVLA: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies.arXiv preprint arXiv:2412.10345, 2024

    Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daum ´e III, Andrey Kolobov, Furong Huang, and Jianwei Yang. TraceVLA: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies.arXiv preprint arXiv:2412.10345, 2024. A VERIFIED NUMBERS All headline values are drawn from a single verified results log. DecodeR2...

    Pith/arXiv arXiv 2024

  44. [44]

    demonstrate layer-localized FFN value-vector steering on real-robot policies, and Molinari et al

  45. [45]

    report a decodableforwardworld model inside VLA representations. Both concern content available at or projected from the current step; neither measures the causal deployment of thepast frame, nor separates retained history from current-frame redundancy, which is the contribution here. OurA 1/A4 split is a temporal analogue of probing work that distinguish...