pith. sign in

arxiv: 2606.07100 · v1 · pith:ARBVRGABnew · submitted 2026-06-05 · 💻 cs.CV · cs.RO

LARA: Latent Action Representation Alignment for Vision-Language-Action Models

Mengya Liu , Baoxiong Jia , Jiangyong Huang , Jingze Zhang , Siyuan Huang This is my paper

Pith reviewed 2026-06-27 22:23 UTC · model grok-4.3

classification 💻 cs.CV cs.RO
keywords vision-language-action modelslatent action modelsrepresentation alignmentrobotic manipulationjoint trainingunlabeled human videos
0
0 comments X

The pith

Jointly aligning representations lets latent action models and vision-language-action models improve each other during training.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces LARA as a plug-and-play method to train latent action models and vision-language-action models together rather than separately. Separate training leaves the latent model ungrounded in real actions and constrains the vision-language model to frozen features. Alignment during joint optimization lets the latent model incorporate action trajectories to filter out spurious visual changes while the vision-language model uses the latent model's forward dynamics to avoid predicting ineffective trajectories. The approach is shown to raise performance on multiple robotic manipulation benchmarks that include both simulation and a real-world setup.

Core claim

LARA enables reciprocal benefits by jointly optimizing LAM and VLA through representation alignment, allowing LAMs to learn from action trajectories and VLAs to be regularized by learned forward dynamics.

What carries the argument

Latent Action Representation Alignment (LARA), a plug-and-play framework that aligns representations between LAM and VLA models during joint optimization.

If this is right

  • LAMs learn to avoid spurious visual changes by incorporating action trajectories during alignment.
  • VLAs are regularized by forward dynamics learned inside LAMs, reducing hallucinations of functionally ineffective trajectories.
  • The same framework supports pre-training from scratch, post-training enhancement of existing VLA models, and refinement of LAMs.
  • Average gains of roughly 10 percent, 5 percent, and 15 percent appear across the three simulation and one real-world robotic manipulation benchmarks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If alignment remains stable across larger unlabeled video collections, VLA training could scale with far less robot-specific action data.
  • The same alignment idea could be tested on other pairs of dynamics models and policy models in robotics.
  • Persistent alignment during continued training might reduce the frequency of out-of-distribution failures in deployed robots.

Load-bearing premise

Forcing representation alignment between separately trained LAM and VLA models will produce reciprocal benefits without destabilizing either model or introducing new failure modes.

What would settle it

A controlled comparison showing that joint training with alignment produces lower success rates or higher rates of ineffective trajectories than separate training on the same benchmarks would falsify the reciprocal-benefit claim.

Figures

Figures reproduced from arXiv: 2606.07100 by Baoxiong Jia, Jiangyong Huang, Jingze Zhang, Mengya Liu, Siyuan Huang.

Figure 1
Figure 1. Figure 1: We present Latent Action Representation Alignment (LARA), a simple yet highly effective Vision-Language-Action (VLA) framework that bridges unlabeled video data and action-labeled robot datasets by jointly training a Latent Action Model (LAM) and a diffusion-based VLA model via latent action representation alignment. LARA supports versatile usage as a pre-training method, a post-training enhancement module… view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of LAM-based VLA models. LAMs are commonly used as pseudo labels for VLA learning (left), where as LARA jointly optimizes LAM and VLA model by explicitly aligning their latent representations (right). • We propose, LARA, a novel and effective framework for jointly improving LAM and VLA model learning via latent action representation alignment. • We show LARA’s versatility as a pre-training metho… view at source ↗
Figure 3
Figure 3. Figure 3: Method overview. We begin with LAM (left), where an Inverse Dynamic Model (IDM) learns a latent action z𝑡 from consecutive image frames, and a Forward Dynamic Model (FDM) learns to reconstruct the subsequent frame conditioned on the preceding frame and the quantized latent action z 𝑞 𝑡 . We then conduct Latent Action Representation Alignment (LARA) training on a diffusion-based VLA model, where LARA explic… view at source ↗
Figure 4
Figure 4. Figure 4: Task Visualization of GR1-Sim-24(30) and G1-Real(50).We illustrate a representative bimanual task from the GR1-Sim-24(30) simulation suite (left) alongside the two real-world tasks evaluated on the G1 humanoid: Pick-n-Place and Grasp-an-Pour (right). For a detailed frame-by-frame breakdown of the G1-Real(50) execution, please refer to Fig. S.4. train the pre-trained GR00T-N1.6 model with an LAM (pre-traine… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation Study on LARA Design. We report success rates on LIBERO-Long, the most challenging subset of LIBERO benchmark. the Moto-GPT (Chen et al., 2025b) framework as a con￾trolled testbed, leveraging its reliance on LAM-generated latent action tokens for VLA supervision. Specifically, Moto-GPT employs a two-stage curriculum, where an ini￾tial LAM-only training phase supervise VLA models exclu￾sively by ps… view at source ↗
read the original abstract

Visual-language action (VLA) models enable robots to predict actions directly from observations and language instructions, but their performance depends on large-scale, high-quality data and is limited by the scarcity of real-world robot action datasets. To facilitate VLA model learning with abundant unlabeled human videos, Latent Action Models (LAM) learn latent action representations from visual dynamics to provide additional supervision for VLA learning. However, LAM and VLA are typically trained separately, leaving LAM ungrounded during VLA training and VLA models constrained by frozen LAM representations. To address these issues, we propose Latent Action Representation Alignment (LARA), a plug-and-play framework that jointly optimizes LAM and VLA via representation alignment. This enables reciprocal benefits where LAMs learn with action trajectories to avoid spurious visual changes, while VLAs are regularized by forward dynamics learned within LAMs to reduce hallucinations of functionally ineffective trajectories. We demonstrate LARA versatility and effectiveness for pre-training, post-training enhancement of pre-trained VLA models, and LAM refinement, achieving an average of ~10%, ~5%, and ~15% improvement over 3 simulation and 1 meticulously designed real-world robotic manipulation benchmarks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes LARA, a plug-and-play framework that jointly optimizes Latent Action Models (LAMs) and Vision-Language-Action (VLA) models through representation alignment. This is intended to ground LAMs with action trajectories (avoiding spurious visual changes) and regularize VLAs with LAM forward dynamics (reducing ineffective trajectories). The approach is applied to pre-training, post-training enhancement, and LAM refinement, with reported average gains of ~10%, ~5%, and ~15% over three simulation benchmarks and one real-world robotic manipulation benchmark.

Significance. If the gains are robust, reproducible, and causally attributable to the alignment mechanism rather than joint optimization or data effects, the work would meaningfully address data scarcity for VLAs by leveraging unlabeled human videos. The plug-and-play design and multi-stage applicability would be practical strengths for the field.

major comments (2)
  1. [Experiments / Results] The central empirical claim (average ~10/5/15% gains) rests on the assertion that representation alignment produces specific reciprocal benefits (LAM avoiding spurious changes; VLA avoiding ineffective trajectories). No ablation isolating the alignment loss from other joint-training effects (e.g., extra gradients, data mixing) is described, leaving the mechanism unverified against the skeptic concern.
  2. [Abstract / Experiments] The abstract and claim summary supply no baselines, error bars, data-exclusion rules, or statistical tests. Without these, the reported deltas cannot be assessed for significance or compared to prior LAM/VLA joint-training methods.
minor comments (1)
  1. [Method] Notation for the alignment objective and the separate LAM/VLA losses should be introduced with explicit equations early in the method section to clarify how the joint optimization is formulated.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the two major comments point-by-point below and commit to revisions that strengthen the empirical support for our claims without overstating current results.

read point-by-point responses

  1. Referee: [Experiments / Results] The central empirical claim (average ~10/5/15% gains) rests on the assertion that representation alignment produces specific reciprocal benefits (LAM avoiding spurious changes; VLA avoiding ineffective trajectories). No ablation isolating the alignment loss from other joint-training effects (e.g., extra gradients, data mixing) is described, leaving the mechanism unverified against the skeptic concern.

    Authors: We agree that the manuscript does not contain an ablation that isolates the alignment loss from other joint-training effects such as additional gradients or data mixing. Existing experiments compare LARA against separate LAM and VLA training but lack a joint-training control without the alignment objective. We will add this ablation in the revision to directly test whether the reported reciprocal benefits are attributable to representation alignment. revision: yes

  2. Referee: [Abstract / Experiments] The abstract and claim summary supply no baselines, error bars, data-exclusion rules, or statistical tests. Without these, the reported deltas cannot be assessed for significance or compared to prior LAM/VLA joint-training methods.

    Authors: We acknowledge that the abstract lacks explicit baselines, error bars, and statistical details. We will revise the abstract to reference the main comparison baselines and note that error bars appear in the result tables. Data-exclusion rules and any statistical tests are already described in the experimental setup; we will add a brief pointer in the abstract. Full comparison to prior joint-training methods remains in the body of the paper due to length constraints. revision: partial

Circularity Check

0 steps flagged

No circularity; empirical performance claims with no derivations or self-referential reductions

full rationale

The paper's central claims consist of an empirical method (LARA) and reported benchmark improvements (~10/5/15%). No equations, derivations, or fitted parameters are presented in the abstract or described structure. The description of reciprocal benefits is a design rationale, not a mathematical reduction to inputs. No self-citation load-bearing steps or ansatz smuggling appear. This is a standard non-circular empirical ML paper.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available, so the ledger is necessarily incomplete. The central claim rests on the unstated premise that representation alignment will produce the described reciprocal benefits without side effects; no free parameters, axioms, or invented entities are identifiable from the given text.

pith-pipeline@v0.9.1-grok · 5747 in / 1132 out tokens · 28039 ms · 2026-06-27T22:23:25.694932+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Semi-Supervised Vision-Language-Action Model

    cs.CV 2026-06 unverdicted novelty 6.0

    SemiVLA improves VLA adaptation under 10% labeled trajectories via self-distilled pseudo-actions, reaching 89% success on LIBERO with OpenVLA backbone.

Reference graph

Works this paper leans on

64 extracted references · 27 linked inside Pith · cited by 1 Pith paper

  1. [1]

    arXiv preprint arXiv:2510.25616 , year=

    Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization , author=. arXiv preprint arXiv:2510.25616 , year=

    arXiv

  2. [2]

    arXiv preprint arXiv:2504.10483 , year=

    Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers , author=. arXiv preprint arXiv:2504.10483 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2406.09246 , year=

    Openvla: An open-source vision-language-action model , author=. arXiv preprint arXiv:2406.09246 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2212.06817 , year=

    Rt-1: Robotics transformer for real-world control at scale , author=. arXiv preprint arXiv:2212.06817 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2503.14734 , year=

    Gr00t n1: An open foundation model for generalist humanoid robots , author=. arXiv preprint arXiv:2503.14734 , year=

    Pith/arXiv arXiv

  6. [6]

    arXiv preprint arXiv:2410.24164 , year=

    _0 : A Vision-Language-Action Flow Model for General Robot Control , author=. arXiv preprint arXiv:2410.24164 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2410.11758 , year=

    Latent action pretraining from videos , author=. arXiv preprint arXiv:2410.11758 , year=

    Pith/arXiv arXiv

  8. [8]

    Moto: Latent motion token as the bridging language for robot manipulation , author=

  9. [9]

    arXiv preprint arXiv:2502.00379 , year=

    Latent action learning requires supervision in the presence of distractors , author=. arXiv preprint arXiv:2502.00379 , year=

    arXiv

  10. [10]

    Lapo: Latent-variable advantage-weighted policy optimization for offline reinforcement learning , author=

  11. [11]

    arXiv preprint arXiv:2507.23682 , year=

    Villa-x: enhancing latent action modeling in vision-language-action models , author=. arXiv preprint arXiv:2507.23682 , year=

    Pith/arXiv arXiv

  12. [12]

    Univla: Learning to act anywhere with task-centric latent actions , author=

  13. [13]

    arXiv preprint arXiv:2410.06940 , year=

    Representation alignment for generation: Training diffusion transformers is easier than you think , author=. arXiv preprint arXiv:2410.06940 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2304.07193 , year=

    Dinov2: Learning robust visual features without supervision , author=. arXiv preprint arXiv:2304.07193 , year=

    Pith/arXiv arXiv

  15. [15]

    arXiv preprint arXiv:2405.12213 , year=

    Octo: An open-source generalist robot policy , author=. arXiv preprint arXiv:2405.12213 , year=

    Pith/arXiv arXiv

  16. [16]

    arXiv preprint arXiv:2501.15830 , year=

    Spatialvla: Exploring spatial representations for visual-language-action model , author=. arXiv preprint arXiv:2501.15830 , year=

    Pith/arXiv arXiv

  17. [17]

    arXiv preprint arXiv:2501.14818 , year=

    Eagle 2: Building post-training data strategies from scratch for frontier vision-language models , author=. arXiv preprint arXiv:2501.14818 , year=

    arXiv

  18. [18]

    arXiv preprint arXiv:2505.11917 , year=

    OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning , author=. arXiv preprint arXiv:2505.11917 , year=

    arXiv

  19. [19]

    arXiv preprint arXiv:2512.01715 , year=

    DiG-Flow: Discrepancy-Guided Flow Matching for Robust VLA Models , author=. arXiv preprint arXiv:2512.01715 , year=

    arXiv

  20. [20]

    arXiv preprint arXiv:2507.15597 , year=

    Being-h0: vision-language-action pretraining from large-scale human videos , author=. arXiv preprint arXiv:2507.15597 , year=

    arXiv

  21. [21]

    arXiv preprint arXiv:2502.19417 , year=

    Hi robot: Open-ended instruction following with hierarchical vision-language-action models , author=. arXiv preprint arXiv:2502.19417 , year=

    Pith/arXiv arXiv

  22. [22]

    arXiv preprint arXiv:2312.10812 , year=

    Learning to act without actions , author=. arXiv preprint arXiv:2312.10812 , year=

    arXiv

  23. [23]

    Genie: Generative interactive environments , author=

  24. [24]

    Dynamo: In-domain dynamics pretraining for visuo-motor control , author=

  25. [25]

    arXiv preprint arXiv:2411.00785 , year=

    Igor: Image-goal representations are the atomic control units for foundation models in embodied ai , author=. arXiv preprint arXiv:2411.00785 , year=

    arXiv

  26. [26]

    2025 , publisher=

    Diffusion policy: Visuomotor policy learning via action diffusion , author=. 2025 , publisher=

    2025

  27. [27]

    Cot-vla: Visual chain-of-thought reasoning for vision-language-action models , author=

  28. [28]

    arXiv preprint arXiv:2501.09747 , year=

    Fast: Efficient action tokenization for vision-language-action models , author=. arXiv preprint arXiv:2501.09747 , year=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2507.04447 , year=

    Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge , author=. arXiv preprint arXiv:2507.04447 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2502.19645 , year=

    Fine-tuning vision-language-action models: Optimizing speed and success , author=. arXiv preprint arXiv:2502.19645 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2412.14058 , year=

    Towards Generalist Robot Policies: What Matters in Building Vision-Language-Action Models , author=. arXiv preprint arXiv:2412.14058 , year=

    Pith/arXiv arXiv

  32. [32]

    arXiv preprint arXiv:2412.10345 , year=

    Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies , author=. arXiv preprint arXiv:2412.10345 , year=

    Pith/arXiv arXiv

  33. [33]

    Magma: A foundation model for multimodal ai agents , author=

  34. [34]

    arXiv preprint arXiv:2210.02747 , year=

    Flow matching for generative modeling , author=. arXiv preprint arXiv:2210.02747 , year=

    Pith/arXiv arXiv

  35. [35]

    Neural discrete representation learning , author=

  36. [36]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0 , author=

  37. [37]

    Libero: Benchmarking knowledge transfer for lifelong robot learning , author=

  38. [38]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  39. [39]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    3drs: Mllms need 3d-aware representation supervision for scene understanding , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  40. [40]

    arXiv preprint arXiv:2512.01809 , year=

    Much Ado About Noising: Dispelling the Myths of Generative Robotic Control , author=. arXiv preprint arXiv:2512.01809 , year=

    arXiv

  41. [41]

    arXiv preprint arXiv:2410.07864 , year=

    Rdt-1b: a diffusion foundation model for bimanual manipulation , author=. arXiv preprint arXiv:2410.07864 , year=

    Pith/arXiv arXiv

  42. [42]

    Scalable diffusion models with transformers , author=

  43. [43]

    arXiv preprint arXiv:2505.15659 , year=

    FLARE: Robot learning with implicit world modeling , author=. arXiv preprint arXiv:2505.15659 , year=

    Pith/arXiv arXiv

  44. [44]

    Bootstrap your own latent-a new approach to self-supervised learning , author=

  45. [45]

    arXiv preprint arXiv:2405.05941 , year=

    Evaluating real-world robot manipulation policies in simulation , author=. arXiv preprint arXiv:2405.05941 , year=

    Pith/arXiv arXiv

  46. [46]

    arXiv preprint arXiv:2503.06669 , year=

    Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems , author=. arXiv preprint arXiv:2503.06669 , year=

    Pith/arXiv arXiv

  47. [47]

    something something

    The" something something" video database for learning and evaluating visual common sense , author=. Proceedings of the IEEE international conference on computer vision , pages=

  48. [48]

    arXiv preprint arXiv:2506.15691 , year=

    What Do Latent Action Models Actually Learn? , author=. arXiv preprint arXiv:2506.15691 , year=

    arXiv

  49. [49]

    arXiv preprint arXiv:2307.00595 , year=

    Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot , author=. arXiv preprint arXiv:2307.00595 , year=

    arXiv

  50. [50]

    Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives , author=

  51. [51]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Masked autoencoders are scalable vision learners , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  52. [52]

    URL https://arxiv

    Rt-2: Vision-language-action models transfer web knowledge to robotic control, 2023 , author=. URL https://arxiv. org/abs/2307.15818 , year=

    Pith/arXiv arXiv 2023

  53. [53]

    arXiv preprint arXiv:2403.09631 , year=

    3d-vla: A 3d vision-language-action generative world model , author=. arXiv preprint arXiv:2403.09631 , year=

    Pith/arXiv arXiv

  54. [54]

    IEEE Robotics and Automation Letters , volume=

    Pointvla: Injecting the 3d world into vision-language-action models , author=. IEEE Robotics and Automation Letters , volume=. 2026 , publisher=

    2026

  55. [55]

    arXiv preprint arXiv:2508.07917 , year=

    Molmoact: Action reasoning models that can reason in space , author=. arXiv preprint arXiv:2508.07917 , year=

    Pith/arXiv arXiv

  56. [56]

    arXiv preprint arXiv:2507.16815 , year=

    Thinkact: Vision-language-action reasoning via reinforced visual latent planning , author=. arXiv preprint arXiv:2507.16815 , year=

    Pith/arXiv arXiv

  57. [57]

    arXiv preprint arXiv:2508.21046 , year=

    Cogvla: Cognition-aligned vision-language-action model via instruction-driven routing & sparsification , author=. arXiv preprint arXiv:2508.21046 , year=

    Pith/arXiv arXiv

  58. [58]

    arXiv preprint arXiv:2505.04999 , year=

    Clam: Continuous latent action models for robot learning from unlabeled demonstrations , author=. arXiv preprint arXiv:2505.04999 , year=

    Pith/arXiv arXiv

  59. [59]

    arXiv preprint arXiv:2502.20321 , year=

    Unitok: A unified tokenizer for visual generation and understanding , author=. arXiv preprint arXiv:2502.20321 , year=

    arXiv

  60. [60]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Denoising token prediction in masked autoregressive models , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  61. [61]

    Forty-first International Conference on Machine Learning , year=

    Prismatic vlms: Investigating the design space of visually-conditioned language models , author=. Forty-first International Conference on Machine Learning , year=

  62. [62]

    Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael Robert and Finn, Chelsea and Fusai, Niccolo and Galliker, Manuel Y and others , booktitle=. _

  63. [63]

    Arnold: A benchmark for language-grounded task learning with continuous states in realistic 3d scenes , author=

  64. [64]

    An embodied generalist agent in 3d world , author=