REVIEW 3 major objections 5 minor 31 references
DiMaS: Distribution Matching for Steering Vision-Language-Action Models
T0 review · 3 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read The paper claims that behavioral control of flow-matching vision-language-action models requires distribution matching, not linear steering, and demonstrates it with DiMaS.
desk verdict Solid adaptation of OT steering to flow-matching VLAs with an honest transfer study, but feature shifts are computed over failed episodes too, so the headline 'controls behavior' is softer than it looks. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the optimal transport map T_l^(m) from D- to D+ over residual-stream representations of the flow-matching action expert, computed by minimizing squared-Euclidean Kantorovich cost and solved with low-rank Sinkhorn. Source and target distributions are the lower and upper quantile tails of the behavioral feature; a linear SVM gates the intervention so only feature-absent representations are moved; and alpha in [0,1] interpolates between original and transported representation, letting a user trade steering strength against task success.
What would settle it
Train DiMaS on the same representations but with the action-feature labels randomly permuted, keeping all other choices fixed; if the steered feature still moves as strongly as with true labels, the representation-to-action correspondence is not the mechanism doing the work. A weaker check: verify on a task where the claimed feature is not recoverable from the action expert's residuals whether control disappears.
Extended reading notes
Core claim
The paper's central claim is that behavioral control in flow-matching VLAs requires distribution matching rather than linear steering. DiMaS learns an optimal-transport map between the set of action-expert residual-stream representations whose associated actions have a low value of a target feature (source) and the set whose actions have a high value (target), applies that map only when a linear classifier judges the feature absent, and blends the transported representation with the original by an interpolation coefficient alpha. Empirically this modulates end-effector speed in both directions on SmolVLA and pi0.5, and vertical displacement on pi0.5, while the linear and prompt baselines are
Load-bearing premise
The load-bearing premise is that each hidden representation can be labeled by the feature value of the single action it supposedly corresponds to, even though the paper admits an output action is shaped by more than one representation in a chunk; if that correspondence is systematically wrong, the source and target distributions used to learn the transport are mislabeled.
Editorial extensions
If this is right
- DiMaS shifts end-effector speed in both directions on both tested VLAs and vertical displacement on pi0.5, with statistically significant changes at alpha=0.5.
- It preserves task success better than linear regression steering, mean-difference steering, and prompt injection, and does not need to pause the intervention.
- The steering generalizes across held-out initial states, held-out tasks within a suite, and sometimes to a disjoint suite, indicating the transport captures behavior-linked structure, not task memorization.
- The interpolation knob alpha provides a continuous trade-off: larger alpha gives stronger feature control but lower success rate, with alpha=0.5 as a sensible default.
- Steering is most effective when amplifying behaviors the policy already tends toward (slower, lower) and weaker when pushing against them (faster, higher).
Reading between the lines
- If 'linearly decodable but not linearly steerable' is a general property of flow-matching visuomotor representations, then any feature-control method for such policies will likely need a nonlinear or distribution-level intervention; optimal transport is one concrete recipe.
- The same distribution-matching recipe could be applied to other continuous behavioral features observable from predicted actions or hidden states—such as trajectory smoothness, gripper open/close timing, or rotation direction—without changing the method.
- The paper's own caveat that one representation is only approximately paired with one action suggests a testable improvement: pooling or attention-weighting all representations that contribute to a chunk before assigning the feature label might sharpen the transport.
- Since steering is strongest when it amplifies behaviors the policy already prefers (slower, lower) and weakest when reversing them, a practical extension is to choose steering targets aligned with the policy's default tendencies rather than against them.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces DiMaS, a post-hoc steering method for flow-matching vision-language-action (VLA) policies. DiMaS extracts residual-stream representations from the action expert, partitions them into low and high tails of a continuous behavioral feature (end-effector speed or vertical displacement), learns an optimal-transport map between these distributions via low-rank Sinkhorn, gates the intervention with a linear SVM, and applies the map with an interpolation coefficient alpha. The method is evaluated on SmolVLA and pi0.5 across LIBERO suites and four transfer settings (held-out initial states, held-out tasks, and disjoint suites), reporting significant feature shifts with smaller success-rate penalties than linear-regression, mean-difference, and prompt baselines. A representation analysis further argues that behavioral features are linearly decodable but not linearly steerable, motivating the distribution-matching design.
Significance. If the empirical claims hold, DiMaS would be a practical post-hoc control tool for current flow-matching VLAs, which have so far received little steering work. The paper's strengths include public code, evaluation on two model scales, several transfer settings, and multiple baselines. The held-out evaluation protocol is a genuine positive: several claims are tested on initial states and tasks not used to fit the transport map. The main weakness is that the central behavioral-control claim is currently entangled with task-failure artifacts, and the per-token action-feature attribution is explicitly acknowledged as an approximation. A success-only reanalysis could settle the main question; the per-token issue needs a sensitivity check.
major comments (3)
- [§4.1, Figs. 2–5, Fig. A4] The paired t-test evidence for feature control is computed over all episodes, including failures. Since DiMaS lowers SR in many settings (e.g., Fig. 5 alpha=0.5: 96→90%; Fig. 4 Setting 2 SmolVLA: Object 89→70%, Spatial 71→52%, Goal 81→46%; Fig. A4 SmolVLA Goal alpha=0.5: 82→20%), the mean speed and z-displacement can drop mechanically from early termination or stalling, not from genuine behavioral modulation. The only success-only speed panel is Fig. 7(a), which is an alpha ablation, not the main baseline comparison. Please recompute all main Δfeature and t-tests restricted to episodes that succeed under both the baseline and the intervention, and report SR alongside. If significant shifts vanish on successful episodes, the central claim is unsupported.
- [§7 and Eq. (2)] The source/target labels pair each extracted representation h^{p,m}_l with the scalar feature φ(a_i) of its exact corresponding action, but the final action chunk is generated jointly from multiple representations across denoising steps. The authors explicitly call this 'only an approximation' in the Conclusion. This is a load-bearing training-signal assumption: a systematic mismatch would mislabel D−/D+ and could make the transport map steer a confounded direction rather than the intended behavior. Please provide a sensitivity analysis—e.g., compare per-token labels with chunk-average labels, or show that h^{p,m} predicts φ(a_p) better than φ(a_{p±1})—and discuss how label noise would affect the learned map.
- [Abstract and §4.1] The abstract claims DiMaS 'effectively controls behavior across two state-of-the-art VLAs,' but the SmolVLA results show substantial success-rate drops in many cells (Fig. 4, Fig. A4), and the vertical-displacement results are acknowledged to be tied to task completion (§4.1). The claim should be qualified to the regimes where SR is preserved, and 'largely preserving success' needs a quantitative definition or, at minimum, a success-only feature report. As currently written, the headline claim overstates the evidence on one of the two models.
minor comments (5)
- [§4.1] Please state the pairing and sample size for the paired t-tests (e.g., paired by initial state across episodes) so the reader can assess statistical power.
- [Figs. 2–3] The axes labeled 'SR (pp)' with negative values are visually ambiguous. Use ΔSR (pp) and Δspeed (%) with an explicit zero line to make the sign convention clear.
- [Fig. 6] The linear-steerability visualization is qualitative and shown for one layer and one flow-matching step. Add quantitative distribution distances (e.g., MMD) before/after linear shift across layers and steps to support the 'linearly decodable but not linearly steerable' claim.
- [Table A1] The hyperparameter table lists alpha=0.5 as a frozen default, but the alpha ablation suggests that task-specific tuning could improve the trade-off. Please clarify whether any hyperparameter selection was performed using test data.
- [§3.2] The phrase 'secret sauce' is informal; consider replacing it with 'interpolation' or 'strength knob' to match the paper's technical style.
Circularity Check
No significant circularity: held-out Settings 1-4 and external benchmarks make the feature-shift result an empirical prediction, not a fit renamed.
full rationale
DiMaS is a supervised, data-driven steering method: D- and D+ are defined by thresholding the target feature phi on rollout actions (Eq. 2), and the transport map is fit to move representations between these distributions (Eq. 3). The central claim is not derived from first principles; it is an empirical claim that applying the fit map at inference shifts phi in settings not used for fitting. The paper explicitly evaluates this under Settings 1-4 (Section 4.2): held-out initial states, held-out tasks within a suite, and disjoint suites. Because the transport map and gate are fit on a training subset and evaluated on disjoint episodes/tasks, the observed feature shift is not the training objective reproduced on the training set; it is a generalization result. The self-citations ([12,19]) are background examples of representation steering and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The conclusion's caveat that chunk-level action/representation correspondence 'is only an approximation' is a correctness limitation, not a circular step: it weakens the assumed training signal but does not make the held-out evaluation equivalent to the fit. The main methodological concerns--possible in-sample reporting in Figs. 2-3 and feature shifts confounded with success-rate drops because feature means include failed episodes--are validity risks, not circularity. For these reasons, no circular step meeting the quoted-reduction bar was found.
Assumptions & free parameters
free parameters (5)
- Interpolation coefficient alpha =
0.5 (default)
- Source/target quantile thresholds q_tau, q_1-tau =
0.25 / 0.75
- Steering layer =
second-to-last action-expert layer
- Sinkhorn regularization epsilon =
1e-4
- SVM regularization C =
0.1
assumptions (4)
- domain assumption A single-layer intervention in the flow-matching action expert is causally sufficient to change output behavior without breaking the denoising dynamics.
- ad hoc to paper Each action token's scalar feature can be attributed to the corresponding per-position representation, despite action chunking.
- domain assumption Euclidean squared distance is the correct ground cost for transporting behavior in representation space.
- domain assumption The 25th/75th percentile tails form clean, well-separated source and target populations.
Cite this review
Pith. "Pith review of DiMaS: Distribution Matching for Steering Vision-Language-Action Models." pith.science (2026). https://pith.science/paper/CS3J7DBE
@misc{pith2026260714280,
author = {Pith},
title = {Pith review of: DiMaS: Distribution Matching for Steering Vision-Language-Action Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/CS3J7DBE}},
note = {Machine review of arXiv:2607.14280}
}
read the original abstract
Flow-matching-based vision-language-action (VLA) models have emerged as powerful policies for robotic manipulation, yet a critical capability remains underexplored: fine-grained behavioral control, the ability to govern how a robot performs a task by intervening on its internal representations. Representation steering is a well-established interpretability tool for language and vision-language models, where behavioral features are typically encoded as linear directions, but we show that these classic methods fall short in VLAs. We propose DiMaS, a Distribution-Matching Steering strategy tailored to flow-matching VLAs, which transports between representation distributions rather than shifting along a fixed direction, and show that it effectively controls behavior across two state-of-the-art VLAs. We further examine the generalizability of this strategy as the tasks it is learned from and evaluated on grow increasingly dissimilar, characterizing where behavioral control transfers and where it weakens. Finally, through an analysis of the representation structure of the action expert, we explain why classical linear steering falls short in the visuomotor setting: behavioral features are linearly decodable but not linearly steerable, which motivates the distribution-matching design of DiMaS. Our code is publicly available at https://github.com/pegah-kh/dimas, with additional results and videos at https://pegah-kh.github.io/dimas/
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Flamingo: a visual language model for few-shot learning.Advances in Neural Information Processing Systems (NeurIPS), 35:23716–23736, 2022
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in Neural Information Processing Systems (NeurIPS), 35:23716–23736, 2022
2022
-
[2]
Refusal in language models is mediated by a single direction.arXiv preprint arXiv:2406.11717, 2024
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction.arXiv preprint arXiv:2406.11717, 2024
arXiv 2024
-
[3]
Do sparse autoencoders capture concept manifolds?arXiv preprint arXiv:2604.28119, 2026
Usha Bhalla, Thomas Fel, Can Rager, Sheridan Feucht, Tal Haklay, Daniel Wurgaft, Siddharth Boppana, Matthew Kowal, Vasudev Shyam, Jack Merullo, et al. Do sparse autoencoders capture concept manifolds?arXiv preprint arXiv:2604.28119, 2026
arXiv 2026
-
[4]
Language models are few-shot learners.Advances in Neural Information Processing Systems (NeurIPS), 33:1877–1901, 2020
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems (NeurIPS), 33:1877–1901, 2020
1901
-
[5]
Hugo Buurmeijer, Carmen Amo Alonso, Aiden Swann, and Marco Pavone. Observing and controlling features in vision-language-action models.arXiv preprint arXiv:2603.05487, 2026
arXiv 2026
-
[6]
Toy models of superposition.arXiv preprint arXiv:2209.10652, 2022
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition.arXiv preprint arXiv:2209.10652, 2022
arXiv 2022
-
[7]
Not all language model features are one-dimensionally linear
Josh Engels, Eric Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. Not all language model features are one-dimensionally linear. InInternational Conference on Learning Repre- sentations, volume 2025, pages 84591–84622, 2025
2025
-
[8]
Molmoact2: Action reasoning models for real-world deployment.arXiv preprint arXiv:2605.02881, 2026
Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei-Chuan Tsai, Shirui Chen, Yi Ru Wang, et al. Molmoact2: Action reasoning models for real-world deployment.arXiv preprint arXiv:2605.02881, 2026
arXiv 2026
Show all 31 references
-
[9]
Into the rabbit hull: From task-relevant concepts in dino to minkowski geometry.arXiv preprint arXiv:2510.08638, 2025
Thomas Fel, Binxu Wang, Michael A Lepori, Matthew Kowal, Andrew Lee, Randall Balestriero, Sonia Joseph, Ekdeep S Lubana, Talia Konkle, Demba Ba, et al. Into the rabbit hull: From task-relevant concepts in dino to minkowski geometry.arXiv preprint arXiv:2510.08638, 2025
2025 arXiv
-
[10]
Pot: Python optimal transport.Journal of Machine Learning Research, 22(78):1–8, 2021
R ´emi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z Alaya, Aur ´elie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, et al. Pot: Python optimal transport.Journal of Machine Learning Research, 22(78):1–8, 2021
2021
-
[11]
Mechanistic interpretability for steering vision-language-action models.Conference on Robot Learning (CoRL), 2025
Bear H ¨aon, Kaylene Stocking, Ian Chuang, and Claire Tomlin. Mechanistic interpretability for steering vision-language-action models.Conference on Robot Learning (CoRL), 2025. 14
2025
-
[12]
An- alyzing fine-tuning representation shift for multimodal llms steering alignment.International Conference on Computer Vision, 2025
Pegah Khayatan, Mustafa Shukor, Jayneel Parekh, Arnaud Dapogny, and Matthieu Cord. An- alyzing fine-tuning representation shift for multimodal llms steering alignment.International Conference on Computer Vision, 2025
2025
-
[13]
Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024
2024 arXiv
-
[14]
Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013
Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013
2013 arXiv
-
[15]
Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022
2022 arXiv
-
[16]
Libero: Benchmarking knowledge transfer for lifelong robot learning.Advances in Neural Information Processing Systems, 36:44776–44791, 2023
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning.Advances in Neural Information Processing Systems, 36:44776–44791, 2023
2023
-
[17]
Sparse autoencoders learn monosemantic features in vision-language models.Advances in Neural Information Processing Systems, 38:95706–95742, 2026
Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, and Zeynep Akata. Sparse autoencoders learn monosemantic features in vision-language models.Advances in Neural Information Processing Systems, 38:95706–95742, 2026
2026
-
[18]
Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681, 2023
Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681, 2023
2023 arXiv
-
[19]
Learning to steer: Input-dependent steering for multimodal llms.Advances in Neural Information Processing Systems, 38:159799–159834, 2026
Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Arnaud Dapogny, Alasdair Newson, and Matthieu Cord. Learning to steer: Input-dependent steering for multimodal llms.Advances in Neural Information Processing Systems, 38:159799–159834, 2026
2026
-
[20]
The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658, 2023
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658, 2023
2023 arXiv
-
[21]
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al.𝜋 0.5: A vision- language-action model with open-world generalization.arXiv preprint arXiv:2504.16054, 2025
2025 arXiv
-
[22]
Hallucination reduction with casal: Contrastive activation steering for amortized learning.arXiv preprint arXiv:2510.02324, 2025
Xinchi Qiu, Lei Yu, Yuchen Zhang, Aobo Yang, Narine Kokhlikyan, Nicola Cancedda, Diego Garcia-Olano, et al. Hallucination reduction with casal: Contrastive activation steering for amortized learning.arXiv preprint arXiv:2510.02324, 2025
2025
-
[23]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...
2021
-
[24]
Controlling language and diffusion models by transporting activations
Pau Rodriguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apostoloff, Xavier Suau, et al. Controlling language and diffusion models by transporting activations. InInternational Conference on Learning Representations, volume 2025, pages 89812–89855, 2025. 15
2025
-
[25]
Low-rank sinkhorn factorization, 2021
Meyer Scetbon, Marco Cuturi, and Gabriel Peyr ´e. Low-rank sinkhorn factorization, 2021
2021
-
[26]
Interfacegan: Interpreting the disentangled face representation learned by gans.IEEE transactions on pattern analysis and machine intelligence, 44(4):2004–2018, 2020
Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. Interfacegan: Interpreting the disentangled face representation learned by gans.IEEE transactions on pattern analysis and machine intelligence, 44(4):2004–2018, 2020
2004
-
[27]
Smolvla: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025
Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. Smolvla: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506...
2025 arXiv
-
[28]
Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- oth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023
2023 arXiv
-
[29]
Steering language models with activation engineering.arXiv preprint arXiv:2308.10248, 2023
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering.arXiv preprint arXiv:2308.10248, 2023
2023 arXiv
-
[30]
Rt-2: Vision-language-action models transfer web knowledge to robotic control
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning, pages 2165–2183. PMLR, 2023
2023
-
[31]
Do this quickly
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023. 16 Supplementary Mater...
2023 arXiv
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.