Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

Bowen Yuan; Han Hu; He Zhang; Hongsheng Wu; Mingqi Gao; Oran Wang; Rui Huang; Shenghao Zhang; Xiaoran Xu; Xumin Yu

A 3B-activated embodied VLM matches much larger models on physical-world agent tasks by training on an action-centric capability taxonomy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-15 02:31 UTC pith:S5LGCRXR

load-bearing objection Abstract-only industrial release of an efficient 3B-active embodied VLM with concrete leaderboard claims; useful if the full evals hold, but currently unauditable. the 3 major comments →

arxiv 2607.12894 v1 pith:S5LGCRXR submitted 2026-07-14 cs.CV

Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

Ziyi Wang , Xumin Yu , Yongming Rao , Yonggen Ling , Yunheng Li , Oran Wang , Mingqi Gao , Yuchen Zhou

show 14 more authors

Yves Liang Zuyan Liu Yani Zhang Rui Huang Xiaoran Xu Bowen Yuan Yifu Yuan Xu Tan He Zhang Yufei Huang Shenghao Zhang Hongsheng Wu Han Hu Zhengyou Zhang

This is my paper

classification cs.CV

keywords embodied AIvision-language modelsMixture-of-Expertsphysical-world agentsaction-centric taxonomymulti-turn reasoningefficient foundation modelsembodied benchmarks

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Hy-Embodied-VLM-1.0 as an efficient foundation model for agents that must perceive, reason about actions, and interact with the physical world. The authors argue that these agentic skills can be cultivated from pre-training if data are organized around three progressive dimensions: understanding states that matter for action, reasoning about how actions change the world, and handling sequential adaptive reasoning over long horizons. Built on a Mixture-of-Experts backbone that activates only 3B parameters, the model is evaluated on 38 benchmarks spanning embodied perception, physical understanding, and reasoning. It leads similarly sized models on 19 of those benchmarks, beats strong peers such as Qwen3.6-A3B and Cosmos 3, improves 8.4% on average over the prior 2B generation, and approaches the performance of an earlier 32B-activated model while also showing multi-turn long-horizon agentic behavior. A sympathetic reader would care because the work claims that careful action-centric data design, rather than sheer parameter count, can deliver deployable physical-world competence.

Core claim

Hy-Embodied-VLM-1.0, with only 3B activated parameters, achieves the best score among similarly sized models on 19 of 38 embodied benchmarks, substantially outperforms Qwen3.6-A3B and Cosmos 3, gains 8.4% on average over Hy-Embodied-0.5 MoT-2B, approaches prior 32B-activated performance, and exhibits strong multi-turn long-horizon embodied agentic behavior.

What carries the argument

An action-centric capability taxonomy of three progressive dimensions—Action-Relevant State Understanding, Action-Transition Reasoning, and Sequential and Adaptive Reasoning—that structures the data pipeline and pre-/post-training mixtures used to instill physical-world agent competence into an efficient MoE VLM.

Load-bearing premise

That the authors' action-centric taxonomy and curated data mixtures are the main drivers of the gains, and that the 38-benchmark suite fairly measures general physical-world agent competence rather than overfitting to related evaluation distributions.

What would settle it

An independent re-evaluation on held-out physical-world agent tasks (multi-turn interaction and long-horizon planning) that use scenes, objects, and protocols outside the authors' data pipeline; if the claimed margins over same-size baselines collapse, the central efficiency claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Abstract-only industrial release of an efficient 3B-active embodied VLM with concrete leaderboard claims; useful if the full evals hold, but currently unauditable.

read the letter

This is an abstract-only industrial model report. The punchline is simple: they claim a 3B-activated MoE embodied VLM that wins 19/38 benchmarks against similar-size models, beats Qwen3.6-A3B and Cosmos 3, gains 8.4% over their prior Hy-Embodied-0.5, and approaches an older 32B-activated system while also looking decent on multi-turn agentic tasks. If the full paper and numbers check out, that is a useful engineering result for latency-sensitive robotics and physical agents. It is not a new first-principles mechanism.

What is actually new is the packaged release: an action-centric taxonomy (state understanding → transition reasoning → sequential/adaptive reasoning) used to drive pre- and post-training data mixtures, stacked on Hy3-A3B + Hy-ViT2 with MoE efficiency. Embodied VLMs, action-centric training, and MoE stacks are established programs; the contribution is the systematic data pipeline plus the reported efficiency–performance tradeoff. Credit where due: the claims are concrete, the efficiency angle is the right one for deployment, and they are explicit about the multi-turn agentic setting rather than only static VQA-style scores.

The soft spots are exactly what you would expect from abstract-only work, and they are load-bearing. We cannot see data composition, sampling ratios, contamination controls, per-benchmark breakdowns, error bars, or the agentic protocols. The same taxonomy that guides curation also frames the evaluation axes, so mild self-alignment is possible. Whether the 38-benchmark suite measures general physical-world competence or in-house distribution match is uncheckable right now. Free parameters (mixture weights, routing budget) and the invented taxonomy entity are not independently verified. That does not make the claims false; it makes them provisional.

Who this is for: people building efficient embodied agents who need a strong baseline and care about activated-parameter cost. Not for theorists looking for a new principle. I would send it to peer review if a full paper with tables, data recipes, and protocols appears—serious referees should pressure-test contamination and agentic eval fairness—but I would not treat the abstract’s leaderboard statements as verified science today. Bring it to reading group only if someone has the full report or code; otherwise skip until the numbers are inspectable.

Referee Report

3 major / 2 minor

Summary. The manuscript introduces Hy-Embodied-VLM-1.0, an efficient embodied vision-language model built on a Hy3-A3B language backbone and Hy-ViT2 vision encoder with a Mixture-of-Experts design that activates only ~3B parameters. Guided by an action-centric capability taxonomy (Action-Relevant State Understanding, Action-Transition Reasoning, Sequential and Adaptive Reasoning), the authors curate pre- and post-training data mixtures intended to instill physical-world agent competence. On a suite of 38 benchmarks spanning embodied perception, physical-world understanding, and embodied reasoning, the model is reported to achieve the best result among similarly sized models on 19 benchmarks, to outperform Qwen3.6-A3B and Cosmos 3, to improve 8.4% on average over Hy-Embodied-0.5 MoT-2B, and to approach the performance of a prior 32B-activated model, while also showing strong multi-turn long-horizon agentic behavior.

Significance. If the reported gains hold under independent scrutiny, the work would be a useful engineering contribution to efficient embodied foundation models: a ~3B-activated MoE system that approaches much larger prior models on physical-world and agentic tasks would matter for latency-sensitive deployment. The explicit action-centric taxonomy and the claim of systematic data curation from pre-training onward are potentially transferable design ideas. However, because only the abstract is available, none of the load-bearing evidence (data recipes, contamination controls, per-benchmark tables, agentic protocols, or ablations) can be inspected, so the significance remains conditional on verification that is currently impossible.

major comments (3)

The central performance claims (best on 19/38 benchmarks, +8.4% average over Hy-Embodied-0.5 MoT-2B, near-32B performance, strong multi-turn agentic results) cannot be audited from the abstract alone. No tables, error bars, evaluation protocols, or per-benchmark breakdowns are provided. Without these, the leaderboard statements cannot be treated as verified results and the manuscript is not reviewable in its present form.
The action-centric taxonomy is presented as the organizing principle for both data curation and evaluation. The abstract supplies no independent evidence that the 38-benchmark suite is free of distribution match or contamination with the curated mixtures, nor any ablation that isolates the taxonomy's contribution from generic scaling or data volume. This leaves the causal claim that the taxonomy drives the gains unsubstantiated.
The multi-turn long-horizon agentic results are asserted without any description of task protocols, success metrics, horizon lengths, or comparison baselines. These results are load-bearing for the 'physical-world agents' claim and must be specified and quantified before the claim can be assessed.

minor comments (2)

Competitor names (Qwen3.6-A3B, Cosmos 3) and the prior Hy-Embodied-0.5 MoT-2B should be accompanied by precise citations and activated-parameter counts so that the 'similarly sized' and 'near-32B' comparisons are unambiguous.
The three taxonomy dimensions are named but not operationally defined in the abstract; even a short formal definition of each would improve clarity for readers.

Circularity Check

0 steps flagged

No circularity detectable from abstract-only material; claims are empirical leaderboard results, not derivation-by-construction.

full rationale

Only the abstract is available, so no equations, uniqueness theorems, fitted constants renamed as predictions, or load-bearing self-citation chains can be inspected. The abstract states an action-centric taxonomy used to guide data curation, then reports empirical wins on a 38-benchmark suite (best on 19/38 among similarly sized models, +8.4% over prior Hy-Embodied-0.5 MoT-2B, near prior 32B-activated performance, multi-turn agentic behavior). That is standard engineering practice: define a capability taxonomy, curate data, train, evaluate. It does not reduce any reported number to an input by construction, nor does it invoke an author-only uniqueness theorem that forces the result. Methodological risks (taxonomy-aligned eval, unauditable data mixtures, possible distribution match) are real but belong to correctness/generalization risk, not circularity under the stated rules. With no full text, no quoteable reduction of the form Eq. X = Eq. Y or fitted parameter renamed as prediction exists. Score 0 is therefore the honest finding: no significant circularity is exhibited.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 1 invented entities

Abstract-only review. Free parameters and invented entities cannot be exhaustively audited from equations or ablations. The load-bearing commitments visible in the abstract are the action-centric taxonomy as a training organizing principle, the curated data mixtures, the Hy3-A3B + Hy-ViT2 MoE stack, and the claim that the 38-benchmark suite measures physical-world agent competence.

free parameters (2)

data mixture weights / sampling ratios
Abstract states curated pre-training and post-training mixtures guided by the taxonomy; mixture weights are almost certainly tuned but not disclosed.
MoE routing / activation budget (~3B active)
Activated parameter count and expert routing are design choices that strongly affect reported efficiency-quality tradeoffs; values are stated at headline level only.

axioms (3)

ad hoc to paper Action-centric taxonomy (Action-Relevant State Understanding, Action-Transition Reasoning, Sequential and Adaptive Reasoning) is a sufficient organizing principle for embodied agent pre-training.
Introduced as the guiding structure for data and capability cultivation; not derived from a formal theory in the abstract.
domain assumption Static and agentic benchmarks in the 38-suite are valid proxies for physical-world interaction competence.
Standard embodied-AI evaluation assumption; fairness and contamination controls are not inspectable here.
domain assumption Hy3-A3B language backbone and Hy-ViT2 vision encoder are appropriate bases for efficient embodied VLMs.
Architectural prior from the authors' stack; treated as given infrastructure.

invented entities (1)

Hy-Embodied-VLM-1.0 action-centric capability taxonomy no independent evidence
purpose: Organize pre-training and post-training data to cultivate embodied agent skills from early stages.
Presented as the paper's conceptual scaffold; independent evidence would require ablations showing taxonomy-driven gains over matched non-taxonomy mixtures, which are not available in the abstract.

pith-pipeline@v1.1.0-grok45 · 6316 in / 2720 out tokens · 51058 ms · 2026-07-15T02:31:34.137852+00:00 · methodology

0 comments

read the original abstract

Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-Embodied-VLM-1.0, an efficient and powerful embodied foundation model specifically designed for embodied agents operating in the physical world. To cultivate such capabilities from the pre-training stage onward, we define an action-centric capability taxonomy comprising three progressive dimensions: Action-Relevant State Understanding, Action-Transition Reasoning, and Sequential and Adaptive Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre-training and post-training. To deliver strong physical-world understanding and interaction capabilities while supporting latency-sensitive deployment, we build our model on the Hy3-A3B language backbone and the Hy-ViT2 vision encoder. Its efficient Mixture-of-Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy-Embodied-VLM-1.0 on a comprehensive suite of 38 benchmarks covering embodied perception, physical-world understanding, and embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38 benchmarks and substantially outperforms strong competitors, including Qwen3.6-A3B and Cosmos 3. Compared with the previous-generation Hy-Embodied-0.5 MoT-2B, Hy-Embodied-VLM-1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous-generation model with 32B activated parameters. Beyond static benchmark evaluation, Hy-Embodied-VLM-1.0 also demonstrates strong performance on embodied agentic tasks requiring multi-turn interaction and long-horizon reasoning.

Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

Core claim

What carries the argument

Load-bearing premise

What would settle it

discussion (0)