{"id":"b2b3160d-8fa9-4674-8ee3-92e90aef1859","arxiv_id":"2509.00576","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A new open-world mobile manipulation dataset and a dual-system VLA model show that single-embodiment pre-training, not cross-embodiment pre-training, drives strong downstream task performance.","lead":"This paper introduces a 500-hour, single-embodiment robot dataset collected in real homes and workplaces, plus a two-part model that plans with a vision-language model and executes with a vision-language-action model. The authors find that pre-training on their single-robot dataset matters more for downstream performance than pre-training on diverse cross-robot data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stage-1 vs Stage-2 comparisons conflate pre-training data with action-decoding architecture (autoregressive FAST vs flow-matching expert), so the 'cross-embodiment pre-training degrades performance' claim is not established.","rationale":"The reader's conditional verdict correctly identifies the reliability of the progress rubrics and the potential confound in action decoding. I focus on the latter as the single most load-bearing concern: even if the rubrics were perfectly reliable, the central causal claim about cross-embodiment pre-training would not follow from the reported comparisons. The paper's own data-only ablation (Full vs Stage-2 400h) actually suggests cross-embodiment pre-training helps slightly on average, contradicting the 'degrades' narrative; the degradation appears only when comparing Stage-1-only or π0, both of which differ architecturally. This confound is easily fixable with an architecture-matched control, making the paper's contribution (dataset and dual-system recipe) still potentially valuable. I therefore keep the conditional verdict, with the explicit condition that the authors run the control experiment or soften the causal claim. The progress-rubric issue, while real, is secondary because it affects all comparisons equally and can be addressed with error bars; the confound, if unaddressed, invalidates the paper's most novel empirical conclusion.","tokens_in":11800,"tokens_out":4291,"duration_ms":47968,"concrete_test":"Run an architecture-matched control: initialize from the Stage-1 checkpoint, append a newly initialized flow-matching action expert exactly as in Stage-2 (§4.3), skip Stage-2 data, and fine-tune on the four benchmark tasks with the same 100-trajectory, 4-epoch recipe. Compare per-skill progress (Figure 11) and average scores against G0 (Scratch) and G0 (Stage-2 400h), with 95% confidence intervals over the 10 runs. If G0 (Stage-1→FM) ≈ G0 (Scratch) on chassis/torso skills, the deficit is caused by the action-decoding change, not by cross-embodiment pre-training. Also report the Full-vs-Stage2-only comparison on the same per-skill rubric to test the 'degrade' claim directly.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline ablation does not isolate pre-training data. In §4.2, Stage-1 trains only the VLM to predict FAST-quantized action tokens autoregressively; in §4.3, Stage-2 and post-training add a newly initialized flow-matching action expert supervised with L_flow. Consequently, every comparison that includes G0 (Stage-1) — e.g., 'G0 (Stage-1) performs worst' (§5.1) and Figure 11's 'cross-embodiment pre-training (Stage-1, π0) underperform scratch' — changes two variables at once: the pre-training corpus and the action-decoding objective/head. π0 is a different architecture entirely, so it cannot serve as an isolated cross-embodiment baseline. The only clean data-only ablation in the paper is G0 (Full) vs G0 (Stage-2 400h) in Figure 9, and the text says G0 (Full) achieves the highest average score — i.e., cross-embodiment pre-training helps slightly, not degrades. The degradation claim rests on the unused-Stage-1 model, whose poor performance may simply reflect the mismatch between an autoregressive tokenizer and the flow-matching fine-tuning objective, not the cross-embodiment data itself.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Galaxea Open-World Dataset, a 500-hour, 100K-trajectory, single-embodiment mobile-manipulation dataset collected in real-world scenes with subtask-level language annotations. It also proposes G0, a dual-system VLA architecture in which a Qwen2.5-VL-based planner (G0-VLM) issues subtask instructions to a PaLiGemma-based action model (G0-VLA). G0-VLA is trained in three stages: cross-embodiment autoregressive pre-training with FAST tokens (Stage-1), single-embodiment flow-matching pre-training on the Galaxea dataset (Stage-2), and task-specific post-training. Experiments on table bussing, microwave operation, bed making, and block stacking report progress scores, few-shot transfer, per-skill bed-making results, and G0-VLM instruction accuracy. The central claim is that single-embodiment pre-training is critical, and that cross-embodiment pre-training can even degrade performance when the embodiment gap is large.","tokens_in":12144,"tokens_out":3543,"duration_ms":41517,"significance":"The dataset is a potentially valuable community resource: 500 hours of consistent-embodiment, real-world, subtask-annotated demonstrations across 50 scenes and 150 tasks substantially exceed the realism and annotation granularity of many existing open datasets. The dual-system design and the systematic comparison of pre-training strategies address a question of active interest. The paper also provides useful baseline comparisons, including π0 and several closed/open VLMs, and reports 10-run averages for the physical benchmarks. If the central empirical claim were cleanly supported, the work would be an important contribution to the debate on cross-embodiment pre-training. However, as detailed below, the central ablation currently conflates pre-training data composition with action-decoding architecture, so the degradation claim is not established by the presented experiments.","major_comments":[{"comment":"The headline claim that cross-embodiment pre-training can degrade performance is confounded. Stage-1 trains only the VLM with FAST-quantized autoregressive action tokens, while Stage-2 and post-training add a newly initialized flow-matching action expert trained with L_flow. Thus every comparison involving G0 (Stage-1) or π0 changes both the pre-training corpus and the action-decoding objective/head. The only data-only ablation with a fixed action head is G0 (Full) vs. G0 (Stage-2 400h), and Figure 9 shows G0 (Full) has the highest average progress score. That result supports the opposite conclusion: adding cross-embodiment data helps slightly. The degradation result in §5.1 and Figure 11 rests on models with a different action head, so it cannot be attributed to cross-embodiment data alone. Please either add a controlled comparison (e.g., a flow-matching head trained on cross-embodiment","section":"§4.2–4.3, §5.1, Figure 9"},{"comment":"The central quantitative evidence relies on author-defined progress scores with no reported variance. The paper states that each test is run 10 times and averaged, but Figure 9 and Figure 10 do not show error bars, confidence intervals, per-run scores, or statistical tests. Given that the differences between G0 (Full) and G0 (Stage-2 400h) may be small, and that the rubric assigns discrete points per subtask (e.g., 1 point per pick/place), the ranking may not be robust. I request at least standard deviations or raw score distributions, and ideally a statement of inter-annotator reliability if the 'successful pick/place' judgments are human-assessed.","section":"§5, Appendix A.1"},{"comment":"The benchmark tasks appear to be drawn from the same distribution as the Galaxea pre-training and fine-tuning data, but the paper never specifies the data split. If the 100 training trajectories per task and the evaluation episodes come from the same 150-task collection, then the results measure in-distribution fine-tuning rather than open-world generalization. The paper claims 'open-world generalization' in the abstract and §1. Please clarify whether the evaluation scenes, objects, and task instances are disjoint from all pre-training and fine-tuning data, and if not, temper the generalization claim accordingly.","section":"§5, §4.4"},{"comment":"The G0-VLM instruction-accuracy results are hard to interpret without knowing whether the benchmark tasks and their subtask annotations were held out from G0-VLM training. G0-VLM is fine-tuned on Dlabeled sampled from the Galaxea dataset, and the benchmark appears to use tasks from the same dataset. If the exact tasks (e.g., 'Table bussing', 'Microwave operation') appear in the training sample, the >50% margin over baselines may reflect memorization of task-specific annotation patterns. Please specify the task-level and scene-level split used for G0-VLM evaluation, and report results on held-out tasks if available.","section":"§6, Table 1"}],"minor_comments":[{"comment":"Typo: 'GalaXea' should be 'Galaxea' for consistency with the rest of the paper.","section":"§4.2"},{"comment":"The progress rubric would benefit from operational definitions of 'successful pick', 'successful place', 'correct food', and 'flatten the quilt'. As written, the scoring criteria are subjective and may not be reproducible.","section":"Appendix A.1"},{"comment":"Please add error bars or per-run score distributions. The current presentation makes it impossible to assess the reliability of the reported differences.","section":"Figure 9 and Figure 10"},{"comment":"The abstract and §3 give both '100K demonstration trajectories' and '500 hours'; the relationship between trajectories and hours is not stated. Please clarify the average trajectory length.","section":"§3"},{"comment":"The statement that image observations are not fed to DeepSeek-R1, with the argument that 'reasoning capabilities of an LLM are sufficient', is an interesting design choice but would benefit from an ablation or at least a discussion of its limitations, since the VLM planner must eventually ground instructions in visual scenes.","section":"§4.5"}],"recommendation":"major_revision","confidential_remarks":"The dataset and dual-system framework are likely to be of interest to the community, but the paper's central empirical claim about cross-embodiment pre-training needs a cleaner experimental isolation. The current text overstates the strength of the evidence, and the requested revisions (controlled action-head comparison, variance reporting, and data-split clarification) are within the scope of a major revision. I also note that the paper states the dataset/models will be open-sourced 'in the coming weeks'; if this is not yet available, the reproducibility assessment may be limited, but this is not itself a blocker."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper as a dataset contribution: 500 hours, 150 tasks, 50 real scenes, single embodiment, subtask annotations. That is genuinely scarce and valuable. The G0 dual-system VLM+VLA is not architecturally novel—it follows Hi Robot, OpenHelix, pi0.5—but the staged training recipe and the question about cross-embodiment pre-training are worth discussing.\n\nWhat the paper does well: the dataset is collected in real homes and offices with a consistent embodiment, with careful subtask-level language annotations. That is a real resource for mobile manipulation research. The experiments are broad: tabletop, few-shot, long-horizon. The few-shot results with Stage-2 pre-training look useful for practitioners.\n\nThe soft spot is the central ablation. In Section 4.2, Stage-1 trains only the VLM to predict FAST-quantized action tokens autoregressively. In Section 4.3, Stage-2 and post-training add a newly initialized flow-matching action expert supervised with L_flow. So comparing G0 (Stage-1) with G0 (Stage-2) or G0 (Full) changes two variables at once: the pre-training corpus and the action-decoding objective/head. π0 is a different architecture entirely. The only clean data-only comparison is G0 (Full) vs G0 (Stage-2 400h) in Figure 9, and the text says G0 (Full) achieves the highest average score—cross-embodiment pre-training helps slightly, not degrades. The degradation claim rests on the unused Stage-1 model, whose poor performance may simply reflect the mismatch between an autoregressive tokenizer and the flow-matching fine-tuning objective, not the cross-embodiment data. This is a real confound and the paper should either fix the experimental design or temper the claim.\n\nOther issues: no error bars or statistical tests despite 10-run averages; the progress rubrics are author-defined and no inter-annotator agreement is reported; the benchmark tasks are drawn from the same distribution as the dataset; and the G0-VLM comparison against Gemini/Qwen shows an unexpectedly large gap, which is suspicious given that G0-VLM was fine-tuned on the same task distribution. Also, no artifacts are released at submission, so the dataset is not yet verifiable.\n\nThis paper deserves a serious referee. The dataset alone warrants careful review. The authors should address the ablation confound, add variance estimates, and ideally benchmark on independent tasks. For a reading group, it is a good case study in confounding in robot learning papers. I would cite the dataset once it is released.","headline":"A valuable real-world dataset with an honest but partially confounded ablation; the 'cross-embodiment pre-training can degrade' claim does not survive the architecture change.","tokens_in":12606,"tokens_out":2135,"would_cite":true,"duration_ms":25302,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cross-embodiment pretraining can hurt a robot VLA; matching the robot's own embodiment is what wins.","keywords":["robot manipulation dataset","vision-language-action model","dual-system planner-executor","single-embodiment pretraining","cross-embodiment transfer","mobile manipulation","few-shot learning","flow matching action generation"],"falsifier":"A controlled comparison that keeps the action decoder identical across all conditions: initialize the flow-matching action expert before Stage-1, or train all variants with the same action-decoding objective, and score each condition with independent blind raters and confidence intervals. If cross-embodiment pretraining no longer lags behind single-embodiment-only pretraining under this setup, the paper's central ranking would not hold.","tokens_in":1787,"feed_emoji":"🤖","tokens_out":1767,"duration_ms":69900,"temperature":0.7,"pith_summary":"This paper tries to establish that for vision-language-action (VLA) robot models, the most valuable pretraining is on a large, consistent dataset from the very robot that will execute the task, not on diverse cross-embodiment data. It introduces a 500-hour, single-embodiment dataset collected in real homes, offices, retail, and catering spaces, with subtask-level language labels, and a dual-system model called G0 that pairs a slow planner with a fast executor. Through controlled fine-tuning benchmarks, the authors claim that adding cross-embodiment pretraining on top of single-embodiment pretraining can actually degrade performance when the robot bodies differ substantially. A sympathetic reader would care because this challenges a common scaling assumption: that more heterogeneous robot data is always better. If correct, the result points dataset builders and robot labs toward investing in embodiment-matched, real-world data rather than pooling arbitrary robot data.","feed_headline":"Cross-embodiment pretraining can hurt robot VLA performance","feed_subtitle":"A 500-hour same-robot dataset plus embodiment-specific pretraining beats general robot data on real-world tasks.","key_machinery":"The load-bearing mechanism is the three-stage curriculum: Stage-1 pretrains the vision-language backbone autoregressively on roughly 1,700 hours of cross-embodiment data using discrete action tokens from a FAST-style action tokenizer; Stage-2 initializes a flow-matching action expert and trains it on the single-embodiment Galaxea Open-World Dataset with subtask-level language labels; post-training fine-tunes on at most 100 task demonstrations. The G0 dual system then couples a fine-tuned open VLM planner (System 2) with the VLA executor (System 1). The argument hinges on the distinction between the two stages: Stage-1 transfers generic visual-language priors and simple action patterns, while","core_discovery":"The paper's central claim is that single-embodiment pretraining on the Galaxea Open-World Dataset is the critical stage of the three-stage curriculum. In the benchmark, G0 (Full), which combines cross-embodiment Stage-1 pretraining with single-embodiment Stage-2 pretraining, achieves the highest average progress score, but G0 (Stage-1) alone performs worse than all other pretrained models, and in some embodiment-specific skills, cross-embodiment pretraining underperforms even training from scratch. The authors interpret this as evidence that when the embodiment gap between the pretraining platform and the target robot is large, cross-embodiment knowledge transfer can become negative, and tha","pith_inferences":["The paper leaves implicit that the comparison between Stage-1 and later stages also changes the action decoding: Stage-1 uses autoregressive discrete action tokens, while Stage-2 and post-training use a newly initialized flow-matching action expert. Part of the reported degradation could stem from this decoder mismatch rather than from the embodiment gap alone; a controlled test holding the action","A testable extension is to run the same curriculum on embodiments that are already well represented in existing cross-embodiment datasets, such as single-arm tabletop robots; the paper's logic predicts cross-embodiment pretraining would then show a benefit, inverting the bed-making result.","The finding suggests that a general principle for VLA scaling may be 'embodiment-aligned data first, diverse data second.' If borne out, it implies that open-world robot learning should be organized around fleets of identical robots rather than heterogeneous data pooling.","The hand-defined progress rubrics could be expanded into continuous, independently rated success metrics; this would let the research community verify whether the reported, sometimes small, score differences persist under stricter measurement."],"forward_implications":["If the central claim is correct, cross-embodiment pretraining should not be assumed beneficial; its value depends on the embodiment gap and may be negative for robots with unique kinematic structures.","Single-embodiment data at scale provides strong few-shot adaptation: models pretrained on the target robot can be fine-tuned successfully with only 20 demonstrations per task.","Subtask-level language annotations are a key ingredient: they enable both precise instruction following by the executor and high-accuracy planning by the VLM, with fine-tuned planning accuracy exceeding general-purpose closed models by over 50 percentage points on the reported tasks.","The benchmark results imply that dataset curation should prioritize real-world scene and object diversity while keeping the embodiment fixed, rather than maximizing heterogeneous embodiment coverage.","The open-sourcing of the dataset and models positions this as a reference point for future VLA training recipes and for comparing pretraining data composition choices."],"supporting_citations":[{"why":"Supplies the cross-embodiment pretraining mixture and the main comparison point for dataset diversity and consistency.","marker":"[1]"},{"why":"Provides the π0 baseline and the flow-matching action-expert design used in Stage-2 and post-training.","marker":"[12]"},{"why":"Supplies the two-generation-paradigm training recipe that G0 follows for combining autoregressive and flow-matching action models.","marker":"[16]"},{"why":"Comparison for a single-embodiment large-scale robot dataset collected on one platform.","marker":"[21]"},{"why":"Comparison for in-the-wild single-embodiment manipulation data and a reference point for real-world collection.","marker":"[22]"},{"why":"Comparison for large-scale multi-embodiment data that the paper positions against its own single-embodiment, real-world dataset.","marker":"[24]"},{"why":"Part of the conflicting evidence on whether large-scale cross-embodiment or web-data pretraining helps robot control.","marker":"[25]"},{"why":"Supplies the pretrained vision-language backbone for the VLA's perception and language understanding.","marker":"[29]"},{"why":"Supplies the open VLM backbone for the G0-VLM planner before instruction tuning.","marker":"[30]"}],"fun_headline_variants":["Cross-embodiment pretraining can hurt robot VLA performance","Why same-robot data is key for robot VLA training","General robot data can backfire for VLA pretraining","Same-robot pretraining outperforms cross-embodiment for VLA"],"cache_read_input_tokens":14336,"weakest_assumption_plain":"The central ranking rests on the hand-defined progress rubrics, averaged over ten runs per condition, being reliable and sensitive enough to separate models whose scores may differ by small amounts, while the comparisons isolate pretraining data composition rather than the change in action decoding.","fun_headline_variants_meta":{"raw":{"variants":["Cross-embodiment pretraining can hurt robot VLA performance","Why same-robot data is key for robot VLA training","General robot data can backfire for VLA pretraining","Same-robot pretraining outperforms cross-embodiment for VLA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001082,"raw_usage":{"total_tokens":4336,"prompt_tokens":691,"completion_tokens":3645,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":435,"completion_tokens_details":{"reasoning_tokens":3573}},"tokens_in":435,"tokens_out":3645,"duration_ms":31389,"temperature":1.0,"reasoning_tokens":3573,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:26:48.737919+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled comparison that keeps the action decoder identical across all conditions: initialize the flow-matching action expert before Stage-1, or train all variants with the same action-decoding objective, and score each condition with independent blind raters and confidence intervals. If cross-embodiment pretraining no longer lags behind single-embodiment-only pretraining under this setup, the paper's central ranking would not hold.","supporting_citations":[{"cited_title":"Bridgedata v2: A dataset for robot learning at scale","cited_arxiv_id":null,"evidence_quote":"Comparison for a single-embodiment large-scale robot dataset collected on one platform."}],"review_version":1}