{"id":"067c1aff-767f-494e-b3df-f66396491038","arxiv_id":"2505.05753","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A policy trained on about one thousand simulated robot bodies generalizes progressively better to unseen bodies as the number of training bodies grows, and it transfers zero-shot to two real robots.","lead":"This paper tests whether a robot controller trained on many different robot bodies becomes better at controlling unseen robot bodies. It finds that adding body diversity helps generalization more than adding more data on a few bodies, and the resulting policy walks on two real robots without retraining.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scaling trend may reflect grid-coverage interpolation within one procedural generator rather than a general embodiment scaling law; the OOD checks do not test the trend itself.","rationale":"The reader's weakest assumption identifies the same core issue: held-out embodiments are sampled from the same procedural generator as training embodiments, so the evaluation measures interpolation within one hand-designed distribution rather than generalization to a broader embodiment space. This is indeed the most load-bearing concern. The reader's verdict of CONDITIONAL is appropriate: the paper is a large, internally consistent empirical study, and the scaling trends are plausible, but the central generalization claim is contingent on the representativeness of GENBOT-1K. The reader also flags missing error bars and a non-matched data-scaling baseline; I find the data-scaling baseline C8 to be a reasonable total-data control, since it varies trajectories per embodiment on a fixed 5% subset and saturates quickly, so I would not press that point as the primary concern. The OOD results in Appendix D are encouraging but address only the full policy at extreme knee-limit values, not the scaling trend across training subset sizes. Thus the decisive, still-open question is whether the positive Jtest-versus-|Etrain| trend persists when the test embodiments are drawn from parameter combinations that are not on the discrete training grid and from axes explicitly held fixed, such as mass distribution, joint damping, and actuation type. Performing that evaluation with the existing subset checkpoints is feasible and would settle whether the reported scaling is a genuine embodiment-diversity effect or a grid-coverage artifact. Until then, CONDITIONAL is the right verdict, and my read does not move it.","tokens_in":29459,"tokens_out":8091,"duration_ms":95358,"concrete_test":"Using the already-trained checkpoints for the scaling subsets (i = 0.05, 0.2, 0.4, 0.6, 0.8, 1.0), evaluate each policy on an out-of-grid test set: (a) the Appendix D embodiments with knee-limit scales 0.1 and 0.001, and (b) a newly generated set in which geometry parameters are sampled continuously within the same ranges and mass distribution, joint damping, and actuation type are varied, which the Limitations section identifies as held fixed. If Jtest still rises monotonically with the number of training embodiments on these out-of-grid test sets, the scaling trend is robust; if it flattens or declines, the reported curves are an artifact of discrete-grid coverage rather than evidence for an embodiment scaling law.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, that increasing the number of training embodiments improves generalization to unseen embodiments, rests on evaluation against held-out embodiments sampled from the same discrete procedural generator used to create the training set (Sec. 3.1, Sec. 4.1, Appendix B.2). The generator's parameter grid is coarse: thigh and calf length scales take only five values, foot size two values, knee-limit scales three values, and topology is varied by knee count in {0,1,2,3} (Table 6). As the training subset grows from 5% to 100%, a test embodiment is increasingly likely to share many or all parameter values with some training body, so the observed rise in Jtest may measure nearest-neighbor coverage of a finite grid rather than a generalizable scaling property of embodiment diversity. Appendix D tests OOD knee-limit scales 0.1 and 0.001, but only for the full policy; it does not test whether the scaling trend itself survives outside the training grid. The real-world robots Go2 and H1 are described as similar in kinematic structure to training embodiments, and are excluded from the training set, but they are likewise near the generated distribution. Because the paper's headline conclusion is about a broad embodiment scaling phenomenon, not just this generator, this external-validity gap is the most load-bearing assumption. The data-scaling baseline C8 is a reasonable total-data control, so the weak point is not the comparison to data scaling but the distribution from which both training and test embodiments are drawn.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether increasing the number of training robot embodiments improves generalization to unseen embodiments, a hypothesis the authors term an \"embodiment scaling law.\" Using locomotion as a testbed, they procedurally generate 1,012 robots spanning humanoids, quadrupeds, and hexapods, with variations in topology, geometry, and joint kinematics. They train single-embodiment expert policies with RL and distill them into a single URMA-based policy via behavior cloning, varying the number of training embodiments from 5% to 100% of an 80% training pool and evaluating on a fixed 20% held-out set. They also compare against a data-scaling baseline (C8) that increases trajectories on a fixed 5% embodiment subset, and they demonstrate zero-shot transfer of the full policy to the Unitree Go2 and H1 robots in the real world, including with artificially restricted knee joint limits. The paper reports positive scaling trends in all three morphology classes and in the combined cross-class setting, and concludes that embodiment scaling enables substantially broader generalization than data scaling.","tokens_in":29665,"tokens_out":7397,"duration_ms":75189,"significance":"If the central claim holds, this would be an important first large-scale empirical step toward understanding how embodiment diversity drives generalization in robot learning, with implications for generalist robot policies, adaptive control, and morphology co-design. The study's strengths are its unprecedented scale (1,012 embodiments, 2 trillion simulation steps), the two-stage RL-to-distillation pipeline that makes such scale tractable, the fixed held-out test set design, the honest limitations section, and the genuine zero-shot transfer to two real robots including a constrained-joint deployment. The latent-space analyses strengthen the plausibility of the mechanism. However, the evidence is currently insufficient to establish a \"law\": the scaling curves are single runs without uncertainty quantification, the test distribution is the same finite procedural generator as the training distribution, and the data-scaling comparison is not matched on total samples or compute. These are fixable with additional experiments and analyses, but they are load-bearing for the headline claims.","major_comments":[{"comment":"The scaling curves C1-C8 are each single training runs with no error bars, confidence intervals, or multiple seeds. The central claims that Jtest increases monotonically with the number of training embodiments, that quadruped and hexapod performance saturates around 100 embodiments, and that humanoid performance \"continues to improve steadily\" cannot be distinguished from run-to-run variance at this level of evidence. I request at least 3-5 seeds per condition, or a bootstrap/confidence-interval analysis over the test embodiments, to quantify the trend and its saturation behavior.","section":"Sec. 4.1, Figure 4"},{"comment":"The held-out test embodiments are sampled from the same discrete procedural generator used to create the training set, whose parameter grid is coarse: thigh and calf length scales take five values, foot size two values, knee-limit scales three values, and topology is varied by knee count in {0,1,2,3}. As the training subset grows from 5% to 100%, a test embodiment is increasingly likely to share all parameter values with some training body, so the observed rise in Jtest may measure nearest-neighbor coverage of a finite grid rather than a generalizable scaling property of embodiment diversity. The out-of-distribution experiments in Appendix D evaluate only the policy trained on the full set, not whether the scaling trend itself survives outside the training grid. To support the claimed law, the scaling curves should be recomputed on a genuinely external family of embodiments, for example from a different generator, from human-designed robots, or using parameter values not present in the training grid.","section":"Sec. 4.1, Appendix B.2, Table 6"},{"comment":"The data-scaling comparison is confounded: C8 fixes the embodiment set at 5% and varies the number of trajectories per embodiment, while the embodiment-scaling curves vary the number of embodiments with roughly fixed per-embodiment data. Consequently, the total number of demonstration samples changes along both axes, so the conclusion that \"embodiment scaling is essential\" is not supported by a matched comparison. Please include a control in which total sample count (or compute) is held constant while the ratio of the number of embodiments to per-embodiment data is varied.","section":"Sec. 4.1, curve C8"},{"comment":"The cross-class comparisons (C4 vs C5-C7) are made on unnormalized rewards whose scales differ across morphology classes, as the caption itself acknowledges. The claimed 2-5x improvement in average reward on the combined test set could be dominated by the class(es) with larger-magnitude rewards where single-class policies fail. Please report per-class normalized rewards or a per-class performance table before drawing the conclusion that training across morphologies enables \"substantially broader generalization.\"","section":"Sec. 4.1, Figure 4 caption"}],"minor_comments":[{"comment":"The softmax denominator in Eq. (3) is written as a sum over the latent dimension Ld, but the attention normalization should be over the joints J within an embodiment; please correct the equation and clarify the output dimensionality of f_phi(d_j).","section":"Eq. (3)"},{"comment":"Section 4.2 states that the full policy was trained on 817 simulated embodiments, but Table 9 in Appendix C.3 reports a training set of 808 embodiments (278 humanoid + 265 quadruped + 265 hexapod); please reconcile this inconsistency.","section":"Sec. 4.2 vs Appendix C.3"},{"comment":"The paragraph on the software-level joint-limit implementation contains a redundant and slightly contradictory pair of sentences: \"we introduce a software-level joint-limit layer into the control loop\" and then \"Instead, we implemented a software-based solution...\"; please edit for clarity.","section":"Appendix E.2"},{"comment":"The x-axis of Figure 4 mixes two different quantities (proportion of training embodiments for C1-C7 and data scale for C8) on the same axis; please relabel or split the panels to avoid conflating these axes.","section":"Figure 4"},{"comment":"The paper uses the term \"embodiment scaling laws\" in the title, abstract, and conclusion, but no functional form of a law is actually fitted or verified; please consider tempering the terminology to \"embodiment scaling trends\" unless a quantitative law is established.","section":"Abstract and Section 5"},{"comment":"References [40] and [89] both refer to the same paper (GET-Zero); please cite it only once.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This paper is likely to be influential in the cross-embodiment learning community and fits the scope of the journal well. The main weaknesses are the lack of uncertainty quantification on the scaling curves, the finite-generator confound in the held-out test distribution, and the unmatched data-scaling comparison; these are load-bearing but fixable with additional experiments and re-analysis. The numeric inconsistency about the training set size (817 vs 808) should be corrected. After revision, the claims about 'scaling laws' should be more carefully calibrated to the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know: this is the first systematic embodiment-scaling study at the ~1,000-robot scale, and the core empirical claim—held-out embodiment generalization improves as you add training embodiments—looks real within the distribution they built. The bigger open question is whether that trend is a law or an artifact of one procedural generator, and the paper does not yet answer it.\n\nWhat is new and good: GENBOT-1K is a serious asset: 1,012 body blueprints across humanoids, quadrupeds, and hexapods, with topology, geometry, and kinematics varied. They hold out a fixed test set, train on 5–100% of the pool, and show monotone reward gains in all three classes and across classes. They also report zero-shot transfer to the Unitree Go2 and H1, including artificially restricted knee joints. That is a lot of work, and the reproducibility detail is strong: fixed splits, PPO hyperparameters, DR ranges, reward tables, architecture, data collection, and training pipeline are all specified. The paper earns credit for shipping a benchmark-scale dataset recipe.\n\nSoft spots, in proportion. Figure 4 curves are single runs, no error bars or seeds, so the scaling trend is not quantified with uncertainty. The data-scaling comparison (C8) is a reasonable control but not matched on total samples or compute, so “substantially broader than data scaling” is under-supported. The bigger issue is external validity: training and test embodiments are sampled from the same discrete generator, with coarse parameter choices (five thigh/calf scales, two foot sizes, three knee ranges). As the training set grows, test bodies become closer neighbors in a finite grid, so the trend may be measuring coverage of that grid rather than a general embodiment scaling law. The OOD experiment in Appendix D only tests the full policy, not whether the trend itself survives outside the grid. The Limitations section honestly concedes that mass, damping, and actuation are held fixed and real-world validation is two platforms, but the abstract and conclusion still use “scaling laws” language that outruns the evidence. None of this breaks the internal result; it just means the headline should be softer and the next paper should test the trend on a second generator or on held-out parameter distributions.\n\nWho this is for: people building generalist locomotion policies and anyone evaluating cross-embodiment benchmarks. It deserves a serious referee—send it out, but expect the authors to add error bars, match the data-scaling baseline, and temper the law wording.","headline":"First ~1k-robot embodiment scaling study with a real in-distribution trend; the law-level claim needs stronger external validation.","tokens_in":30271,"tokens_out":3836,"would_cite":true,"duration_ms":37093,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Increasing the number of robot bodies a locomotion policy is trained on improves its ability to control unseen bodies, and this embodiment scaling helps more than adding data on a fixed set of bodies.","keywords":["cross-embodiment learning","embodiment scaling laws","robot locomotion","procedural embodiment generation","behavior cloning distillation","attention-based policy architecture","zero-shot sim-to-real transfer","morphology generalization"],"falsifier":"Retrain the scaling curves with test bodies drawn from a separate generator that includes factors the paper holds fixed, such as mass distribution, joint damping, and actuation type, and with joint limits outside the training ranges; if held-out reward no longer rises with training embodiment count, the observed law is an artifact of sampling density inside the generator.","tokens_in":29212,"feed_emoji":"🤖","tokens_out":8339,"duration_ms":88283,"temperature":0.7,"pith_summary":"The paper sets out to establish an embodiment scaling law for robot locomotion: increasing the number of distinct robot bodies used in training improves a single policy's ability to control bodies it has never seen. It supports this with GENBOT-1K, about 1,000 procedurally generated humanoids, quadrupeds, and hexapods, and with scaling curves that rise as the training embodiment count grows from 5% to 100% of the pool. The same experiments show that adding demonstration data on a fixed small set of bodies saturates quickly, so the gains are attributed to embodiment diversity rather than data volume. If the law holds, the way to build a generalist locomotion controller is to train it on many bodies, not just many trajectories, and such a policy can transfer zero-shot to new hardware.","feed_headline":"Training on more robot bodies, not more data, generalizes better","feed_subtitle":"Policies trained on 1,000 procedurally varied robots transfer zero-shot to unseen simulated and real robots.","key_machinery":"The load-bearing mechanism is an embodiment-conditioned attention policy built on URMA, a joint-level architecture that handles arbitrary robot morphologies by splitting observations into fixed general features and variable-length per-joint features. A multi-head attention encoder fuses joint observations, weighted by learned joint-description vectors derived from the embodiment descriptor $\\phi(e)$, so the same network can output actions for bodies with different joint counts and kinematic properties. The scaling study is carried out on GENBOT-1K, a dataset of 1,012 procedurally generated blueprints varying topology (number of knee joints), geometry (link lengths and sizes), and kinematics (joint limits), with a fixed 20% held-out test set. Training follows a two-stage pipeline: per-embodiment RL experts provide demonstrations, and a single student policy is distilled from them by behavior cloning.","core_discovery":"The paper's central claim is that, for flat-ground proprioceptive locomotion, generalization to held-out robot bodies improves as the number of training embodiments grows, and that this embodiment scaling is not reducible to data scaling. On a fixed test set of 204 procedurally generated robots, held-out reward roughly doubles when the training embodiment fraction rises from 5% to 100% of the generated pool. A control trained on only 5% of the bodies but with four times as many demonstrations per body nearly saturates, which the paper reads as evidence that body diversity, not trajectory count, drives the gain. Training across humanoids, quadrupeds, and hexapods together yields one policy that beats class-only policies on the mixed test set and transfers zero-shot to the Unitree Go2 and H1, including stable adaptation when a real knee joint's range is artificially reduced to 20% of nominal.","pith_inferences":["If the scaling trend reflects a general law rather than generator interpolation, the same axis should appear in manipulation and whole-body control, though the curve may shift because those tasks add perceptual variation and contact-rich dynamics beyond morphology.","A stricter test of the law would adversarially select training bodies to maximize coverage of kinematic extremes; if the curve flattens, the active variable is distribution coverage rather than embodiment count itself.","The joint-limit adaptation shown on one real robot suggests that a single policy could control modular or reconfigurable robots whose geometry changes between deployments, provided the changes stay within the parameter ranges the generator was built from.","The data-scaling saturation result implies a practical stopping rule for data collection: once additional trajectories from a fixed robot stop improving held-out performance, the remaining budget is better spent on new morphologies, but this rule is an extrapolation beyond the paper's measured regime."],"forward_implications":["A single locomotion policy trained on a sufficiently diverse set of simulation bodies can be deployed on new robot hardware without per-robot fine-tuning, as demonstrated by the zero-shot real-world transfers.","When the goal is cross-embodiment generalization, spending a fixed data budget on more robot bodies is more effective than collecting more demonstrations from a few bodies.","Scaling curves can serve as a planning tool: harder morphology classes, such as humanoids, may require more training embodiments to reach the same level of held-out performance as easier classes.","The policy's latent representations organize by morphology class and joint count, suggesting that embodiment-aware controllers can adapt to a changed body, such as a restricted joint, by reading its descriptor and applying the nearest learned behavior."],"supporting_citations":[{"why":"Supplies the Unified Robot Morphology Architecture that the paper extends with multi-head attention to handle arbitrary joint configurations.","marker":"[41]"},{"why":"Supplies the generalist-specialist two-stage training idea (RL experts plus distillation) used to scale learning to about 1,000 embodiments.","marker":"[42]"},{"why":"Provides the PPO algorithm used to train all single-embodiment expert policies whose demonstrations are distilled.","marker":"[58]"},{"why":"Defines the approach of abstracting physical characteristics from real robots into a grounded, diverse robot design space used for the procedural generator.","marker":"[57]"},{"why":"Prior simulation study of zero-shot embodiment generalization at roughly 10^2 robots that this work scales past with roughly 10^3 robots.","marker":"[40]"},{"why":"Mirrors the offline multi-robot data-reuse pipeline that motivates collecting expert demonstrations once and reusing them across training runs.","marker":"[21]"}],"fun_headline_variants":["Robot body diversity beats data quantity for zero-shot transfer","More robot bodies, less data: scaling law for locomotion","Embodiment scaling: 1,000 bodies generalize to unseen bots","Train on many robot shapes, transfer to new ones"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The held-out test embodiments come from the same procedural generator as the training bodies, and the real robots are similar in kinematic structure to bodies in the training set, so the measured scaling is interpolation inside one hand-designed distribution rather than generalization across the full space of possible robot bodies.","fun_headline_variants_meta":{"raw":{"variants":["Robot body diversity beats data quantity for zero-shot transfer","More robot bodies, less data: scaling law for locomotion","Embodiment scaling: 1,000 bodies generalize to unseen bots","Train on many robot shapes, transfer to new ones"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000447,"raw_usage":{"total_tokens":2214,"prompt_tokens":862,"completion_tokens":1352,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":478,"completion_tokens_details":{"reasoning_tokens":1285}},"tokens_in":478,"tokens_out":1352,"duration_ms":12044,"temperature":1.0,"reasoning_tokens":1285,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:57:20.124348+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the scaling curves with test bodies drawn from a separate generator that includes factors the paper holds fixed, such as mass distribution, joint damping, and actuation type, and with joint limits outside the training ranges; if held-out reward no longer rises with training embodiment count, the observed law is an artifact of sampling density inside the generator.","supporting_citations":[],"review_version":1}