{"id":"47f30a70-240d-49ca-9da9-18681bf77a45","arxiv_id":"2506.00276","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"An LLM-driven framework that jointly proposes robot morphologies and reward functions, using diversity reflection and alternating refinement, claims large efficiency gains over baselines across eight locomotion tasks.","lead":"RoboMoRe uses a large language model to propose both robot body shapes and reward functions in a co-optimization loop, then trains each candidate in simulation to measure performance. The authors report large efficiency gains over human designs and prior LLM-based methods across eight MuJoCo locomotion tasks, but the comparison baseline and reported numbers contain inconsistencies.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Efficiency comparison in Table 2 is not apples-to-apples: only RoboMoRe is stated to optimize efficiency, so baseline gaps may reflect objective mismatch rather than superior co-design.","rationale":"Reading the paper in good faith, the central claim is that joint morphology-reward optimization beats human and automated baselines on eight tasks. The only quantitative support is Table 2, with efficiency as the headline metric. The paper explicitly says efficiency is RoboMoRe's optimization target, but it never says the same for the baselines. That asymmetric objective makes the comparison structurally unfair: efficiency is fitness divided by volume, so a method that ignores volume will be systematically penalized. This is particularly damaging because RoboMoRe's prompts explicitly encourage reducing material cost (Appendix A.1.2), giving it an unfair advantage on the volume axis. The fitness column itself shows RoboMoRe is not dominant on raw performance (Half-Cheetah, Swimmer fitness lower than Eureka/Human), so the 'outperforms' claim depends entirely on the efficiency metric. Without evidence that baselines optimized the same objective, the conclusion is unsupported. I also note the Table 2 vs Table 4 inconsistency as a secondary red flag: the 'Full' ablation efficiencies are 2-5x lower than the main results for every task, which should be reconciled (e.g., different evaluation budget, seeds, or a typo). The concrete test proposed—checking and re-running baselines with the efficiency objective—would settle the primary concern. The reader identified the same core weakness, so I agree with the verdict to reject pending clarification.","tokens_in":19026,"tokens_out":6337,"duration_ms":58771,"concrete_test":"Locate the released code or Appendix D's comparison-method config and check what objective function the BO acquisition function, Eureka's sample selection, and Eureka(Mor.)'s morphology proposals maximize. If any of them use raw fitness rather than fitness/volume, rerun those baselines with the efficiency objective (same 5e5/1e6 step budgets, same volume computation, same seeds) and compare Table 2. If the efficiency gap shrinks to within variability (e.g., factor <2 or overlapping confidence intervals), the headline 'significantly outperforms' claim is not supported; if the gap persists at the reported magnitude, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 states that 'efficiency serves as the core metric in our optimization iterations' for RoboMoRe, but the descriptions of Bayesian Optimization, Eureka, and Eureka(Mor.) in the same section and Appendix D never specify their optimization objective. If these baselines maximize raw fitness (distance/height), their efficiency in Table 2 is computed as fitness/volume for designs that never traded off volume, so the large efficiency gaps (e.g., Ant 31,038 vs 68 for Human; Half-Cheetah 495,374 vs 12,158 for Eureka) are partly an artifact of the evaluation metric. This is not a minor detail: the abstract's claim of 'significantly outperforms' rests entirely on the efficiency metric, and Table 2's fitness column shows RoboMoRe is not the best on raw fitness (e.g., Half-Cheetah fitness 135.71 vs Eureka 257.55). The formal problem definition in Eq. (2) also maximizes fitness F, not efficiency, so there is an internal inconsistency between the stated objective and the implemented one. A second, compounding issue is that Table 4's 'RoboDesign (Full)' efficiencies differ systematically from Table 2's RoboMoRe values for the same configuration (e.g., Ant 10,464 vs 31,038; Half-Cheetah 129,441 vs 495,374), suggesting unreported protocol differences or reporting errors. The objective mismatch alone is sufficient to invalidate the headline comparison unless the baselines are shown to have optimized the same efficiency objective.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RoboMoRe, an LLM-driven framework for robot co-design that jointly optimizes morphology and reward functions. The method proceeds in two stages: a coarse stage in which an LLM with a diversity-reflection mechanism proposes morphology-reward pairs, and a fine stage in which the top candidates are refined through alternating LLM-guided morphology and reward updates. Experiments are reported across eight MuJoCo environments, with efficiency (fitness divided by volume) as the headline metric. The paper claims that RoboMoRe significantly outperforms human-designed robots and prior methods such as Bayesian optimization, Eureka, and Eureka(Mor.) on this metric.","tokens_in":19353,"tokens_out":3846,"duration_ms":40142,"significance":"If the reported results were fully supported, the paper would make a useful empirical contribution: it demonstrates a task-agnostic prompting scheme for LLM-based co-design, provides a diversity-reflection mechanism that improves sample diversity, includes ablations isolating reward shaping and morphology design, and reports robustness checks across gear powers and terrains. The appendix is detailed, with full prompts and examples of generated reward functions. However, the central comparison is currently undermined by an apparent mismatch between the formal optimization objective and the metric used in implementation, by inconsistent numbers between Table 2 and Table 4 for what appears to be the same configuration, and by the absence of any error bars or statistical tests. These issues directly affect the paper's headline claim of significant outperformance, so the contribution cannot be assessed as presented.","major_comments":[{"comment":"The formal objective in Eq. (2) maximizes fitness F, but the implementation and Table 2 use efficiency = fitness/volume, and Section 5 states that efficiency is the core metric in the optimization iterations. The manuscript never states whether Bayesian Optimization, Eureka, and Eureka(Mor.) were run optimizing the same efficiency objective or raw fitness. If the baselines optimized raw fitness, their volume-heavy designs are penalized only at evaluation time, so the large efficiency gaps in Table 2 (e.g., Ant 31,038 vs. 68 for Human; Half-Cheetah 495,374 vs. 12,158 for Eureka) would reflect an objective mismatch rather than superior co-design. The abstract's claim of \"significantly outperforms\" rests entirely on this efficiency comparison, and the raw fitness column shows RoboMoRe is not best on fitness (e.g., Half-Cheetah 135.71 vs. Eureka 257.55). This must be resolved either by specifying and enforcing the same optimization objective for all methods or by demoting efficiency to a secondary metric.","section":"Section 5, Evaluation Metrics; Eq. (2); Table 2"},{"comment":"Table 4 reports a row labeled \"RoboDesign (Full)\" that appears to denote the full RoboMoRe method, but its efficiency values differ systematically from the RoboMoRe row in Table 2 for the same tasks (e.g., Ant 10,464 vs. 31,038; Half-Cheetah 129,441 vs. 495,374; Walker 1,482.60 vs. 6,665.85). The text never defines \"RoboDesign\" or explains why the same configuration yields different numbers in the two tables. This is a load-bearing inconsistency: unless there is an unreported difference in training budget, selection procedure, or evaluation protocol, one of the tables is incorrect, and the reported results cannot be reproduced from the manuscript.","section":"Table 2 and Table 4"},{"comment":"All performance comparisons are reported as point estimates with no error bars, confidence intervals, or statistical tests, despite Appendix D stating that results are averaged over 100 independent runs. The claim that RoboMoRe \"significantly outperforms\" human designs and competing methods requires some measure of variability or a significance test, especially because the differences in the fitness columns are often small or in the baseline's favor. The absence of variance reporting makes it impossible to judge whether the headline efficiency advantages are robust or the result of a single favorable seed.","section":"Tables 2-4; Appendix D"}],"minor_comments":[{"comment":"The table is titled \"Ablation study of RoboDesign,\" but the method introduced in the paper is RoboMoRe; the name \"RoboDesign\" is never defined and should be corrected.","section":"Table 4 title"},{"comment":"The text references \"Fig. ??\" in the discussion of the 5 x 25 efficiency matrices; this is a broken cross-reference and should point to the actual figure.","section":"Appendix E.1"},{"comment":"The description of Bayesian optimization, Eureka, and Eureka(Mor.) gives iteration counts and population sizes but does not specify the objective function optimized by each baseline, which is the key missing information identified in the major comments.","section":"Appendix D, Comparison Methods"},{"comment":"The morphology prompt contains typos such as \"Fintess function\" and \"eend attachment\"; these should be corrected for clarity.","section":"Appendix A.1.2"},{"comment":"The volume computation is described as using \"custom scripts,\" but no details or code are provided in the appendix; since efficiency is the central metric, the volume computation should be specified precisely or released with the code.","section":"Section 5, Evaluation Metrics"}],"recommendation":"reject","confidential_remarks":"The manuscript has genuinely interesting ideas, and the appendix suggests substantial engineering effort, but the current evidence does not support the headline claim. The most serious issue is not the choice of efficiency as a metric per se but the ambiguity about whether the baselines optimized the same objective; this is compounded by the unexplained numerical discrepancy between Tables 2 and 4 and by the complete absence of error bars. Together these are load-bearing problems, and fixing them requires new experiments or major corrections to the reported data, not just local edits. I therefore recommend rejection rather than major revision, although I would view a revised manuscript addressing these points favorably if the results survive re-evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this paper proposes something genuinely useful jointly optimizing reward and morphology in an LLM loop, with a diversity-reflection trick and a coarse-to-fine search, and it demonstrates the idea on eight MuJoCo tasks. That integration is new, and the qualitative examples (long-legged ant hopping, low-profile robot rolling) are compelling. But the quantitative headline the efficiency numbers in Table 2 is not currently supported. The comparison is likely not apples-to-apples: the paper states that RoboMoRe uses efficiency (fitness/volume) as its core optimization metric, but nothing in the descriptions of Bayesian Optimization, Eureka, or Eureka (Mor.) says they optimized the same objective. If those baselines were maximizing raw fitness or task reward, then their efficiency in Table 2 was computed post hoc, and the large gaps (Ant 31,038 vs 68; Half-Cheetah 495,374 vs 12,158) could be mostly an artifact of penalizing designs that never traded off volume. The raw fitness column actually shows RoboMoRe losing to Eureka on Half-Cheetah, so the whole \"significantly outperforms\" claim rests on the efficiency metric.\n\nThere is also an internal inconsistency: Table 4 calls the full method RoboDesign (Full) and reports Ant 10,464.17 and Half-Cheetah 129,440.98, while Table 2 reports RoboMoRe at 31,038.41 and 495,373.71 for the same configuration. No protocol difference is explained. That either is a reporting error or suggests the runs differ in some unreported way, and either way it undermines trust in the numbers. The formal problem definition (Eq. 2) maximizes fitness, but the implementation optimizes efficiency; the authors admit this in a table note, but the formal statement and the experiments are not aligned. Also, there are no error bars or statistical tests anywhere in the main results, even though RL training is stochastic they say they average over runs, but they do not report variance.\n\nOn the positive side, the method is clearly described, the appendix prompts are detailed, and the ablation on diversity reflection is informative. The paper does not ship code or data, which makes the unreported details harder to resolve.\n\nBottom line: the framework is worth a look, but the empirical claim as written is not solid. A serious referee should engage, but the authors need to rerun baselines with the same efficiency objective, reconcile Tables 2 and 4, report variances, and ideally release code. I would not cite the efficiency numbers as they stand.","headline":"A promising integration of LLM reward shaping and morphology co-design, but the headline efficiency comparisons are not apples-to-apples and the results tables contradict each other.","tokens_in":19885,"tokens_out":3046,"would_cite":false,"duration_ms":29246,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RoboMoRe claims that robot co-design fails when rewards are fixed, and that jointly optimizing morphology and reward with an LLM in a coarse-to-fine loop produces dramatically more efficient robots across eight tasks.","keywords":["robot co-design","morphology optimization","reward shaping","large language models","coarse-to-fine optimization","diversity reflection","MuJoCo","efficiency metric"],"falsifier":"Rerun Bayesian Optimization, Eureka, and Eureka (Mor.) with $\\mathrm{efficiency} = \\mathrm{fitness}/\\mathrm{volume}$ as the explicit optimization objective, and report their efficiency after the same training budget; if the gaps shrink to a small factor or reverse, the central claim that joint morphology–reward optimization is what drives the gains would fail. A confirmatory second check: measure the volume distributions of RoboMoRe versus baseline designs to see whether the advantage is mostly smaller bodies rather than better behavior.","tokens_in":18841,"feed_emoji":"🤖","tokens_out":6343,"duration_ms":60441,"temperature":0.7,"pith_summary":"The paper tries to establish that the standard co-design recipe, optimizing a robot's body while keeping a fixed reward, systematically misses the best designs because a body's best motion mode depends on its shape. RoboMoRe instead lets a large language model propose and refine morphology and reward functions together, first casting a wide net of diverse candidates and then alternating between body and reward improvements. On eight MuJoCo locomotion and jumping tasks, the framework reports efficiency (distance or height per unit volume) far above human-engineered designs and three baselines, with no task-specific prompting or reward/morphology templates. The payoff, if the results hold, is a general recipe for letting robots discover their own shapes and their own ways of moving.","feed_headline":"LLM co-design beats human robots on efficiency in 8 tasks","feed_subtitle":"Jointly tuning body and reward unlocks gaits like rolling, hopping, and scuttling.","key_machinery":"The load-bearing mechanism is the coarse-to-fine LLM optimization loop. In the coarse stage, a Diversity Reflection prompt tells the LLM to propose a new design that is both high-fitness and 'quite different' from all previous ones, producing diverse morphology–reward pairs that are then evaluated by reinforcement learning; this replaces the random sampling that evolutionary and Bayesian methods rely on. In the fine stage, the LLM plays the role of a gradient oracle: given top candidates and their scores, it proposes a morphology update, then a reward update, accepting each only if retraining improves efficiency, in an alternating optimization cycle. The objective that shapes all decisions is $\\mathrm{efficiency} = \\mathrm{fitness}/\\mathrm{volume}$, computed for each robot type, which is meant to force material-aware, compact designs rather than brute-force scaling.","core_discovery":"The paper's central claim is that jointly optimizing the reward function with the morphology, rather than treating reward as fixed, is what unlocks high-performance co-design. Concretely, RoboMoRe runs a coarse stage that generates 25 morphologies and 5 rewards through LLM diversity reflection, trains all 125 combinations with SAC, then takes the top 5% into a fine stage where the LLM alternately proposes morphology improvements and reward improvements, keeping only changes that raise efficiency. Reported headline numbers include Ant efficiency jumping from 68.22 (human) to 31,038.41 and Half-Cheetah from 11,975.34 to 495,373.71, with the fitness column often close to or above baselines at far smaller volumes. The paper reads these results as evidence that reward shaping lets each body find its own suited gait, such as rolling, jumping, hopping, or scuttling, and that morphology design remains essential.","pith_inferences":["The quantitative gap in Table 2 is best read as an upper bound until the baselines are re-run with efficiency as their explicit objective, since the paper only states that RoboMoRe optimized efficiency.","The 'LLM momentum' refinement is a heuristic substitute for gradients; replacing it with differentiable morphology proxies or learned surrogates could be a natural test of whether the alternating scheme or the LLM proposals carry the gain.","Efficiency as fitness over volume is a single scalar that ignores energy, control cost, and task-specific constraints; multi-objective variants could change which morphologies are preferred.","Because the prompts are deliberately general, the coarse-to-fine recipe should transfer to other generative design domains such as soft robotics or mechanisms, though that transfer is untested in this paper."],"forward_implications":["If the claim holds, fixed-reward co-design pipelines are leaving a large fraction of performance on the table, and reward shaping should become a standard axis in morphology search.","Diversity reflection should be useful beyond this setting: any LLM design loop that samples candidate structures can improve coverage by explicitly prompting against previous samples.","The fine-stage alternating update should compound with the coarse stage: ablations indicate the full pipeline beats either stage alone, most clearly in high-dimensional tasks like Half-Cheetah.","Efficiency-as-objective should generalize to material-constrained design: robots that move well per unit volume are cheaper to build, suggesting direct relevance to real-world fabrication."],"supporting_citations":[{"why":"Supplies the LLM reward-shaping method that RoboMoRe adapts and serves as a baseline.","marker":"[21]"},{"why":"Pioneers LLM-based morphology design with diversity reflection; RoboMoRe generalizes its mechanism.","marker":"[28]"},{"why":"Provides an LLM-driven morphology optimization baseline that RoboMoRe compares against.","marker":"[23]"},{"why":"Provides the EvoGym co-design benchmark and the evolutionary/Bayesian baselines context.","marker":"[5]"},{"why":"Source of the MuJoCo Gym environments and the human-designed morphologies and rewards used as baselines.","marker":"[30]"},{"why":"Soft Actor-Critic is the RL backbone used to train every morphology–reward pair.","marker":"[14]"},{"why":"Shows how to generate complete MuJoCo XML robot designs from parameter vectors, used by RoboMoRe's design scripts.","marker":"[13]"},{"why":"Provides the convergence foundation for the alternating optimization scheme used in the fine stage.","marker":"[4]"},{"why":"Supplies the Bayesian optimization algorithm that serves as one of the comparison baselines.","marker":"[27]"}],"fun_headline_variants":["LLM co-designs robot body and reward to beat human designs","RoboMoRe: Jointly optimizing morphology and reward wins in 8 tasks","AI robot co-design: tuning reward with shape outperforms fixed reward","LLM-driven co-design finds efficient gaits like rolling and hopping"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes every method was scored under the same objective, $\\mathrm{efficiency} = \\mathrm{fitness}/\\mathrm{volume}$; if the baselines optimized raw fitness rather than efficiency, their high-volume designs would be penalized disproportionately in Table 2, and much of the reported gap could be an artifact of that mismatch.","fun_headline_variants_meta":{"raw":{"variants":["LLM co-designs robot body and reward to beat human designs","RoboMoRe: Jointly optimizing morphology and reward wins in 8 tasks","AI robot co-design: tuning reward with shape outperforms fixed reward","LLM-driven co-design finds efficient gaits like rolling and hopping"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1361,"prompt_tokens":930,"completion_tokens":431,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":353}},"tokens_in":546,"tokens_out":431,"duration_ms":4457,"temperature":1.0,"reasoning_tokens":353,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:08:26.901393+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun Bayesian Optimization, Eureka, and Eureka (Mor.) with $\\mathrm{efficiency} = \\mathrm{fitness}/\\mathrm{volume}$ as the explicit optimization objective, and report their efficiency after the same training budget; if the gaps shrink to a small factor or reverse, the central claim that joint morphology–reward optimization is what drives the gains would fail. A confirmatory second check: measure the volume distributions of RoboMoRe versus baseline designs to see whether the advantage is mostly smaller bodies rather than better behavior.","supporting_citations":[{"cited_title":"Laser: Towards diversified and generalizable robot design with large language models","cited_arxiv_id":null,"evidence_quote":"Pioneers LLM-based morphology design with diversity reflection; RoboMoRe generalizes its mechanism."},{"cited_title":"Evolution gym: A large-scale benchmark for evolving soft robots","cited_arxiv_id":null,"evidence_quote":"Provides the EvoGym co-design benchmark and the evolutionary/Bayesian baselines context."},{"cited_title":"Reinforcement learning for improving agent design","cited_arxiv_id":null,"evidence_quote":"Shows how to generate complete MuJoCo XML robot designs from parameter vectors, used by RoboMoRe's design scripts."},{"cited_title":"Convergence of alternating optimization","cited_arxiv_id":null,"evidence_quote":"Provides the convergence foundation for the alternating optimization scheme used in the fine stage."}],"review_version":1}