{"id":"c3945098-a5a7-4a9d-bf7c-66ab3608f5f2","arxiv_id":"2510.14828","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A 3B VLM trained with SFT plus GRPO and an LCS-based reward reaches 55.3% on EmbodiedBench's EB-ALFRED, beating GPT-4o-mini and the 7B REBP planner.","lead":"This paper trains a 3-billion-parameter vision-language model with supervised fine-tuning and then reinforcement learning to plan multi-step robot actions. On the EmbodiedBench simulator, the small model beats several larger and closed-source baselines, especially on long tasks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline gain is in-domain EB-ALFRED only; EB-Habitat results (22% vs GPT-4o-mini 35%) undercut the abstract's unqualified claim of outperforming GPT-4o-mini on EmbodiedBench.","rationale":"The reader's weakest assumption is precisely the one I find most load-bearing: the headline evaluation is in-domain, with training data distilled from the same benchmark family and the LCS reward using those distilled references as ground truth. The paper's own EB-Habitat results show a large drop (55.33% to 22%), and on this out-of-domain suite RoboGPT-R1 is worse than GPT-4o-mini (22% vs 35%), directly conflicting with the abstract's unqualified statement about outperforming GPT-4o-mini 'on the EmbodiedBench benchmark.' The paper does disclose the in-domain/out-domain distinction in §4.1, which is a point in its favor, but the abstract and introduction do not carry that caveat. The ablations (Tables 2–4) support the method's internal consistency and the value of the LCS reward, so the concern does not invalidate the contribution; it narrows the scope of the claim. The Table 4 labeling inconsistency is real but secondary, as it affects ablation interpretation rather than the central comparison. The concrete test I propose would directly determine whether the RL gain is generalizable planning or in-distribution memorization. Because the reader already assigned CONDITIONAL based on the same concern, my read does not change the verdict.","tokens_in":21898,"tokens_out":6961,"duration_ms":59246,"concrete_test":"Hold out a random subset of EB-ALFRED task types or use the ALFRED unseen split (new room layouts/task instances) that was not used in the distillation, and evaluate the SFT-only and SFT+RFT models there with the same n_shots=0 protocol. Compare their averages to GPT-4o-mini at 10 shots on the same split. If the SFT+RFT model's advantage over SFT-only collapses or the model falls below GPT-4o-mini, the 55.33% headline result is mostly in-distribution memorization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of broad superiority over GPT-4o-mini and larger models is supported only by the EB-ALFRED (seen) column of Table 1. The training data for both SFT and RFT are distilled from EB-ALFRED/ALFRED (Base from REBP/Gemini-2.0-flash, Aug from the ALFRED trajectory dataset), as stated in §3.2 and §4.1. The out-of-domain EB-Habitat results in the same table show RoboGPT-R1 at 22% average, below GPT-4o-mini (35%) and GPT-4o (57%), contradicting the abstract's unqualified 'outperforms ... on the EmbodiedBench benchmark' claim. Moreover, the LCS reward (Eq. 8–9) takes the distilled sequences as ground truth; if Gemini-2.0-flash plans are systematically imperfect, RL will amplify those imperfections. The paper is transparent about the seen/unseen distinction in §4.1, but the headline claim is not. The RL gain (SFT 42% → SFT+RFT 55.33%) could reflect better fitting of the seen task distribution rather than transferable embodied reasoning ability.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes RoboGPT-R1, a two-stage post-training framework for embodied task planning. Starting from Qwen2.5-VL-3B-Instruct, it first applies SFT on a dataset distilled from Gemini-2.0-flash/REBP (the 'Base' set), then applies GRPO reinforcement fine-tuning on an augmented near-domain ALFRED-derived dataset ('Aug'). The reward is a weighted combination of a format reward (section, type, action-validity checks) and an LCS-based accuracy reward between predicted and reference action sequences. Evaluation on EmbodiedBench reports 55.33% average success on EB-ALFRED (treated as in-domain) and 22% on EB-Habitat (treated as out-of-domain), outperforming several 7B-90B and closed-source baselines on EB-ALFRED, with ablations of training stages, data sources, and accuracy rewards.","tokens_in":22185,"tokens_out":11443,"duration_ms":91616,"significance":"If the results are robust, this is a practically valuable empirical result: a 3B VLM with two-stage RL can beat much larger open and closed models on an embodied planning benchmark at lower inference cost, and the LCS reward is a simple rule-based alternative to sparse or prefix rewards. The comparison protocol is conservative in one respect: general baselines are evaluated with 10-shot prompting while the proposed model uses 0-shot, which strengthens the reported gains. The paper is also transparent in marking EB-ALFRED as in-domain and EB-Habitat as out-of-domain in §4.1. However, the generalization claim is limited by the EB-Habitat results, all numbers appear to be single-run point estimates, and several ablations contain confounds or labeling inconsistencies. The significance is therefore mainly as an in-domain fine-tuning recipe until these issues are resolved.","major_comments":[{"comment":"The abstract and introduction claim that RoboGPT-R1 'significantly outperforms GPT-4o-mini by 21.33% ... on the EmbodiedBench benchmark.' This number is the EB-ALFRED (seen) column only: 55.33 vs 34.00. On the EB-Habitat (unseen) column of the same table, the model scores 22.00, below GPT-4o-mini (35.00), GPT-4o (57.00), Qwen2.5-VL-72B (50.33), and several others. The main text is transparent about the in-domain/out-of-domain split (§4.1), but the abstract and §1 are not. Please qualify all headline comparisons as 'EB-ALFRED (seen)' and either report a combined metric or explicitly discuss the EB-Habitat gap.","section":"Abstract and §1, Table 1"},{"comment":"Every success rate appears to be a single point estimate from one run. Several load-bearing comparisons are small: SFT+RFT w Base (44.33) vs SFT-only (42.00) in Table 3; Step Accuracy (43.67) vs RFT Base (42.00) and REBP Acc (48.33) in Table 4; and GPT-4o-mini vs RoboGPT-R1 on several sub-suites in Table 1. Without the number of evaluation tasks per sub-suite, confidence intervals, or multiple training/evaluation seeds, the 2–13 point deltas cannot be distinguished from noise. Please report the evaluation set sizes and repeated-run statistics so the reader can assess whether the reported gains are statistically meaningful.","section":"Tables 1–4, §4.3–4.4"},{"comment":"The reference row labeled 'RFT Base' in Table 4 has exactly the values of 'Only SFT w Base' in Table 3 (42.00, 48, 44, 58, 38, 38, 26), not 'SFT+RFT w Base' (44.33, 56, 56, 54, 32, 36, 32). As written, the deltas for the three accuracy rewards are measured against a model that did not undergo RFT, despite the name and the claim that the comparison is made under a fixed training budget. Please correct the label and make the reference condition explicit; otherwise the head-to-head comparison of accuracy rewards is not properly defined.","section":"Table 4, §4.4"},{"comment":"The data-source ablation conflates dataset domain with dataset size. 'SFT+RFT w Base' uses the ~5k-sample Base set (Appendix B.2), while 'SFT+RFT w Aug' uses the ~45k Aug set; the 44.33→55.33 improvement could be due to the larger RL data pool rather than the near-domain character of Aug. In addition, 'Only SFT w Aug' does not include the Base samples that are deliberately added to Aug for the RFT condition, so the SFT-only vs RFT comparison is not symmetric. Please add an RFT condition with a size-matched sample of Aug, or otherwise isolate the domain factor from dataset size.","section":"Table 3, §B.2"},{"comment":"The LCS reward as defined is not robust to action-sequence padding. Since R_lcs = k/n with n fixed to the reference length, any predicted sequence that contains the reference as a subsequence—regardless of how many extra valid actions are inserted—receives the same accuracy reward as an exact match. The format reward checks fields, types, and dictionary validity but imposes no length or step-count constraint, so nothing in Eq. (10) enforces 'reasonable length' as claimed in §3.4.3. This is precisely the over-generation failure the introduction attributes to string-matching rewards. Please add a length penalty or report output-length statistics showing that the trained policy does not exploit this flatness.","section":"§3.4.2–3.4.3, Eqs. (8)–(10)"}],"minor_comments":[{"comment":"'1 is an exponential function' should read 'indicator function'; the notation is nonstandard.","section":"§3.4.1, Eq. (5)"},{"comment":"'REBP[50]' is the wrong reference: [50] is GRPO-MA, not REBP. The sentence following it is also duplicated ('Following REBP [54], we employ...').","section":"§3.2"},{"comment":"The phrase 'approximately 54% relative improvement' is arithmetically wrong: 1.33→55.33 is +54 percentage points (≈4060% relative), while 42→55.33 is ≈31.7% relative. Please correct or rephrase as percentage points.","section":"§4.2"},{"comment":"The abstract uses 'long-view manipulation tasks' while the rest of the paper says 'long-horizon'; unify the terminology.","section":"Abstract"},{"comment":"The caption could clarify that the RoboGPT-R1 numbers on the EB-ALFRED side are in-domain and the EB-Habitat side is out-of-domain; currently this is stated only in §4.1, which invites misreading of the headline comparisons.","section":"Table 1"},{"comment":"The hyperparameter table has a formatting typo ('template qwen2_vl') and the appendix title 'EXPERIENTAL DETAILS' is misspelled.","section":"Appendix A.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is best read as an in-domain EB-ALFRED fine-tuning recipe, not as a general EmbodiedBench result; the abstract overclaims relative to the paper's own EB-Habitat numbers. The single-run statistics and the Table 4 labeling issue are the main robustness concerns. I would not reject—the ablations and conservative 10-shot protocol are valuable—but the manuscript needs a careful revision before a journal-level accept."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent, incremental paper that deserves a serious referee, but the headline should be read narrowly. The genuinely new pieces are normalized LCS as the GRPO accuracy reward, dynamic action IDs, and zero-shot data cleaning. The ablations support the claims: RL on near-domain data (Aug) gives 55.33% vs 44.33% when RFT stays on the in-domain Base, and LCS beats both step accuracy and REBP's reward, especially on long-horizon tasks (26 to 50 vs 26 to 34). The comparison protocol is conservative (0-shot for the trained model, 10-shot for GPT/Qwen baselines), which is the right call.\n\nThe soft spots are mostly framing and evidence quality. The abstract says the 3B model outperforms GPT-4o-mini on EmbodiedBench, but on EB-Habitat the same table shows 22% vs GPT-4o-mini's 35%. The paper is transparent about the seen/unseen split in Section 4.1, so this is an abstract problem, not a hidden one. The bigger caveat is that SFT and RFT data are distilled from the same EB-ALFRED/ALFRED family, and the LCS reward treats those Gemini-distilled plans as ground truth; the in-domain gain may partly be fitting that distribution, and the EB-Habitat numbers suggest limited transfer. Also, all results are single runs without error bars or significance tests, which matters for a 13-point improvement claim. Table 4's row labeled 'RFT Base' repeats the SFT-only 42.00 from Table 2; that needs cleanup. Code isn't released yet, but the description is clear enough for replication.\n\nThis paper is for people working on RL for embodied planning or VLM fine-tuning. Not paradigm-shifting, but the LCS reward and the data-source ablation are useful. I'd send it to peer review; with honest framing and error bars it could be a solid workshop or conference paper. I'd cite the LCS reward result, not the unqualified benchmark claim.","headline":"Solid incremental result, but the abstract oversells it: RL gains are real in-domain, the LCS reward is a genuine improvement, yet out-of-domain numbers undercut the unqualified superiority claim.","tokens_in":22720,"tokens_out":3053,"would_cite":true,"duration_ms":23583,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 3-billion-parameter vision-language model, trained with supervised fine-tuning followed by reinforcement learning on an order-sensitive reward, outperforms much larger general models and a larger embodied-planning specialist on household","keywords":["robot task planning","reinforcement learning","vision-language model","GRPO","longest common subsequence","long-horizon planning","reward design","embodied AI"],"falsifier":"Take the trained 3B model and evaluate it on the same benchmark but with reference plans generated by a different, independently verified method (e.g., a formal planner or human-annotated plans), and run a shuffled-corruption test where action names are permuted; if success collapses to near the SFT-only level, the LCS reward was exploiting reference noise or vocabulary shortcuts rather than learning generalizable ordering.","tokens_in":21742,"feed_emoji":"🤖","tokens_out":4611,"duration_ms":37227,"temperature":0.7,"pith_summary":"This paper argues that the standard recipe of supervised fine-tuning on expert demonstrations is not enough to make a vision-language model a reliable long-horizon robot planner. The authors propose a second reinforcement-learning stage, using GRPO with a rule-based reward that combines formatting checks with a longest-common-subsequence (LCS) score between predicted and reference action sequences. On a household instruction-following benchmark, the resulting 3-billion-parameter model reaches 55% average success, outperforming a much larger general-purpose proprietary model and a 7-billion-parameter planning specialist, and its long-horizon score more than doubles from 26% to 50%. This matters because it suggests that small models can close much of the planning gap through RL with dense, order-sensitive rewards rather than through sheer scale.","feed_headline":"3B planner beats larger models with sequence-aware RL","feed_subtitle":"Reinforcement learning with an order-sensitive reward lifts long-horizon success from 26% to 50%.","key_machinery":"GRPO with a two-part rule-based reward. The format reward (20% weight) checks that the output contains the required fields (visual state description, reasoning, language plan, executable plan), that action steps are well-formed, and that action id–name pairs are valid against a per-task action dictionary with dynamic ids. The accuracy reward (80% weight) is the normalized longest common subsequence (LCS) between the predicted action-name sequence and the reference sequence; LCS preserves order while tolerating local deviations, so a plan that errs early and recovers still earns partial credit. This combined reward is the mechanism claimed to produce the long-horizon gains.","core_discovery":"The central claim is that a two-stage training scheme—SFT to instill basic planning, then GRPO reinforcement fine-tuning—can transform a small vision-language model into a competitive embodied planner, provided the reward is sequence-aware. The key evidence is the jump from 42% to 55% average success on the in-domain suite and from 26% to 50% on long-horizon tasks after RL, with the LCS reward shown by ablation to be the driver: step-wise accuracy barely moves the average and even hurts long-horizon, while the LCS reward yields the largest gains. The authors interpret this as RL teaching the model to recover from early mistakes and maintain action order, something imitation alone does not pr","pith_inferences":["If the reference plans are noisy, LCS training will amplify those noise patterns; a direct test is to retrain on reference plans from a different, independently verified source and see whether gains persist.","The reward gives partial credit for subsequences, which may incentivize longer outputs; an extension would measure whether sequence length correlates with reward in ways that could be gamed.","A harder generalization probe—e.g., new action vocabularies, unseen scene layouts, or real-robot execution—would clarify whether the learned ordering skill transfers or is tied to the benchmark's action set.","The same two-stage recipe with an LCS-style reward could apply to other sequential decision tasks, such as web navigation or tool use, where ordering and recovery matter."],"forward_implications":["A small (3B) planner can outperform larger general models and larger specialists on in-domain embodied planning, lowering inference cost.","LCS-based rewards provide a dense, order-sensitive training signal that step-wise or prefix rewards lack, especially for long tasks with error recovery.","Reinforcement fine-tuning can absorb near-domain data that supervised fine-tuning cannot effectively exploit.","The long-horizon success jump from 26% to 50% is attributable mainly to the RL stage, not to better imitation.","Out-of-domain performance (22%) remains modest, so the gains are partly benchmark-specific and do not yet imply broad physical understanding."],"fun_headline_variants":["RL with order-aware reward lifts robot planning from 26% to 50%","Sequence reward, not step accuracy, boosts long-horizon robot planning","Tiny 3B planner beats 7B and GPT-4o-mini after RL fine-tuning","Reinforcement learning doubles long-horizon success for robot planner","3B model outperforms larger rivals with sequence-aware RL planning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The reward treats the distilled reference action sequences as ground truth; if those references are noisy, biased, or tied to a specific action vocabulary, the RL stage will optimize toward those biases rather than toward robust physical understanding.","fun_headline_variants_meta":{"raw":{"variants":["RL with order-aware reward lifts robot planning from 26% to 50%","Sequence reward, not step accuracy, boosts long-horizon robot planning","Tiny 3B planner beats 7B and GPT-4o-mini after RL fine-tuning","Reinforcement learning doubles long-horizon success for robot planner","3B model outperforms larger rivals with sequence-aware RL planning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000822,"raw_usage":{"total_tokens":3439,"prompt_tokens":759,"completion_tokens":2680,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":2578}},"tokens_in":503,"tokens_out":2680,"duration_ms":16201,"temperature":1.0,"reasoning_tokens":2578,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T09:29:15.016832+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained 3B model and evaluate it on the same benchmark but with reference plans generated by a different, independently verified method (e.g., a formal planner or human-annotated plans), and run a shuffled-corruption test where action names are permuted; if success collapses to near the SFT-only level, the LCS reward was exploiting reference noise or vocabulary shortcuts rather than learning generalizable ordering.","supporting_citations":[],"review_version":1}