Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

A 3-billion-parameter vision-language model, trained with supervised fine-tuning followed by reinforcement learning on an order-sensitive reward, outperforms much larger general models and a larger embodied-planning specialist on household

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A 3B VLM trained with SFT plus GRPO and an LCS-based reward reaches 55.3% on EmbodiedBench's EB-ALFRED, beating GPT-4o-mini and the 7B REBP planner.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Solid incremental result, but the abstract oversells it: RL gains are real in-domain, the LCS reward is a genuine improvement, yet out-of-domain numbers undercut the unqualified superiority claim. the 5 major comments →

arxiv 2510.14828 v3 pith:UGQ7E5KG submitted 2025-10-16 cs.AI cs.RO

RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning

Jinrui Liu , Bingyan Nie , Boyu Li , Yaran Chen , Yuze Wang , Shunsen He , Haoran Li This is my paper
classification cs.AI cs.RO
keywords robot task planningreinforcement learningvision-language modelGRPOlongest common subsequencelong-horizon planningreward designembodied AI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard recipe of supervised fine-tuning on expert demonstrations is not enough to make a vision-language model a reliable long-horizon robot planner. The authors propose a second reinforcement-learning stage, using GRPO with a rule-based reward that combines formatting checks with a longest-common-subsequence (LCS) score between predicted and reference action sequences. On a household instruction-following benchmark, the resulting 3-billion-parameter model reaches 55% average success, outperforming a much larger general-purpose proprietary model and a 7-billion-parameter planning specialist, and its long-horizon score more than doubles from 26% to 50%. This matters because it suggests that small models can close much of the planning gap through RL with dense, order-sensitive rewards rather than through sheer scale.

Core claim

The central claim is that a two-stage training scheme—SFT to instill basic planning, then GRPO reinforcement fine-tuning—can transform a small vision-language model into a competitive embodied planner, provided the reward is sequence-aware. The key evidence is the jump from 42% to 55% average success on the in-domain suite and from 26% to 50% on long-horizon tasks after RL, with the LCS reward shown by ablation to be the driver: step-wise accuracy barely moves the average and even hurts long-horizon, while the LCS reward yields the largest gains. The authors interpret this as RL teaching the model to recover from early mistakes and maintain action order, something imitation alone does not pr

What carries the argument

GRPO with a two-part rule-based reward. The format reward (20% weight) checks that the output contains the required fields (visual state description, reasoning, language plan, executable plan), that action steps are well-formed, and that action id–name pairs are valid against a per-task action dictionary with dynamic ids. The accuracy reward (80% weight) is the normalized longest common subsequence (LCS) between the predicted action-name sequence and the reference sequence; LCS preserves order while tolerating local deviations, so a plan that errs early and recovers still earns partial credit. This combined reward is the mechanism claimed to produce the long-horizon gains.

Load-bearing premise

The reward treats the distilled reference action sequences as ground truth; if those references are noisy, biased, or tied to a specific action vocabulary, the RL stage will optimize toward those biases rather than toward robust physical understanding.

What would settle it

Take the trained 3B model and evaluate it on the same benchmark but with reference plans generated by a different, independently verified method (e.g., a formal planner or human-annotated plans), and run a shuffled-corruption test where action names are permuted; if success collapses to near the SFT-only level, the LCS reward was exploiting reference noise or vocabulary shortcuts rather than learning generalizable ordering.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A small (3B) planner can outperform larger general models and larger specialists on in-domain embodied planning, lowering inference cost.
  • LCS-based rewards provide a dense, order-sensitive training signal that step-wise or prefix rewards lack, especially for long tasks with error recovery.
  • Reinforcement fine-tuning can absorb near-domain data that supervised fine-tuning cannot effectively exploit.
  • The long-horizon success jump from 26% to 50% is attributable mainly to the RL stage, not to better imitation.
  • Out-of-domain performance (22%) remains modest, so the gains are partly benchmark-specific and do not yet imply broad physical understanding.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the reference plans are noisy, LCS training will amplify those noise patterns; a direct test is to retrain on reference plans from a different, independently verified source and see whether gains persist.
  • The reward gives partial credit for subsequences, which may incentivize longer outputs; an extension would measure whether sequence length correlates with reward in ways that could be gamed.
  • A harder generalization probe—e.g., new action vocabularies, unseen scene layouts, or real-robot execution—would clarify whether the learned ordering skill transfers or is tied to the benchmark's action set.
  • The same two-stage recipe with an LCS-style reward could apply to other sequential decision tasks, such as web navigation or tool use, where ordering and recovery matter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes RoboGPT-R1, a two-stage post-training framework for embodied task planning. Starting from Qwen2.5-VL-3B-Instruct, it first applies SFT on a dataset distilled from Gemini-2.0-flash/REBP (the 'Base' set), then applies GRPO reinforcement fine-tuning on an augmented near-domain ALFRED-derived dataset ('Aug'). The reward is a weighted combination of a format reward (section, type, action-validity checks) and an LCS-based accuracy reward between predicted and reference action sequences. Evaluation on EmbodiedBench reports 55.33% average success on EB-ALFRED (treated as in-domain) and 22% on EB-Habitat (treated as out-of-domain), outperforming several 7B-90B and closed-source baselines on EB-ALFRED, with ablations of training stages, data sources, and accuracy rewards.

Significance. If the results are robust, this is a practically valuable empirical result: a 3B VLM with two-stage RL can beat much larger open and closed models on an embodied planning benchmark at lower inference cost, and the LCS reward is a simple rule-based alternative to sparse or prefix rewards. The comparison protocol is conservative in one respect: general baselines are evaluated with 10-shot prompting while the proposed model uses 0-shot, which strengthens the reported gains. The paper is also transparent in marking EB-ALFRED as in-domain and EB-Habitat as out-of-domain in §4.1. However, the generalization claim is limited by the EB-Habitat results, all numbers appear to be single-run point estimates, and several ablations contain confounds or labeling inconsistencies. The significance is therefore mainly as an in-domain fine-tuning recipe until these issues are resolved.

major comments (5)
  1. [Abstract and §1, Table 1] The abstract and introduction claim that RoboGPT-R1 'significantly outperforms GPT-4o-mini by 21.33% ... on the EmbodiedBench benchmark.' This number is the EB-ALFRED (seen) column only: 55.33 vs 34.00. On the EB-Habitat (unseen) column of the same table, the model scores 22.00, below GPT-4o-mini (35.00), GPT-4o (57.00), Qwen2.5-VL-72B (50.33), and several others. The main text is transparent about the in-domain/out-of-domain split (§4.1), but the abstract and §1 are not. Please qualify all headline comparisons as 'EB-ALFRED (seen)' and either report a combined metric or explicitly discuss the EB-Habitat gap.
  2. [Tables 1–4, §4.3–4.4] Every success rate appears to be a single point estimate from one run. Several load-bearing comparisons are small: SFT+RFT w Base (44.33) vs SFT-only (42.00) in Table 3; Step Accuracy (43.67) vs RFT Base (42.00) and REBP Acc (48.33) in Table 4; and GPT-4o-mini vs RoboGPT-R1 on several sub-suites in Table 1. Without the number of evaluation tasks per sub-suite, confidence intervals, or multiple training/evaluation seeds, the 2–13 point deltas cannot be distinguished from noise. Please report the evaluation set sizes and repeated-run statistics so the reader can assess whether the reported gains are statistically meaningful.
  3. [Table 4, §4.4] The reference row labeled 'RFT Base' in Table 4 has exactly the values of 'Only SFT w Base' in Table 3 (42.00, 48, 44, 58, 38, 38, 26), not 'SFT+RFT w Base' (44.33, 56, 56, 54, 32, 36, 32). As written, the deltas for the three accuracy rewards are measured against a model that did not undergo RFT, despite the name and the claim that the comparison is made under a fixed training budget. Please correct the label and make the reference condition explicit; otherwise the head-to-head comparison of accuracy rewards is not properly defined.
  4. [Table 3, §B.2] The data-source ablation conflates dataset domain with dataset size. 'SFT+RFT w Base' uses the ~5k-sample Base set (Appendix B.2), while 'SFT+RFT w Aug' uses the ~45k Aug set; the 44.33→55.33 improvement could be due to the larger RL data pool rather than the near-domain character of Aug. In addition, 'Only SFT w Aug' does not include the Base samples that are deliberately added to Aug for the RFT condition, so the SFT-only vs RFT comparison is not symmetric. Please add an RFT condition with a size-matched sample of Aug, or otherwise isolate the domain factor from dataset size.
  5. [§3.4.2–3.4.3, Eqs. (8)–(10)] The LCS reward as defined is not robust to action-sequence padding. Since R_lcs = k/n with n fixed to the reference length, any predicted sequence that contains the reference as a subsequence—regardless of how many extra valid actions are inserted—receives the same accuracy reward as an exact match. The format reward checks fields, types, and dictionary validity but imposes no length or step-count constraint, so nothing in Eq. (10) enforces 'reasonable length' as claimed in §3.4.3. This is precisely the over-generation failure the introduction attributes to string-matching rewards. Please add a length penalty or report output-length statistics showing that the trained policy does not exploit this flatness.
minor comments (6)
  1. [§3.4.1, Eq. (5)] '1 is an exponential function' should read 'indicator function'; the notation is nonstandard.
  2. [§3.2] 'REBP[50]' is the wrong reference: [50] is GRPO-MA, not REBP. The sentence following it is also duplicated ('Following REBP [54], we employ...').
  3. [§4.2] The phrase 'approximately 54% relative improvement' is arithmetically wrong: 1.33→55.33 is +54 percentage points (≈4060% relative), while 42→55.33 is ≈31.7% relative. Please correct or rephrase as percentage points.
  4. [Abstract] The abstract uses 'long-view manipulation tasks' while the rest of the paper says 'long-horizon'; unify the terminology.
  5. [Table 1] The caption could clarify that the RoboGPT-R1 numbers on the EB-ALFRED side are in-domain and the EB-Habitat side is out-of-domain; currently this is stated only in §4.1, which invites misreading of the headline comparisons.
  6. [Appendix A.1] The hyperparameter table has a formatting typo ('template qwen2_vl') and the appendix title 'EXPERIENTAL DETAILS' is misspelled.

Circularity Check

0 steps flagged

No significant circularity: the central claim is an empirical benchmark result, not a fitted derivation. The in-domain training/evaluation overlap is transparent in the paper and is an external-validity concern rather than a definitional or self-citation circularity.

full rationale

The paper does not claim to derive a first-principles result from a fitted parameter. Its pipeline is SFT on REBP/Gemini-2.0-flash-distilled EB-ALFRED/ALFRED data followed by GRPO with a hand-designed rule-based reward. The abstract emphasizes the EB-ALFRED (seen) numbers, but Section 4.1 explicitly distinguishes EB-ALFRED as in-domain and EB-Habitat as out-of-domain, so the strongest result is scoped honestly in the body. The LCS reward (Eqs. 8-9) is a training signal, not a fitted predictor of the evaluation metric; it is not derived from the benchmark's success oracle, and the evaluation is performed by the environment rather than by the reward function. No load-bearing self-citation chain is present: REBP [54] is external prior work, and the authors' own RoboGPT [8] is cited only as related work, not as justification for the central claim. There is no uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result masquerading as derivation. The benchmark-overlap concern that training data are distilled from the same EB-ALFRED/ALFRED task family used for the seen evaluation is a real limitation on generalization claims, but it is a data-contamination and external-validity issue, not a logical circularity. Accordingly, the paper is best assessed as an empirical in-domain result with honest caveats about out-of-domain transfer, not as a derivation that reduces to its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central claim rests on standard RL machinery plus a few hand-chosen reward weights and dataset assumptions. No new scientific entities or forces are introduced. The most consequential assumption is that Gemini-distilled reference trajectories are reliable ground truth; the most consequential free parameters are the reward weights (0.8/0.2 and 0.3/0.3/0.4), which are not ablated.

free parameters (4)
  • overall reward weight lambda_lcs = 0.8 (format weight 0.2)
    Eq. 10 sets R = 0.2*R_format + 0.8*R_lcs; no sensitivity analysis is reported for these weights.
  • format reward sub-weights = 0.3 / 0.3 / 0.4
    Eq. 4 assigns weights to section, type, and validity rewards; these are hand-chosen and not ablated.
  • RFT training steps = 80
    Appendix A.2 states the RFT iteration count was uniformly set to 80 for fairness; no study of convergence horizon is provided.
  • SFT epochs = 2
    Appendix A.1 sets num_train_epochs to 2 as an empirical choice to avoid overfitting.
axioms (5)
  • standard math GRPO advantage and update equations (Eqs. 1–3) are correct and applicable to VLM policies.
    The paper builds on DeepSeek-R1's GRPO formulation without proving or questioning it.
  • standard math LCS dynamic programming recurrence (Eq. 8) correctly measures sequence similarity for planning.
    The recurrence is standard; the text even mislabels the indicator function as 'an exponential function'.
  • domain assumption Reference action sequences from Gemini-2.0-flash-distilled REBP data are correct enough to serve as RL ground truth.
    §3.2 and Eq. 8–9 use these references for the LCS reward; if they contain errors, RL actively reinforces them.
  • domain assumption EB-ALFRED task success is a valid proxy for embodied planning ability.
    The paper uses EmbodiedBench success rates as its central metric without justifying that simulator success transfers to real robots.
  • ad hoc to paper Near-domain ALFRED trajectories transfer to EB-ALFRED via RL but not via SFT.
    This is the empirical conclusion of Table 3, used to justify the Aug dataset in RFT; the mechanism is asserted rather than derived.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning." pith.science (2026). https://pith.science/paper/UGQ7E5KG

@misc{pith2026251014828,
  author       = {Pith},
  title        = {Pith review of: RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UGQ7E5KG}},
  note         = {Machine review of arXiv:2510.14828}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Improving the reasoning capabilities of embodied agents is crucial for robots to complete complex human instructions in long-view manipulation tasks successfully. Despite the success of large language models and vision language models based on Supervised Fine-Tuning (SFT) in planning tasks, they continue facing challenges in performing long-horizon manipulation tasks in complex real-world environments, owing to their restricted common sense and reasoning capabilities. Considering that aligning general-purpose vision language models to robotic planning tasks via supervised fine-tuning suffers from poor generalization and insufficient physical understanding, we propose RoboGPT-R1, a two-stage fine-tuning framework for embodied planning. In this framework, supervised training acquires foundational knowledge through expert sequences, followed by RL to address the model's shortcomings in visual-spatial understanding and reasoning. To achieve physical understanding and action sequence consistency in multi-step reasoning tasks, we design a rule-based reward function that simultaneously considers long-horizon performance and action constraint in the environment. The reasoning model, trained on Qwen2.5-VL-3B, significantly outperforms the larger-scale model, GPT-4o-mini, by 21.33% and surpasses other work trained on Qwen2.5-VL-7B by 20.33% on the EmbodiedBench benchmark.

Figures

Figures reproduced from arXiv: 2510.14828 by Bingyan Nie, Boyu Li, Haoran Li, Jinrui Liu, Shunsen He, Yaran Chen, Yuze Wang.

Figure 1
Figure 1. Figure 1: An overview of RoboGPT-R1. RoboGPT-R1 adopts a two-stage learning paradigm. In the initial phase, supervised fine [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Reward Curves in RFT. Our LCS-based accuracy [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Success rates with different stages. Bars show the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data

    cs.RO 2026-05 unverdicted novelty 6.0

    A co-evolutionary VLM-VGM loop on 500 unlabeled images raises planner success by 30 points and simulator success by 48 percent while beating fully supervised baselines.

  2. RoboAgent: Chaining Basic Capabilities for Embodied Task Planning

    cs.RO 2026-04 unverdicted novelty 5.0

    RoboAgent chains basic vision-language capabilities inside a single VLM via a scheduler and trains it in three stages (behavior cloning, DAgger, RL) to improve embodied task planning.

Reference graph

Works this paper leans on

68 extracted references · 50 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Alibaba DAMO Academy. 2024. EasyR1: A unified framework for reward model- ing and RLHF. https://github.com/alibaba/EasyR1

    2024

  2. [2]

    Alibaba DAMO Academy. 2024. GRPO: Generalized Reward Preference Opti- mization for LLM alignment. https://github.com/alibaba/GRPO

    2024

  3. [3]

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, K...

    Pith/arXiv arXiv 2022

  4. [4]

    Meta AI. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, cus- tomizable models. https://ai.meta.com/blog/llama-3-2-connect-2024-vision- edge-mobile-devices/

    2024

  5. [5]

    Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and Aviral Kumar. 2024. DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning. arXiv:2406.11896 [cs.LG] https://arxiv. org/abs/2406.11896

    Pith/arXiv arXiv 2024

  6. [6]

    Zitong Bo, Yue Hu, Jinming Ma, Mingliang Zhou, Junhui Yin, Yachen Kang, Yuqi Liu, Tong Wu, Diyun Xiang, and Hao Chen. 2025. Reinforced Embod- ied Planning with Verifiable Reward for Real-World Robotic Manipulation. arXiv:2509.25852 [cs.RO] https://arxiv.org/abs/2509.25852

    arXiv 2025

  7. [7]

    Shaoyu Chen, Bo Jiang, Hao Gao, Bencheng Liao, Qing Xu, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. 2024. VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning. arXiv:2402.13243 [cs.CV] https://arxiv.org/abs/2402.13243

    Pith/arXiv arXiv 2024

  8. [8]

    Yaran Chen, Wenbo Cui, Yuanwen Chen, Mining Tan, Xinyao Zhang, Jinrui Liu, Haoran Li, Dongbin Zhao, and He Wang. 2025. RoboGPT: an LLM-based Long-term Decision-making Embodied Agent for Instruction Following Tasks. IEEE Transactions on Cognitive and Developmental Systems(2025), 1–11. https: //doi.org/10.1109/TCDS.2025.3543364

    arXiv 2025

  9. [9]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yiming Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Huipeng Deng, Ji...

    Pith/arXiv arXiv 2025

  10. [10]

    Le, Sergey Levine, and Yi Ma

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. 2025. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training.CoRR abs/2501.17161 (2025)

    Pith/arXiv arXiv 2025

  11. [11]

    Alibaba Cloud. 2024. Qwen-VL-Max: Large Vision-Language Model. https: //github.com/QwenLM/Qwen-VL

    2024

  12. [12]

    Alibaba Cloud. 2025. Qwen2.5-VL Technical Report. arXiv:2502.13923 [cs.CL] https://arxiv.org/abs/2502.13923

    Pith/arXiv arXiv 2025

  13. [13]

    Google DeepMind. 2024. Introducing Gemini 2.0: Our new AI model for the agentic era. https://blog.google/technology/google-deepmind/google-gemini-ai- update-december-2024/

    2024

  14. [14]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    Pith/arXiv arXiv 2025

  15. [15]

    Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. 2022. A Sur- vey of Embodied AI: From Simulators to Research Tasks. arXiv:2103.04918 [cs.AI] https://arxiv.org/abs/2103.04918

    Pith/arXiv arXiv 2022

  16. [16]

    Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong, Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Li Fei-Fei, and Jianfeng Gao. 2024. Agent AI: Surveying the Horizons of Multimodal Interaction. arXiv:2401.03568 [cs.AI] https://arxiv.org/ abs/2401.03568

    Pith/arXiv arXiv 2024

  17. [17]

    Zhaoye Fei, Li Ji, Siyin Wang, Junhao Shi, Jingjing Gong, and Xipeng Qiu. 2025. Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning. arXiv:2506.23127 [cs.CL] https://arxiv.org/abs/2506.23127

    Pith/arXiv arXiv 2025

  18. [18]

    Sicheng Feng, Kaiwen Tuo, Song Wang, Lingdong Kong, Jianke Zhu, and Huan Wang. 2025. RewardMap: Tackling Sparse Rewards in Fine-grained Visual Rea- soning via Multi-Stage Reinforcement Learning. arXiv:2510.02240 [cs.CV] https://arxiv.org/abs/2510.02240

    arXiv 2025

  19. [19]

    Yunhai Feng, Jiaming Han, Zhuoran Yang, Xiangyu Yue, Sergey Levine, and Jianlan Luo. 2025. Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulation. arXiv:2502.16707 [cs.RO] https://arxiv. org/abs/2502.16707

    Pith/arXiv arXiv 2025

  20. [20]

    Yuqian Fu, Tinghong Chen, Jiajun Chai, Xihuai Wang, Songjun Tu, Guojun Yin, Wei Lin, Qichao Zhang, Yuanheng Zhu, and Dongbin Zhao. 2025. SRFT: A Single- Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning. arXiv:2506.19767 [cs.CL] https://arxiv.org/abs/2506.19767

    Pith/arXiv arXiv 2025

  21. [21]

    Hiyouga. 2023. LLaMA Factory: Open-source instruction tuning framework for LLMs. https://github.com/hiyouga/LLaMA-Factory

    2023

  22. [22]

    Wenlong Huang, Fei Xia, Dhruv Shah, Danny Driess, Andy Zeng, Yao Lu, Pete Florence, Igor Mordatch, Sergey Levine, Karol Hausman, and Brian Ichter. 2023. Grounded decoding: guiding text generation with grounded models for embodied agents. InProceedings of the 37th International Conference on Neural Information Processing Systems(New Orleans, LA, USA)(NIPS ...

    2023

  23. [23]

    Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, Xinda Xue, Qinghang Su, Huaihai Lyu, Xiaolong Zheng, Jiaming Liu, Zhongyuan Wang, and Shanghang Zhang. 2025. RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. arXiv:2502.21257 [cs.RO] https://arxiv.o...

    Pith/arXiv arXiv 2025

  24. [24]

    Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, and Xinggang Wang. 2025. AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Rein- forcement Learning and Reasoning.CoRRabs/2503.07608 (2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, Aniruddha Kem- bhavi, Abhinav Gupta, and Ali Farhadi. 2022. AI2-THOR: An Interactive 3D Envi- ronment for Visual AI. arXiv:1712.05474 [cs.CV] https://arxiv.org/abs/1712.05474

    Pith/arXiv arXiv 2022

  26. [26]

    Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. 2025. VideoChat-R1: Enhancing Spatio- Temporal Perception via Reinforcement Fine-Tuning. arXiv:2504.06958 [cs.CV] https://arxiv.org/abs/2504.06958

    Pith/arXiv arXiv 2025

  27. [27]

    Wenlong Liang, Rui Zhou, Yang Ma, Bing Zhang, Songlin Li, Yijia Liao, and Ping Kuang. 2025. Large Model Empowered Embodied AI: A Survey on Decision- Making and Embodied Learning.CoRRabs/2508.10399 (2025)

    Pith/arXiv arXiv 2025

  28. [28]

    Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, and Liang Lin. 2025. Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI. arXiv:2407.06886 [cs.CV] https://arxiv.org/abs/2407. 06886

    Pith/arXiv arXiv 2025

  29. [29]

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. 2025. Visual-RFT: Visual Reinforcement Fine-Tuning. CoRRabs/2503.01785 (2025)

    Pith/arXiv arXiv 2025

  30. [30]

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, Ping Luo, Yu Qiao, Qiaosheng Zhang, and Wenqi Shao. 2025. MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning. arXiv:2503.07365 [cs.CV] https://arxiv.org/abs/2503.07365

    Pith/arXiv arXiv 2025

  31. [31]

    Youssef Mroueh. 2025. Reinforcement Learning with Verifiable Rewards: GRPO’s Effective Loss, Dynamics, and Success Amplification. arXiv:2503.06639 [cs.LG] https://arxiv.org/abs/2503.06639

    arXiv 2025

  32. [32]

    OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexan- der Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, An- dre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Ko...

    Pith/arXiv arXiv 2024

  33. [33]

    OpenAI. 2024. GPT-4o mini: Advancing cost-efficient intelligence. https://openai. com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/

    2024

  34. [34]

    OpenAI. 2024. Hello GPT-4o. https://openai.com/index/hello-gpt-4o/

    2024

  35. [35]

    OpenAI. 2025. Introducing GPT-4.1 in the API. https://openai.com/index/gpt-4- 1/

    2025

  36. [36]

    Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wij- mans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. 2019. Habitat: A Platform for Embodied AI Research. arXiv:1904.01201 [cs.CV] https://arxiv.org/abs/1904.01201

    Pith/arXiv arXiv 2019

  37. [37]

    Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil J Joshi, Pete Florence, Wei Han, Robert Baruch, Yao Lu, Suvir Mirchandani, Peng Xu, Pannag Sanketi, Karol Hausman, Izhak Shafran, Brian Ichter, and Yuan Cao. 2023. RoboVQA: Multimodal Long-Horizo...

    Pith/arXiv arXiv 2023

  38. [38]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeek- Math: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 [cs.CL] https://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv 2024

  39. [39]

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. 2025. VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model.CoRRabs/2504.07615 (2025)

    Pith/arXiv arXiv 2025

  40. [40]

    Lucy Xiaoyang Shi, Brian Ichter, Michael Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, Adrian Li-Bell, Danny Driess, Lachy Groom, Sergey Levine, and Chelsea Finn. 2025. Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language- Action Models. arXiv:2502.19417 [cs.RO] https://arxiv....

    Pith/arXiv arXiv 2025

  41. [41]

    Suyeon Shin, Sujin jeon, Junghyun Kim, Gi-Cheon Kang, and Byoung-Tak Zhang

  42. [42]

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks. arXiv:1912.01734 [cs.CV] https://arxiv.org/abs/1912.01734

    Pith/arXiv arXiv 2020

  43. [43]

    Sadler, Wei-Lun Chao, and Yu Su

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M. Sadler, Wei-Lun Chao, and Yu Su. 2023. LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models. arXiv:2212.04088 [cs.AI] https://arxiv.org/ abs/2212.04088

    Pith/arXiv arXiv 2023

  44. [44]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieil- lard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas B...

    Pith/arXiv arXiv 2025

  45. [45]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Cong- cong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Ha...

    Pith/arXiv arXiv 2025

  46. [46]

    Songjun Tu, Jiahao Lin, Xiangyu Tian, Qichao Zhang, Linjing Li, Yuqian Fu, Nan Xu, Wei He, Xiangyuan Lan, Dongmei Jiang, and Dongbin Zhao. 2025. Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation. arXiv:2503.12854 [cs.CL] https://arxiv.org/abs/2503.12854

    Pith/arXiv arXiv 2025

  47. [47]

    Songjun Tu, Jiahao Lin, Qichao Zhang, Xiangyu Tian, Linjing Li, Xiangyuan Lan, and Dongbin Zhao. 2025. Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL. arXiv:2505.10832 [cs.CL] https://arxiv.org/abs/2505.10832

    arXiv 2025

  48. [48]

    Songjun Tu, Jingbo Sun, Qichao Zhang, Xiangyuan Lan, and Dongbin Zhao. 2024. Online Preference-based Reinforcement Learning with Self-augmented Feedback from Large Language Model. arXiv:2412.16878 [cs.LG] https://arxiv.org/abs/ 2412.16878

    Pith/arXiv arXiv 2024

  49. [49]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291 [cs.AI] https://arxiv.org/ abs/2305.16291

    Pith/arXiv arXiv 2023

  50. [50]

    Hongcheng Wang, Yinuo Huang, Sukai Wang, Guanghui Ren, and Hao Dong

  51. [51]

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. 2024. Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset. arXiv:2402.14804

    Pith/arXiv arXiv 2024

  52. [52]

    arXiv:2509.24494 [cs.CL] https://arxiv.org/abs/2509

    GRPO-MA: Multi-Answer Generation in GRPO for Stable and Efficient Chain-of-Thought Training. arXiv:2509.24494 [cs.CL] https://arxiv.org/abs/2509. 24494

    arXiv

  53. [53]

    Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I. Wang. 2025. SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Soft- ware Evolution.CoRRabs/2502.18449 (2025)

    Pith/arXiv arXiv 2025

  54. [54]

    Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Am- manabrolu. 2022. ScienceWorld: Is your Agent Smarter than a 5th Grader? arXiv:2203.07540 [cs.CL] https://arxiv.org/abs/2203.07540

    Pith/arXiv arXiv 2022

  55. [56]

    Di Wu, Jiaxin Fan, Junzhe Zang, Guanbo Wang, Wei Yin, Wenhao Li, and Bo Jin. 2025. Reinforced Reasoning for Embodied Planning.CoRRabs/2505.22050 (2025)

    Pith/arXiv arXiv 2025

  56. [57]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huang, a...

    Pith/arXiv arXiv 2023

  57. [58]

    Zhenyu Wu, Ziwei Wang, Xiuwei Xu, Jiwen Lu, and Haibin Yan. 2023. Embodied Task Planning with Large Language Models. arXiv:2307.01848 [cs.CV] https: //arxiv.org/abs/2307.01848

    Pith/arXiv arXiv 2023

  58. [59]

    Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, Heng Ji, Huan Zhang, and Tong Zhang. 2025. EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents. arXiv:2502.09560 [cs.AI] https://arxiv.org/abs/2502.09560

    Pith/arXiv arXiv 2025

  59. [60]

    Zhiyuan Xu, Kun Wu, Junjie Wen, Jinming Li, Ning Liu, Zhengping Che, and Jian Tang. 2024. A Survey on Robotics with Foundation Models: toward Embodied AI. arXiv:2402.02385 [cs.RO] https://arxiv.org/abs/2402.02385

    Pith/arXiv arXiv 2024

  60. [61]

    Michał Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. 2025. Robotic Control via Embodied Chain-of-Thought Reasoning. arXiv:2407.08693 [cs.RO] https://arxiv.org/abs/2407.08693

    Pith/arXiv arXiv 2025

  61. [62]

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. De- mystifying Long Chain-of-Thought Reasoning in LLMs. arXiv:2502.03373 [cs.CL] https://arxiv.org/abs/2502.03373

    Pith/arXiv arXiv 2025

  62. [63]

    Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. 2025. ACECODER: Acing Coder RL via Automated Test-Case Synthesis. arXiv:2502.01718 [cs.SE] https://arxiv.org/abs/2502.01718

    Pith/arXiv arXiv 2025

  63. [64]

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. 2023. AgentTuning: Enabling Generalized Agent Abilities for LLMs. arXiv:2310.12823

    Pith/arXiv arXiv 2023

  64. [65]

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. 2025. R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization. arXiv:2503.12937 [cs.AI] https://arxiv.org/abs/2503.12937

    Pith/arXiv arXiv 2025

  65. [66]

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. 2024. Vision-Language Models for Vision Tasks: A Survey. arXiv:2304.00685 [cs.CV] https://arxiv.org/ abs/2304.00685

    Pith/arXiv arXiv 2024

  66. [67]

    num_train_epochs

    Zijing Zhang, Ziyang Chen, Mingxiao Li, Zhaopeng Tu, and Xiaolong Li. 2025. RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents.CoRRabs/2507.22844 (2025). Appendix A EXPERIENTAL DETAILS A.1 SFT Details In all experiments conducted in this paper, the hyperparameter settings for the supervised fine-tuning (SFT...

    Pith/arXiv arXiv 2025

  67. [68]

    Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Weiming Lu, Peng Li, and Yueting Zhuang. 2025. Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks. arXiv:2503.21696 [cs.CL] https://arxiv.org/abs/2503.21696

    Pith/arXiv arXiv 2025

  68. [2025]

    arXiv:2404.15190 [cs.AI] https://arxiv.org/abs/2404.15190

    Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruc- tion Following. arXiv:2404.15190 [cs.AI] https://arxiv.org/abs/2404.15190

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.