Reinforced Reasoning for Embodied Planning

Bo Jin; Di Wu; Guanbo Wang; Jiaxin Fan; Junzhe Zang; Wei Yin; Wenhao Li

REVIEW 4 major objections 5 minor 2 cited by

Reinforced Reasoning for Embodied Planning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Supervised fine-tuning plus GRPO with a prefix-match reward turns a 7B vision-language model into a competitive embodied planner: 35.6% success on EB-ALFRED and 20% on unseen EB-Habitat.

desk verdict A legitimate first application of R1-style RL to long-horizon embodied planning, with real transfer evidence on EB-Habitat, but the in-domain headline needs a held-out ALFRED split before it can be believed. read the letter →

arxiv 2505.22050 v2 pith:3DYOZFXN submitted 2025-05-28 cs.AI cs.LG

Di Wu , Jiaxin Fan , Junzhe Zang , Guanbo Wang , Wei Yin , Wenhao Li , Bo Jin This is my paper

classification cs.AIcs.LG

keywords embodiedplanningreinforcementfine-tuningGRPOvision-languagemodelsrule-basedrewardlong-horizonR1-stylereasoninginteractivebenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that R1-style reinforcement fine-tuning, the same reward-driven recipe that boosted math and code reasoning, can be carried over to embodied planning, where an agent must act step by step in a simulated home from egocentric images and language goals. The authors claim that a two-stage pipeline does this: supervised fine-tuning on plans distilled from a large proprietary model, then GRPO optimization with a rule-based reward that scores structured output format and exact prefix matches against gold action sequences. On the interactive Embench benchmark, the resulting 7B model reaches 35.6% task success on seen EB-ALFRED tasks, above GPT-4o-mini (22.0%) and 70B+ open-source models, and beats all similar-size 7B baselines on unseen EB-Habitat tasks. The broader point is that reinforcement-driven reasoning can generalize beyond static question answering to long-horizon interactive decision-making, and that training without simulator interaction can still transfer to interactive evaluation.

What carries the argument

The load-bearing object is a rule-based composite reward combined with GRPO. The reward splits into $R_{\text{format}}$ (structure, valid steps, matched action id/name pairs) and $R_{\text{accuracy}} = \frac{n(n+1)}{k(k+1)}$, where $n$ is the number of steps the generated plan matches the gold action sequence as a prefix and $k$ is the total gold length. This curve assigns disproportionately more reward to longer correct prefixes, steering the policy toward complete rather than myopic plans; GRPO turns the reward into a group-relative advantage, and an online filtering step discards prompt groups whose responses are all too good or too bad, stabilizing learning.

What would settle it

Take EB-ALFRED tasks with multiple valid action sequences and evaluate the trained policy after swapping the gold ordering for another valid ordering in the reward computation; if success on the same tasks drops sharply merely because the reference order changed, the policy has learned trace-matching rather than goal-directed planning. A cheaper check is to verify whether any alternative valid plan receives near-zero accuracy reward under Eq. (10).

Watch

Extended reading notes

Core claim

The central claim is that a 7-billion-parameter vision-language model can be made into a strong interactive planner by first imitating high-quality distilled trajectories and then optimizing a rule-based reward with GRPO. The accuracy reward is computed by exact prefix matching: $n$ consecutive correct actions out of $k$ give $R_{\text{accuracy}} = \frac{n(n+1)}{k(k+1)}$, a triangular normalization that grows quadratically with the length of the correctly matched prefix, plus format rewards for structure, valid steps, and matched action id/name pairs. The authors report 35.6% average success on EB-ALFRED, outperforming GPT-4o-mini and much larger models such as Qwen2.5-VL-72B and LLaMA3.2-90B-Vision-Ins, and 20% on the out-of-domain EB-Habitat environment, best among 7B baselines. They interpret this as evidence that RL fine-tuning, not scale or verbosity, is what improves spatial and temporal planning; they also report that reasoning length did not grow and long-horizon tasks remained difficult.

Load-bearing premise

The load-bearing premise is that one gold action sequence extracted from ALFRED fairly represents all valid ways to complete each task; the paper itself notes embodied tasks have no unique ground-truth trajectories, so if that single sequence is unrepresentative, the reward pushes the model to imitate one trace rather than to satisfy the goal.

Editorial extensions

If this is right

If the pipeline works as claimed, a 7B open VLM can beat closed-source and 70B+ models on interactive embodied planning without any simulator interaction during training.
RFT is what transfers across environments: SFT alone improves seen EB-ALFRED but not unseen EB-Habitat, while SFT followed by RFT raises both.
Longer reasoning traces are not a reliable signal for planning quality; after math-style pretraining encourages verbosity, the model's output length shrinks back toward concise plans during embodied RFT.
The gains concentrate in Base-type tasks because the RFT dataset is built from ALFRED base instructions, implying broader training diversity would be needed for other task categories.
Long-horizon tasks remain a weak spot, so the method's success on shorter plans does not yet solve temporal consistency over very long sequences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the gold ALFRED trace is just one of many valid solutions, the prefix-match reward may penalize correct alternative orderings; a direct test is to score alternative valid plans and see whether the reward assigns them low values.
Editorial inference: the out-of-domain EB-Habitat gain, concentrated in Base tasks, suggests the model transfers action-skill patterns rather than memorized scenes, but this would be better supported by an ablation that removes shared action vocabulary between the two environments.
Editorial inference: the offline reward design could be upgraded to goal-conditioned rewards computed by the simulator at each step, turning the admitted Section 5 limitation into a testable extension of the method.
Editorial inference: coupling this planning policy with a low-level controller in a real robot is the natural next evaluation, since success in simulation does not by itself establish physical-world deployment readiness.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read it. Short version: the paper's real contribution is the first clean application of R1-style RL (SFT + GRPO) to long-horizon embodied planning, and the EB-Habitat transfer result is the strongest evidence that the method does something beyond memorizing ALFRED. But the headline in-domain numbers—35.6% vs GPT-4o-mini's 22% and 70B models around 33%—are not yet trustworthy, because the paper never states a train/eval split for the ALFRED side. That's a serious reporting gap, and it needs fixing before the claim can be believed.

What's genuinely good: the authors release code and data, evaluate in an interactive simulator (Embench), and include ablations that show the two-stage order (SFT before RFT) and the data filter both matter. The EB-Habitat result (20% vs 12.7% base, best among 7-8B baselines) is a real out-of-domain signal, and it's consistent with the method having learned a transferable planning skill. They also admit in Section 5 that the reward is offline and doesn't interact with the simulator—honest, and a fair limitation.

Soft spots, in order of severity. First, the train/eval overlap issue. Appendix B.1 says the SFT data was distilled from 'a subset of EB-ALFRED tasks'—the benchmark itself. The RFT data comes from ALFRED traces, and EB-ALFRED is 'built upon the ALFRED dataset.' No split is stated. The stress-test note is right: without a holdout, the 35.6% could be at least partly trained in. The reward (Eq. 10-11) makes this worse: exact prefix matching against a single gold trajectory rewards reproducing that path, even though the paper itself notes embodied tasks have multiple valid solutions. Second, the EB-Habitat table undercuts the abstract's 'significantly outperforms GPT-4o-mini': on that split GPT-4o-mini gets 32.3%, better than the method's 20%. The superiority claim survives only on the possibly contaminated in-domain split. Third, no error bars, no seeds, no task counts. Single runs. Minor, but it makes the differences harder to parse.

On balance: the method is plausible, the transfer result is encouraging, and the eval protocol is fixable. But as written, the central empirical claim is under-supported. The paper deserves a serious referee, not a desk reject, because the question—whether RL fine-tuning helps embodied planning—is real and the authors have done the work to set up a reasonable test. I'd recommend accept-with-major-revisions, with the mandatory revision being a held-out ALFRED/EB-ALFRED re-evaluation with the training instances explicitly excluded, plus seeds and error bars. A DPO baseline or an interactive RL comparison would be a nice addition, though not strictly required.

Referee Report

4 major / 5 minor

Summary. The paper presents a two-stage training pipeline for embodied planning with a 7B vision-language model (Qwen2.5-VL-7B). Stage 1 is supervised fine-tuning (SFT) on a dataset distilled from Gemini-2.0-flash, using prompts and responses from EB-ALFRED tasks. Stage 2 is reinforcement fine-tuning (RFT) via GRPO with a hand-designed rule-based reward that combines a format reward and a prefix-matching accuracy reward computed against gold action sequences from ALFRED. The method is evaluated on Embench's EB-ALFRED (labeled 'seen') and EB-Habitat (labeled 'unseen') interactive environments, reporting success rates of 35.6% on EB-ALFRED and 20% on EB-Habitat. The paper claims that this significantly outperforms GPT-4o-mini and larger open-source models on EB-ALFRED, and that it generalizes to EB-Habitat better than other 7B-scale baselines. The authors release code and data.

Significance. If the headline results hold, the contribution is a simple and reproducible recipe: SFT plus GRPO with a rule-based prefix-match reward can turn a 7B VLM into a competitive interactive planner, closing a large part of the gap to much larger proprietary models. The paper includes open code and data, a recent interactive benchmark, ablations of the two training stages and of the reward curve and data-filtering components, and an out-of-domain evaluation on EB-Habitat that provides some independent evidence of transfer. The EB-Habitat result (20% vs 12.7% for the base model, and best among 7–8B baselines) is the most trustworthy part of the empirical contribution and is not affected by the training/evaluation overlap concern that applies to EB-ALFRED.

major comments (4)

[§4.2.1 and Appendix B.1] The paper does not state that the EB-ALFRED evaluation tasks are held out from training at the instance level. Appendix B.1 says the SFT dataset was built by prompting Gemini-2.0-flash 'on a subset of EB-ALFRED tasks from the Embench benchmark', and Section 3.3 builds the 43,898-sample RFT set from ALFRED execution traces, while EB-ALFRED is itself 'built upon the ALFRED dataset' (Appendix D.1). Without an explicit exclusion or a separate held-out task set, the headline 35.6% vs 22.0% comparison against GPT-4o-mini on EB-ALFRED may reflect training on test. Please specify the exact split, or re-evaluate on a disjoint set of tasks and report those numbers separately.
[§3.3, Eqs. (10)–(11)] The accuracy reward is an exact prefix match against a single gold action sequence, but Section 1 correctly notes that embodied tasks have no unique ground-truth trajectories. This reward optimizes imitation of one particular path rather than satisfaction of the goal conditions defined in Eq. (3). As a result, the in-domain gains on EB-ALFRED could be driven by memorizing ALFRED traces rather than by improved planning. The authors should either justify why the chosen gold sequence is representative, or replace/augment the reward with a goal-checking or multiple-trajectory reward. The offline nature of the reward (no simulator interaction during training, as admitted in Section 5) further weakens the connection between the training objective and the interactive evaluation metric.
[Tables 1–2 and §4.2] The paper reports single-point success rates without the number of tasks per category, the number of evaluation seeds, or any variance/confidence information. The claim that the method 'significantly outperforms' baselines is not backed by statistical evidence; a difference of 35.6% vs 33.7% could be noise if per-category task counts are small. Please report per-category task counts and at least three independent evaluation seeds with standard deviations, or provide a significance test.
[Abstract and §4.2.2] The abstract states that the method 'significantly outperforms ... GPT-4o-mini', but this is only true on EB-ALFRED. On the EB-Habitat (unseen) split in Table 2, GPT-4o-mini achieves 32.3% while the proposed method achieves 20%. The abstract and Section 1 should qualify the claim by environment, for example by saying that the method outperforms GPT-4o-mini on the seen in-domain benchmark while exhibiting strong transfer relative to similar-scale open models on the unseen benchmark.

minor comments (5)

[Table 1] The main text cites Qwen2.5-VL-72B (33.7%) and LLaMA3.2-90B-Vision-Ins (32.0%), but these rows are missing from Table 1; they appear only in Figure 3. Either add them to the table or clearly indicate that the cited numbers come from the figure.
[§4.4] The sentence 'As shown in Figure 3, our model does not generate longer reasoning outputs' mistakenly references Figure 3, which plots success rate versus parameters. The response-length analysis is presented in the right panel of Table 3 and in Figure 9 of Appendix C.3; please correct the cross-reference.
[§3.3, Eq. (12)] The GRPO advantage formula divides by the standard deviation of rewards in a group. If all responses in a group receive the same reward, this is a division by zero; please state how this case is handled in the implementation.
[Appendix C.1] The statement that the RFT dataset 'does not reuse the SFT-distilled dataset' is helpful but should be accompanied by a quantitative comparison of the two datasets' task overlap and instruction templates, since both ultimately derive from the ALFRED family.
[§2.2 and §4.2.1] The paper labels the approach 'R1-style' but the optimization is GRPO from DeepSeekMath; clarifying the lineage (e.g., that GRPO is the algorithm used by DeepSeek-R1) would avoid confusion for readers not familiar with the terminology.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: training rewards and evaluation goals are distinct, and the EB-Habitat out-of-domain result provides independent support for the generalization claim.

full rationale

The paper's derivation chain is empirical rather than definitional. SFT (Eq. 7) maximizes log-likelihood on Gemini-distilled responses; RFT (Eqs. 8-13) optimizes a rule-based reward whose accuracy term (Eqs. 10-11) is an exact-prefix match against ALFRED gold action sequences, while the reported evaluation metric is task success defined by goal-checking conditions (Eq. 3). These are different quantities, so the training signal is not identical by construction to the evaluation metric. No load-bearing result is imported from a self-citation: the cited methods (DeepSeek-R1, GRPO, PRIME, MM-Eureka, EmbodiedBench) are external prior work with no author overlap. The only caveat is benchmark hygiene: Appendix B.1 states the SFT set was built by querying Gemini-2.0-flash 'on a subset of EB-ALFRED tasks from the Embench benchmark', and Appendix D.1 says 'we use EB-ALFRED for in-domain training and evaluation'; Table 1 labels this split 'Seen'. The paper never states a task-level holdout, so the 35.6% EB-ALFRED number should be read as in-distribution fitting rather than as an independent prediction. This is a data-splitting caveat, not a circular reduction, and the EB-Habitat results (Table 2, where the method beats all 7B baselines) plus Section 5's explicit admission that RFT is offline provide independent, honest evidence. Accordingly, no circular step is established.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central method contains no fitted scientific constants, but it relies on several hand-chosen reward constants, filtering thresholds, and a quadratic reward curve, none of which are derived from data or tested for sensitivity. The key domain assumptions concern the quality of distilled demonstrations, the validity of exact prefix matching against one gold trajectory as a learning signal, and the transfer of offline rewards to interactive evaluation.

free parameters (3)

Format reward constants = 0.125, 0.125, 0.25, penalty -0.25
Hand-chosen constants in Eq. 9 that determine how much format, valid-step, and action-match quality contribute to the total reward; no sensitivity analysis is provided.
GRPO data filtering bounds = epsilon_lower=0.1, epsilon_upper=0.9
Hand-chosen group-accuracy thresholds in Eq. 14 and Table 5 that decide which prompt groups are kept for training, affecting stability and final policy.
Multi-step reward allocation curve = R(n;k)=n(n+1)/(k(k+1))
The quadratic triangular normalization in Eq. 11 is a design choice that sets how much reward longer correct prefixes receive relative to shorter ones; it is not derived from data or ablations.

assumptions (5)

domain assumption Single gold ALFRED trajectories are valid and representative planning references
The RFT reward compares every generated plan against one reference trajectory via prefix matching in Section 3.3, even though the paper notes multiple valid plans can exist for a task.
domain assumption Gemini-2.0-flash distilled outputs provide high-quality supervision without filtration
Section 3.2 builds the SFT dataset from Gemini-2.0-flash outputs without human verification or quality filtering, so the SFT initialization depends on the closed-source model's reliability.
ad hoc to paper Rule-based prefix-match reward is a sufficient proxy for embodied task success
Eqs. 9 to 11 define the reward as format correctness plus exact prefix matching against gold actions, but a plan that achieves the goal through a different valid action sequence receives a lower reward.
domain assumption Offline training signals transfer to interactive evaluation
The RL stage never interacts with the simulator, as stated in Section 5, yet the evaluation is interactive; the method assumes the offline reward teaches transferable planning behavior.
domain assumption ALFRED and EB-ALFRED share action semantics sufficiently for transfer
The RFT data is built from ALFRED with its own action list, while evaluation uses EB-ALFRED prompts with different action ids and validity rules; the method assumes this gap is bridgeable.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reinforced Reasoning for Embodied Planning." pith.science (2026). https://pith.science/paper/3DYOZFXN

@misc{pith2026250522050,
  author       = {Pith},
  title        = {Pith review of: Reinforced Reasoning for Embodied Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3DYOZFXN}},
  note         = {Machine review of arXiv:2505.22050}
}

read the original abstract

Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and natural language goals. While recent vision-language models (VLMs) excel at static perception tasks, they struggle with the temporal reasoning, spatial understanding, and commonsense grounding needed for planning in interactive environments. In this work, we introduce a reinforcement fine-tuning framework that brings R1-style reasoning enhancement into embodied planning. We first distill a high-quality dataset from a powerful closed-source model and perform supervised fine-tuning (SFT) to equip the model with structured decision-making priors. We then design a rule-based reward function tailored to multi-step action quality and optimize the policy via Generalized Reinforced Preference Optimization (GRPO). Our approach is evaluated on Embench, a recent benchmark for interactive embodied tasks, covering both in-domain and out-of-domain scenarios. Experimental results show that our method significantly outperforms models of similar or larger scale, including GPT-4o-mini and 70B+ open-source baselines, and exhibits strong generalization to unseen environments. This work highlights the potential of reinforcement-driven reasoning to advance long-horizon planning in embodied AI.

Figures

Figures reproduced from arXiv: 2505.22050 by the authors.

**Figure 2.** Overview of our proposed framework. We adopt a two-stage training paradigm [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Success rate vs. parameters (Seen) Stage1:SFT Stage2:RFT [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (9 more)

**Figure 5.** Figure 5: Success rate vs. parameters (Unseen) Stage2:RFT Stage1:SFT GPT-4o GPT-4o-mini Qwen2-VL-7B-Ins Qwen2.5-VL-7B-Ins Ours InternVL2_5-8B Llama-3.2-11B-Vision-Ins InternVL2_5-38B Qwen2-VL-72B-Ins Qwen2.5-VL-72B-Ins Llama-3.2-90B-Vision-Ins InternVL2_5-78B Ours Llama InternVL…

**Figure 7.** Figure 7: Summary of SFT training results. Metric Value Epochs 3.0 Total FLOPs 3.13e13 Training Loss 0.252 Runtime (s) 21111.79 Samples/sec 0.142 Steps/sec 0.018 Training Loss of SFT stage [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

**Figure 8.** Figure 8: Training loss curve during SFT stage. C Additional Details of RFT training stage C.1 Training Dataset Details We construct our reinforcement fine-tuning (RFT) dataset based on the ALFRED benchmark, following the decomposition and formatting strategy described in Sectio…

**Figure 9.** Figure 9: Training curve during reinforcement fine-tuning. The figure shows the progression [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]

**Figure 10.** Figure 10: Case comparison (Our model vs Base model) in EB-ALFRED [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]

**Figure 11.** Figure 11: Case comparison (Our model vs Base model) in EB-Habitat [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]

**Figure 12.** Figure 12: Our model’s Full Trajectory execution in EB-ALFRED [PITH_FULL_IMAGE:figures/full_fig_p027_12.png]

**Figure 13.** Figure 13: Our model’s Full Trajectory execution example 1 in EB-Habitat [PITH_FULL_IMAGE:figures/full_fig_p028_13.png]

**Figure 15.** Figure 15: Our model’s Full Trajectory execution example 2 (part 2) in EB-Habitat [PITH_FULL_IMAGE:figures/full_fig_p030_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learning
cs.AI 2025-10 conditional novelty 5.0 of 10

A 3B VLM trained with SFT plus GRPO and an LCS-based reward reaches 55.3% on EmbodiedBench's EB-ALFRED, beating GPT-4o-mini and the 7B REBP planner.
Large Language Models for Next-Generation Wireless Network Management: A Survey and Tutorial
cs.NI 2025-09 conditional novelty 4.0 of 10

A survey and tutorial that organizes LLM-enabled wireless network optimization into formulation, solution, and verification stages, with case studies drawn from the authors' own prior papers.

Reference graph

Works this paper leans on

65 extracted references · 18 canonical work pages · cited by 2 Pith papers

[1]

URL: https://openai.com/index/ gpt-4o-mini-advancing-cost-efficient-intelligence/

GPT-4o mini: advancing cost-efficient intelligence. URL: https://openai.com/index/ gpt-4o-mini-advancing-cost-efficient-intelligence/

work page
[2]

URL: https://openai.com/index/hello-gpt-4o/

Hello GPT-4o | OpenAI. URL: https://openai.com/index/hello-gpt-4o/

work page
[3]

URL: https://www.anthropic.com/news/ claude-3-5-sonnet

Introducing claude 3.5 sonnet \ anthropic. URL: https://www.anthropic.com/news/ claude-3-5-sonnet

work page
[4]

URL: https://blog

Introducing gemini 2.0: our new AI model for the agentic era. URL: https://blog. google/technology/google-deepmind/google-gemini-ai-update-december-2024/

work page 2024
[5]

URL: https://ai.meta.com/blog/ llama-3-2-connect-2024-vision-edge-mobile-devices/

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. URL: https://ai.meta.com/blog/ llama-3-2-connect-2024-vision-edge-mobile-devices/

work page 2024
[6]

Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, By- ron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J. Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang...

arXiv doi:10.48550/arxiv.2204.01691
[7]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

arXiv 2025
[8]

RoboGPT: an intelligent agent of making embodied long- term decisions for daily instruction tasks

Yaran Chen, Wenbo Cui, Yuanwen Chen, Mining Tan, Xinyao Zhang, Dongbin Zhao, and He Wang. RoboGPT: an intelligent agent of making embodied long- term decisions for daily instruction tasks. URL: http://arxiv.org/abs/2311.15649, arXiv:2311.15649[cs], doi:10.48550/arXiv.2311.15649

arXiv doi:10.48550/arxiv.2311.15649

Show all 65 references

[9]

Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024

2024 arXiv
[10]

Process reinforcement through implicit rewards

Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456, 2025

2025 arXiv
[11]

A survey of embodied ai: From simulators to research tasks

Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230–244, 2022

2022
[12]

Open r1: A fully open reproduction of deepseek-r1, January 2025

Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025. URL: https://github.com/huggingface/open-r1

2025
[13]

What can vlms do for zero-shot embodied task planning? In ICML 2024 Workshop on LLMs and Cognition, 2024

Xian Fu, Min Zhang, Peilong Han, Hao Zhang, Lei Shi, Hongyao Tang, et al. What can vlms do for zero-shot embodied task planning? In ICML 2024 Workshop on LLMs and Cognition, 2024

2024
[14]

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. 12

2025 arXiv
[15]

Lora: Low-rank adaptation of large language models

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

2022
[16]

Open- rlhf: An easy-to-use, scalable and high-performance rlhf framework

Jian Hu, Xibin Wu, Zilin Zhu, Weixun Wang, Dehao Zhang, Yu Cao, et al. Open- rlhf: An easy-to-use, scalable and high-performance rlhf framework. arXiv preprint arXiv:2405.11143, 2024

2024 arXiv
[17]

Look before you leap: Un- veiling the power of GPT-4v in robotic vision-language planning

Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, and Yang Gao. Look before you leap: Un- veiling the power of GPT-4v in robotic vision-language planning. URL: http://arxiv. org/abs/2311.17842, arXiv:2311.17842[cs], doi:10.48550/arXiv.2311.17842

doi arXiv
[18]

Vision-r1: Incentivizing reasoning capability in multimodal large language models

Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

2025 arXiv
[19]

RoboBrain: A unified brain model for robotic manipulation from abstract to concrete

Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, Xinda Xue, Qinghang Su, Huaihai Lyu, Xiaolong Zheng, Jiaming Liu, Zhongyuan Wang, and Shanghang Zhang. RoboBrain: A unified brain model for robotic manipu...

doi arXiv
[20]

Context-aware planning and environment-aware memory for instruction following embodied agents

Byeonghwi Kim, Jinyeon Kim, Yuyeong Kim, Cheolhong Min, and Jonghyun Choi. Context-aware planning and environment-aware memory for instruction following embodied agents. URL: http://arxiv.org/abs/2308.07241, arXiv:2308.07241[cs], doi:10.48550/arXiv.2308.07241

doi arXiv
[21]

Openvla: An open-source vision-language-action model

Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, et al. Openvla: An open-source vision-language-action model. In 8th Annual Conference on Robot Learning
[22]

Ai2-thor: An interactive 3d environment for visual ai

Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Her- rasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474, 2017

2017 arXiv
[23]

Videochat-r1: Enhancing spatio-temporal percep- tion via reinforcement fine-tuning

Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal percep- tion via reinforcement fine-tuning. arXiv preprint arXiv:2504.06958, 2025

2025 arXiv
[24]

Let’s verify step by step

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023

2023
[25]

Visual-rft: Visual reinforcement fine-tuning

Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785, 2025

2025 arXiv
[26]

A survey on vision-language-action models for embodied ai

Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. A survey on vision-language-action models for embodied ai. arXiv preprint arXiv:2405.14093, 2024

2024 arXiv
[27]

Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning

Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

2025 arXiv
[28]

Composi- tional chain-of-thought prompting for large multimodal models

Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Composi- tional chain-of-thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14420–14431, 2024. 13

2024
[29]

Kam-cot: Knowledge augmented multimodal chain-of-thoughts reasoning

Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh, and Godawari Sud- hakar Rao. Kam-cot: Knowledge augmented multimodal chain-of-thoughts reasoning. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 18798– 18806, 2024

2024
[30]

EmbodiedGPT: Vision-language pre-training via embodied chain of thought

Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. EmbodiedGPT: Vision-language pre-training via embodied chain of thought. URL: http://arxiv.org/abs/2305.15021, arXiv:2305. 15021[cs], doi:10.48550/arXiv.2305.15021

doi arXiv
[31]

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:2773...

2022
[32]

Reasoning with large language models, a survey

Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. Reasoning with large language models, a survey. arXiv preprint arXiv:2407.11511, 2024

2024
[33]

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

2023
[34]

Say- Plan: Grounding large language models using 3d scene graphs for scalable robot task planning

Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou-Chakra, and Ian Reid. Say- Plan: Grounding large language models using 3d scene graphs for scalable robot task planning
[35]

Habitat: A platform for embodied ai research

Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. InProceedings of the IEEE/CVF international conference on computer vision, p...

2019
[36]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

2024 arXiv
[37]

Vlm-r1: A stable and generalizable r1-style large vision-language model

Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025

2025 arXiv
[38]

Hi robot: Open- ended instruction following with hierarchical vision-language-action models

Lucy Xiaoyang Shi, Brian Ichter, Michael Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, et al. Hi robot: Open- ended instruction following with hierarchical vision-language-action models. arXiv preprint arXiv:2502.19417, 2025

2025 arXiv
[39]

Socratic planner: Inquiry-based zero-shot planning for embodied instruction following

Suyeon Shin, Sujin jeon, Junghyun Kim, Gi-Cheon Kang, and Byoung-Tak Zhang. Socratic planner: Inquiry-based zero-shot planning for embodied instruction following. URL: http://arxiv.org/abs/2404.15190, arXiv:2404.15190[cs]

arXiv
[40]

Alfred: A benchmark for interpreting grounded instructions for everyday tasks

Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern ...

2020
[41]

Tenenbaum, Leslie Kaelbling, and Michael Katz

Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B. Tenenbaum, Leslie Kaelbling, and Michael Katz. Generalized planning in PDDL domains with pretrained large language models. 38(18):20256–20264. URL: https://ojs.aaai.org/index.php/AAAI/article/ view/30006, doi:10.1609/aaai.v38i18.30006

doi
[42]

ProgPrompt: Generating situated robot task plans using large language models

Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. ProgPrompt: Generating situated robot task plans using large language models. URL: http://arxiv.org/abs/ 2209.11302, arXiv:2209.11302[cs], do...

doi arXiv
[43]

Trial and error: Exploration-based trajectory optimization for llm agents

Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. Trial and error: Exploration-based trajectory optimization for llm agents. arXiv preprint arXiv:2403.02502, 2024

2024 arXiv
[44]

Reason-rft: Reinforcement fine-tuning for visual reasoning

Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint arXiv:2503.20752, 2025

2025
[45]

Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

2025 arXiv
[46]

World modeling makes a better planner: Dual preference optimization for embodied task planning

Siyin Wang, Zhaoye Fei, Qinyuan Cheng, Shiduo Zhang, Panpan Cai, Jinlan Fu, and Xipeng Qiu. World modeling makes a better planner: Dual preference optimization for embodied task planning. URL: http://arxiv.org/abs/2503.10480, arXiv:2503. 10480[cs], doi:10.48550/arXiv.2503.10480

doi arXiv
[47]

Multimodal chain-of-thought reasoning: A comprehensive survey

Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thought reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12605, 2025

2025 arXiv
[48]

Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey on emerging trends in multimodal reasoning

Yiqi Wang, Wentao Chen, Xiaotian Han, Xudong Lin, Haiteng Zhao, Yongfei Liu, Bohan Zhai, Jianbo Yuan, Quanzeng You, and Hongxia Yang. Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey on emerging trends in multimodal reasonin...

2024 arXiv
[49]

Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning

Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, et al. Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning. arXiv preprint arXiv:2504.20073, 2025

2025 arXiv
[50]

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

2022
[51]

Embodied task planning with large language models

Zhenyu Wu, Ziwei Wang, Xiuwei Xu, Jiwen Lu, and Haibin Yan. Embodied task planning with large language models. arXiv preprint arXiv:2307.01848, 2023

2023 arXiv
[52]

The rise and potential of large language model based agents: A survey

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. Science China Information Sciences, 68(2):121101, 2025

2025
[53]

A survey on robotics with foundation models: toward embodied ai

Zhiyuan Xu, Kun Wu, Junjie Wen, Jinming Li, Ning Liu, Zhengping Che, and Jian Tang. A survey on robotics with foundation models: toward embodied ai. arXiv preprint arXiv:2402.02385, 2024

2024 arXiv
[54]

Em- bodiedbench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents

Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, et al. Em- bodiedbench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents. arXiv preprint a...

2025 arXiv
[55]

Limo: Less is more for reasoning

Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387, 2025

2025 arXiv
[56]

Robotic control via embodied chain-of-thought reasoning

Michał Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied chain-of-thought reasoning. In 8th Annual Conference on Robot Learning, 2024

2024
[57]

Hirt: Enhancing robotic control with hierarchical robot transformers

Jianke Zhang, Yanjiang Guo, Xiaoyu Chen, Yen-Jen Wang, Yucheng Hu, Chengming Shi, and Jianyu Chen. Hirt: Enhancing robotic control with hierarchical robot transformers. arXiv preprint arXiv:2410.05273, 2024. 15

2024 arXiv
[58]

Vision-language models for vision tasks: A survey

Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

2024
[59]

R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization

Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503.12937, 2025

2025 arXiv
[60]

Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks

Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, et al. Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks. arXiv preprint arXiv:2503.21696, 2025

2025 arXiv
[61]

Multimodal chain-of-thought reasoning in language models

Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923, 2023

2023 arXiv
[62]

Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning

Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, and Wenwu Zhu. Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning. arXiv preprint arXiv:2504....

2025 arXiv
[63]

messages

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372, 2024. 16 A Appendix Contents • Section B: Additional Details of SFT Training...

2024 arXiv
[64]

reasoning_and_reflection\

**Reflection on History and Feedback**: Use interaction history and feedback from the environment to refine and improve your current plan. ## Expected JSON output format ```json {\"reasoning_and_reflection\": \"< string>\", \"visual_state_description\": \"<string>\", \"languag...
[224]

from the available actions to execute." 33 Part of EB-ALFRED Action list action id 1: find a Potato, action id 2: find a Faucet, action id 3: find a Ottoman, action id 4: find a CoffeeMachine, action id 5: find a Candle, action id 6: find a CD, action id 7: find a Pan, action ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

URL: https://openai.com/index/ gpt-4o-mini-advancing-cost-efficient-intelligence/

GPT-4o mini: advancing cost-efficient intelligence. URL: https://openai.com/index/ gpt-4o-mini-advancing-cost-efficient-intelligence/

work page

[2] [2]

URL: https://openai.com/index/hello-gpt-4o/

Hello GPT-4o | OpenAI. URL: https://openai.com/index/hello-gpt-4o/

work page

[3] [3]

URL: https://www.anthropic.com/news/ claude-3-5-sonnet

Introducing claude 3.5 sonnet \ anthropic. URL: https://www.anthropic.com/news/ claude-3-5-sonnet

work page

[4] [4]

URL: https://blog

Introducing gemini 2.0: our new AI model for the agentic era. URL: https://blog. google/technology/google-deepmind/google-gemini-ai-update-december-2024/

work page 2024

[5] [5]

URL: https://ai.meta.com/blog/ llama-3-2-connect-2024-vision-edge-mobile-devices/

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. URL: https://ai.meta.com/blog/ llama-3-2-connect-2024-vision-edge-mobile-devices/

work page 2024

[6] [6]

Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, By- ron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J. Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang...

arXiv doi:10.48550/arxiv.2204.01691

[7] [7]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

arXiv 2025

[8] [8]

RoboGPT: an intelligent agent of making embodied long- term decisions for daily instruction tasks

Yaran Chen, Wenbo Cui, Yuanwen Chen, Mining Tan, Xinyao Zhang, Dongbin Zhao, and He Wang. RoboGPT: an intelligent agent of making embodied long- term decisions for daily instruction tasks. URL: http://arxiv.org/abs/2311.15649, arXiv:2311.15649[cs], doi:10.48550/arXiv.2311.15649

arXiv doi:10.48550/arxiv.2311.15649