Pith. sign in

REVIEW 4 major objections 5 minor 44 references

RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a 3-billion-parameter on-board LLM can beat cloud-based GPT-4o at adapting a race car's control behavior after closed-loop reinforcement learning.

desk verdict Genuinely useful closed-loop RLVR/GRPO method for LLM-driven MPC adaptation, but the headline 3B-beats-GPT-4o claim rests on an undefined aggregate and an arithmetic slip. read the letter →

arxiv 2505.03238 v2 pith:T5VLBTEN submitted 2025-05-06 cs.RO

Liam Boyle , Nicolas Baumann , Paviththiren Sivasothilingam , Michele Magno , Luca Benini This is my paper · ORCID
classification cs.RO
keywords RobotxR1closed-loopreinforcementlearningRLVRGRPOembodiedAIautonomousdrivingmodelpredictivecontrolsmalllanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that small, edge-deployable language models can learn robotic control skills by interacting with a simulated environment through closed-loop reinforcement learning, rather than only by imitating large models through supervised fine-tuning. It extends the R1-zero recipe, originally built for static math datasets, to a driving task where an LLM proposes changes to a model predictive controller's parameters and receives a reward based on actual driving error. The key claimed result is that a Qwen2.5-3B model trained with supervised fine-tuning followed by closed-loop reinforcement learning reaches a 63.3% control adaptability score, surpassing the 58.5% obtained by the much larger, cloud-bound GPT-4o. If correct, the result would mean that practical embodied intelligence does not require huge cloud models, since a small model trained through environmental feedback can outperform them on a real robot.

What carries the argument

The central mechanism is the MPCxR1 closed-loop training loop: an LLM outputs a candidate set of MPC parameters (new_mpc_params), the car drives a lap in the F1TENTH simulator under those parameters, and a driving reward Rdrive = max((EMPC_B − ELLM_B)/EMPC_B, −4) is computed from behavior-specific RMSE metrics, with GRPO updating the LLM. The trailing modules are DecisionxR1, which learns to judge whether the car adheres to a prompted behavior, and MPCxR1, which learns to adapt control parameters; both use RAG-augmented prompts and combine accuracy, formatting, and parameter-validity rewards.

What would settle it

Reproduce Table 2 by computing each model's control adaptability as the mean of the four per-metric improvement percentages over the default MPC on The Grand Tour map: for Qwen2.5-3B with SFT+RLVR this gives (39.9 + 90.2 + 91.2 + 31.8)/4 = 63.3%, and for GPT-4o (5.0 + 93.2 + 97.8 + 38.0)/4 = 58.5%, so the headline ranking is checkable from the table alone; a different aggregation rule would change the ranking. A further check is to run the five rephrased prompts per behavior on the same map with the released code and compare the per-metric RMSEs against the reported values.

Watch

Extended reading notes

Core claim

The paper claims that moving RLVR training from static datasets into a closed-loop simulator lets small LLMs learn to retune an MPC's cost weights and constraints so that a scaled autonomous car follows natural-language driving prompts. In the proposed two-stage procedure, a Qwen2.5-3B model first undergoes SFT and then GRPO-based RLVR, during which it proposes MPC parameter sets, the car completes laps in simulation, and the reward measures behavioral error reduction relative to the default MPC. The trained 3B model achieves a 63.3% control adaptability improvement over the default MPC on an unseen complex track with rephrased prompts, compared with GPT-4o's 58.5%; the 1.5B model gains 20.2 percentage points over its SFT-only baseline. The paper further claims that this works because the training signal is behavioral, derived from actual driving outcomes, not from text imitation.

Load-bearing premise

The central comparison rests on an unreported aggregation of four hand-chosen driving metrics into the 'Improve' column, and if those metrics or their weighting do not faithfully capture the prompted behaviors, the 63.3% versus 58.5% gap loses its meaning; no variance or formula is reported in the text.

Editorial extensions

If this is right

  • A 3B LLM trained with closed-loop RLVR can be quantized and deployed on an embedded Jetson Orin AGX, reaching about 38.78 tokens/s and 8.3 s average latency while handling compound prompts like 'Drive smoothly at 2 m/s'.
  • Behavior learned on a simple circular track transfers to a more complex, unseen evaluation track, indicating the closed-loop reward, not memorization of the training map, drives the adaptation.
  • SFT alone is insufficient: adding RLVR improves the 1.5B model from 11.3% to 41.5% control adaptability and the 3B model from 50.4% to 63.3%, while RLVR alone is worse than SFT, so the two-stage order matters.
  • Training is feasible on a single consumer GPU (about 24 hours and 11 GB VRAM for the 3B model), lowering the barrier to embodied RLVR experiments.
  • The comparison with GPT-4o suggests that for embodied control, closed-loop environmental feedback can compensate for a large gap in parameter count and cloud-scale pretraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'Improve' column in Table 2 matches the unweighted mean of the four per-metric improvement percentages: for Qwen2.5-3B with SFT+RLVR, (39.9 + 90.2 + 91.2 + 31.8)/4 = 63.3%, and for GPT-4o, (5.0 + 93.2 + 97.8 + 38.0)/4 = 58.5%, so the headline gap is reproducible from the table even though the paper never states the aggregation formula.
  • Because the reward is relative improvement over the default MPC, the metric's value depends on how well that default is tuned; a deliberately poor default would inflate all scores, so cross-paper or cross-system comparisons only make sense with an identical baseline controller.
  • A natural testable extension is to apply the same closed-loop GRPO recipe to another high-level controller interface, such as a manipulation planner with differentiable cost terms; the paper's own limitation section notes the framework is currently tailored to the autonomous car platform.
  • The appendix shows output token length decreases during RLVR despite no brevity reward, opposite to the long chain-of-thought behavior in DeepSeek R1, suggesting that embodied tasks favor short, decisive parameter edits, which could inform reward design in other embodied domains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper extends the R1-Zero/RLVR recipe to embodied robotics: Qwen2.5 LLMs (1.5B and 3B) are SFT-tuned and then trained with GRPO in a closed loop with an F1TENTH-simulated 1:10 autonomous car, where the LLM must adapt MPC parameters to satisfy natural-language behavioral prompts (centerline tracking, target velocity, reversing, smooth driving). A second module, DecisionxR1, is trained on static binary adherence labels. Evaluation is conducted with rephrased prompts on an unseen, more complex track (The Grand Tour), and a 'control adaptability' improvement over the default MPC is reported. The headline results are that Qwen2.5-3B with SFT+RLVR reaches 63.3% versus GPT-4o's 58.5% on the aggregate 'Improve [%]' score, and that Qwen2.5-1.5B improves from 11.3% (SFT-only) to 41.5% (SFT+RLVR). The paper also reports quantization on a Jetson Orin AGX and a physical-robot demonstration of compound prompt following.

Significance. If the central comparison were robust, this would be a valuable contribution: it shows that small, edge-deployable LLMs can learn control-adaptation behavior through closed-loop RL on a single consumer GPU, without distilling from a large model. The experimental design has real strengths: training and evaluation on different tracks, randomized rephrased prompts, a controlled ablation across base/SFT/RLVR conditions, released code, training curves in Appendix A, and a physical deployment experiment. However, the headline '63.3% vs 58.5%' comparison is currently neither checkable nor robust: the aggregation rule behind the 'Improve [%]' column is undefined, no variance is reported, a defensible alternative aggregation reverses the ranking, and the evaluation metric is essentially the same objective as the training reward. These issues are fixable in revision, but the central claim should not be accepted as stated.

major comments (4)
  1. [Section 4.3, Table 2] The 'Improve [%]' column that carries the paper's headline comparison is never defined. The reported values are consistent with an unweighted mean of the four per-metric percentage improvements (e.g., GPT-4o: (5.0+93.2+97.8+38.0)/4 = 58.5; Qwen2.5-3B SFT+RLVR: (39.9+90.2+91.2+31.8)/4 = 63.3), but no formula, weighting, or justification is given in the text, the table caption, or Appendix C. The choice of aggregation is load-bearing: on three of the four metrics (EV, ER, ES) GPT-4o is better than Qwen2.5-3B SFT+RLVR (0.14 vs 0.19 m/s, 0.12 vs 0.48 m/s, 1.10 vs 1.21 m/s^2), and the 3B model wins only on EC (0.41 vs 0.65 m). An equally simple aggregation, the sum of raw RMSEs relative to the default sum (0.68+1.98+5.44+1.77 = 9.87), yields 79.6% for GPT-4o (sum 2.01) versus 76.8% for Qwen2.5-3B (sum 2.29), reversing the reported ordering. Because the claim that a small LLM surpasses GPT-4o rests entirely on this column, the paper must state the aggregation rule explicitly, justify it (the metrics have different units and baselines; the ER improvement is nearly trivially achieved by setting negative velocity bounds, as shown in Appendix D), and demonstrate that the conclusion is stable across reasonable aggregation choices.
  2. [Section 4.3] No variability information is reported for the control adaptability scores. The caption of Table 2 states that each entry is the mean over five independent runs using different prompts in B', but the per-prompt metric values, standard deviations, and confidence intervals are absent, as is any significance test. With only five evaluation prompts, the headline 4.8-point gap between 63.3% and 58.5% may be within run-to-run noise. The authors should provide the per-prompt breakdown (or at least the per-metric dispersion), add more evaluation prompts or temper the superiority claim, and apply a consistent policy for the extraction-failure rows: the Qwen2.5-3B SFT-only row (one extraction failure, Improve 50.4%†) is not directly comparable to rows with zero failures, so the reported 12.9-point gain over that baseline should be recomputed with a stated handling of failures.
  3. [Section 3.3, Eq. (3)] The evaluation metric substantially overlaps with the training reward. Rdrive in Eq. (3) is the relative RMSE improvement (EMPC_B - ELLM_B)/EMPC_B over the default MPC, and the 'Improve [%]' column in Table 2 is an aggregate of exactly this kind of relative improvement against the same default MPC, with baselines re-measured on the evaluation map. The SFT+RLVR models are therefore evaluated on the objective they were trained to maximize, whereas GPT-4o was never trained on this objective; the ranking thus partly reflects optimization of the authors' chosen metric rather than general control adaptability. The train/evaluation track separation and the rephrased prompts mitigate this concern but do not remove it. As a concrete test, I ask the authors to report at least one evaluation that is not derived from Rdrive (for example, a fixed-weight composite cost chosen before training, or judged adherence of the resulting driving trajectories), or to show that the Qwen-versus-GPT-4o ranking is stable under alternative metric definitions.
  4. [Abstract, Section 4.3, Section 5] The claimed 20.2%-point improvement for Qwen2.5-1.5B over its SFT-only baseline contradicts the numbers in Table 2. The text states 'SFT: 11.3% / SFT+RLVR: 41.5%' and attributes to this a 20.2%-point improvement, but 41.5 - 11.3 = 30.2 points. The same 20.2 value appears in the abstract, in contribution (iii), and in the conclusion, so this is not a localized typo; the arithmetic should be corrected and all affected claims revised.
minor comments (5)
  1. [Various] Typos and wording issues should be fixed: 'succesfully' (Section 4.4), 'improvment' (Section 4.3), 'Adjsut' (Appendix C), 'trackign' (Appendix D), and 'qual contribution' (author footnote).
  2. [Abstract and Section 5] The abstract and conclusion should state that the 63.3% score refers to the full-precision model: the deployed Q5-quantized 3B model achieves 47.9% (Table 5), which is below the GPT-4o reference of 58.5%. As written, the claim that on-board deployment can outperform larger models overstates what the deployed configuration actually demonstrates.
  3. [Table 2 caption] The caption should give the definition of the 'Improve [%]' aggregation in one sentence rather than requiring the reader to infer the rule from the numbers.
  4. [Section 4.1] Clarify which model configuration requires approximately 11 GB of VRAM for GRPO training, and give the batch size and sequence settings used.
  5. [Appendix B] The map figure would benefit from a scale bar and from explicit labels connecting the Circle map to the training rows and The Grand Tour map to Table 2.

Circularity Check

1 steps flagged · score 5.0 of 10

The control adaptability score used to rank models is the same relative-RMSE improvement that defines the training reward Rdrive, so the headline comparison is partially circular; however, held-out prompts and an unseen track preserve some independent content.

  1. self definitional [Section 3.3, Eq. (3); Section 4.3, Table 2 caption]
    "RMPCxR1 = Rdrive + Rfmt + Rparam, R drive = max(EMPC B − ELLM B / EMPC B, −4) ... Percentage improvements are computed relative to the baseline MPC, which tracks the racing line. The improvement column aggregates overall performance across all metrics."

    The evaluation metric reported as 'Improve [%]' in Table 2 is the same relative RMSE improvement (EMPC_B - ELLM_B)/EMPC_B that defines the training reward Rdrive in Eq. (3), aggregated over the same four behavioral metrics (EC, EV, ER, ES) used during RLVR training. A model trained to maximize Rdrive is therefore being scored by the very objective it was optimized against, so the 63.3% control adaptability score is partly a measure of reward optimization rather than an independent capability benchmark. The comparison retains some independent content because GPT-4o was not trained on this objective and evaluation uses rephrased prompts on an unseen track, but the central ranking claim is substantially a report of how well each model optimized the authors' chosen reward.

full rationale

The paper's central quantitative claim is that Qwen2.5-3B trained with SFT and closed-loop RLVR achieves a 63.3% control adaptability score, surpassing GPT-4o's 58.5%. The load-bearing metric, however, is defined by the same formula as the training reward: Rdrive in Eq. (3) is the relative RMSE improvement over the default MPC, and Table 2's percentage improvements are computed relative to the same baseline MPC with the same four error components. Thus the evaluation score is not independent of the training objective; it is the training objective, evaluated on rephrased prompts and a different map. This is a real circularity, but it is partial rather than total: the rephrased behavioral set B' and the unseen Grand Tour map do test generalization, and GPT-4o was never trained on this reward, so the comparison to GPT-4o is not vacuous. A separate concern, not itself circularity, is that the paper never gives the aggregation formula for the Improve column; the numbers imply an unweighted mean of the four percentage improvements, and different aggregations can reorder the models. There is also an arithmetic inconsistency in the claimed 20.2-point SFT-to-SFT+RLVR gain for Qwen2.5-1.5B: Table 2 shows 11.3% to 41.5%, a 30.2-point difference. The paper also cites the authors' prior framework [32] for SFT, RAG, and the MPC baseline, but that prior work is a separate, code-backed system and is not used to forbid alternatives, so self-citation is not a significant circularity contributor here. Overall, the reduction of the evaluation metric to the training reward is the main circular step, giving a score of 5 rather than a higher score because of the held-out evaluation and untrained comparison model.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on the default MPC as a baseline, on simulation fidelity, on hand-picked behavior definitions, and on the choice of relative RMSE improvement as the reward. The aggregate evaluation metric is not defined in the text, which prevents full accounting of the headline score. No new physical entities are introduced.

free parameters (3)
  • Rdrive clipping lower bound = -4
    Equation (3) clips the driving reward to the range [1, -4]. This bound is chosen by hand and shapes the RL training signal.
  • GRPO training steps = 750
    Section 4.1 states GRPO training was performed for 750 steps for both models; this schedule is a training choice rather than a quantity derived from data.
  • Evaluation improvement aggregation rule = unspecified
    Table 2's "Improve [%]" column aggregates EC, EV, ER, and ES percentage improvements, but no formula is given. The central claim depends on this hand-defined summary.
assumptions (4)
  • domain assumption The default MPC, which tracks the minimum-curvature racing line, is an appropriate baseline, and relative improvement over it is a complete measure of control adaptability.
    Used in Equation (3) for Rdrive and in Table 2 for the evaluation improvement percentages. If this baseline is not the right reference, the 63.3% score loses meaning.
  • domain assumption The F1TENTH simulator and the kinematic MPC faithfully represent the real robot's dynamics well enough for behaviors learned in simulation to transfer to deployment.
    RL training and the closed-loop reward rely entirely on simulator feedback (Section 3.3, Figure 1). The physical deployment in Section 4.4 provides partial but limited evidence for this assumption.
  • domain assumption Training on the simple circle map with a small behavioral set transfers to the complex Grand Tour map and to rephrased, unseen prompts.
    Evaluation is deliberately separated from training (Section 4.3, Appendices B and C). If transfer fails, the reported evaluation numbers would not reflect the training objective.
  • domain assumption The programmatic rules B and the RMSE formulations EC, EV, ER, and ES correctly capture the human-prompted behavior requested in each prompt.
    DecisionxR1 labels are computed programmatically (Section 3.2), and all control adaptability rewards and metrics are computed from these hand-defined RMSE definitions (Section 3.3, Appendix C).

how reviews work

0 comments
Cite this review

Pith. "Pith review of RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning." pith.science (2026). https://pith.science/paper/T5VLBTEN

@misc{pith2026250503238,
  author       = {Pith},
  title        = {Pith review of: RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T5VLBTEN}},
  note         = {Machine review of arXiv:2505.03238}
}
read the original abstract

Future robotic systems operating in real-world environments will require on-board embodied intelligence without continuous cloud connection, balancing capabilities with constraints on computational power and memory. This work presents an extension of the R1-zero approach, which enables the usage of low parameter-count Large Language Models (LLMs) in the robotic domain. The R1-Zero approach was originally developed to enable mathematical reasoning in LLMs using static datasets. We extend it to the robotics domain through integration in a closed-loop Reinforcement Learning (RL) framework. This extension enhances reasoning in Embodied Artificial Intelligence (Embodied AI) settings without relying solely on distillation of large models through Supervised Fine-Tuning (SFT). We show that small-scale LLMs can achieve effective reasoning performance by learning through closed-loop interaction with their environment, which enables tasks that previously required significantly larger models. In an autonomous driving setting, a performance gain of 20.2%-points over the SFT-based baseline is observed with a Qwen2.5-1.5B model. Using the proposed training procedure, Qwen2.5-3B achieves a 63.3% control adaptability score, surpassing the 58.5% obtained by the much larger, cloud-bound GPT-4o. These results highlight that practical, on-board deployment of small LLMs is not only feasible but can outperform larger models if trained through environmental feedback, underscoring the importance of an interactive learning framework for robotic Embodied AI, one grounded in practical experience rather than static supervision.

Figures

Figures reproduced from arXiv: 2505.03238 by the authors.

Figure 1
Figure 1. Overview of the proposed Embodied AI agent for autonomous driving. The agent consists of a DecisionxR1 module and an MPCxR1 module that work in tandem to achieve a user-specific driving behavior. The LLMs at the core of each module are trained with SFT and RL. During RL training, the system interacts with its simulation environment, where it is rewarded for behavioral adherence. The trained agent is deployed on a sc… view at source ↗
Figure 2
Figure 2. The DecisionxR1 module consists of a RAG enhanced LLM that uses robot state information to de￾termine if the car is adhering to the desired behavior prompted by the human. Each answer of the LLM is as￾signed a format and correctness reward that are used to train the model with GRPO. The objective of the DecisionxR1 mod￾ule, depicted in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Schematic overview of the proposed MPCxR1 training procedure. The LLM leverages RAG context, following [32], to generate MPC parameters tailored to the desired driving behavior B. The LLM and MPC operate in a closed-loop with a simulator, which computes both the baseline Root Mean Square Error (RMSE) under default MPC parameters (EMPC B ) and the RMSE resulting from the LLM-generated parameters (ELLM B ), both relat… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Adaptation of robot behavior in re￾sponse to user prompts during embedded deploy￾ment with the proposed RLVR and SFT trained and Q5 k m quantized Qwen2.5-3B. In this experiment, the proposed system is de￾ployed on a physical 1:10 scaled open-source autonomous racing ca…
Figure 5
Figure 5. Figure 5: Visualization of MPCxR1 RLVR (GRPO) training with the standard deviation shaded. Qwen2.5 3B and 1.5B LLMs, both pretrained via SFT, are used as base models. Left: The reward signal RMPCxR1 that the LLM learns to maximize through interaction with the simulation environ￾…
Figure 6
Figure 6. Figure 6: DecisionxR1 RLVR (GRPO) training curves. Left: Reward RDecisionxR1 learned from static binary adherence classification. Right: Output token length over the training steps. B MPCxR1 Training and Evaluation Map The maps used in the proposed MPCxR1 RLVR training and evalu…
Figure 7
Figure 7. Figure 7: Tracks used for training and evaluation. Left: [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 22 canonical work pages

  1. [1]

    Caesar, V

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom. Nuscenes: A Multimodal Dataset for Autonomous Driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020

    work page 2020

  2. [2]

    P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V . Patnaik, P. Tsui, J. Guo, Y . Zhou, Y . Chai, B. Caine, et al. Scalability in Perception for Autonomous Driving: Waymo Open Dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2446–2454, 2020

    work page 2020

  3. [3]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open X-Embodiment: Robotic Learning Datasets and Rt-X Models: Open X-Embodiment Collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024

    work page 2024

  4. [4]

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zit- nick. Microsoft COCO: Common Objects in Context. In D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, editors, Computer Vision – ECCV 2014, pages 740–755, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10602-1

    work page 2014

  5. [5]

    Schuhmann, R

    C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki. LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs, 2021. URL https://arxiv.org/abs/2111.02114

    arXiv 2021

  6. [6]

    Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Y . Qiao, and J. Dai. BEV- Former: Learning Bird’s-Eye-View Representation From Multi-Camera Images Via Spatiotemporal Transformers. In Computer Vision – ECCV 2022: 17th Eu- ropean Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part IX , page 1–18, Berlin, Heidelberg, 2022. Springer...

    work page doi:10.1007/978-3-031-20077-9_ 2022

  7. [7]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. OpenVLA: An Open-Source Vision-Language-Action Model. In P. Agrawal, O. Kroemer, and W. Burgard, editors, Proceedings of The 8th Conference on Robot Le...

    work page 2025

  8. [8]

    Ghosh, H

    Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreiman, Y . Tan, L. Y . Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine. Octo: An Open-Source Generalist Robot Policy. In Proceedings of Robotics: Science and Systems, Delft, Netherlands, 2024

    work page 2024

Show all 44 references
  1. [9]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. Joshi, R. Ju- lian, D. Kalashnikov, Y . Kuang, I. Leal, K.-H. Lee, S. Levine, Y . Lu, U. Malla, D. ...

    2023

  2. [10]

    Beyer, A

    L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang, D. Salz, M. Neumann, I. Alabdul- mohsin, M. Tschannen, E. Bugliarello, T. Unterthiner, D. Keysers, S. Koppula, F. Liu, A. Gryc- ner, A. A. Gritsenko, N. Houlsby, M. Kumar, K. Rong, J. Eisenschlos, R. Kabra, M. Bauer, M...

    2024 doi arXiv

  3. [11]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning Transferable Visual Mod- els From Natural Language Supervision. In M. Meila and T. Zhang, editors, Proceedings of the 38th Inter...

    2021

  4. [12]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, qiang liu, Y . Zhu, and P. Stone. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2023. URL https:// openreview.net/for...

    2023

  5. [13]

    Zhang, Z

    S. Zhang, Z. Xu, P. Liu, X. Yu, Y . Li, Q. Gao, Z. Fei, Z. Yin, Z. Wu, Y .-G. Jiang, and X. Qiu. VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Ma- nipulation With Long-Horizon Reasoning Tasks, 2024. URLhttps://arxiv.org/abs/ 2412.18194

    2024 arXiv

  6. [14]

    L. Wen, D. Fu, X. Li, X. Cai, T. Ma, P. Cai, M. Dou, B. Shi, L. He, and Y . Qiao. Dilu: A Knowledge-Driven Approach to Autonomous Driving With Large Language Models. arXiv preprint arXiv:2309.16292, 2023

    2023 arXiv

  7. [15]

    M. Pavone. Decision Making and Control With LLMs. Lecture presented at NVIDIA GTC 2024, 2024. URL https://www.nvidia.com/en-us/on-demand/session/ gtc24-s62855/. Accessed: 2024-07-02

    2024

  8. [16]

    J. Duan, W. Yuan, W. Pumacay, Y . R. Wang, K. Ehsani, D. Fox, and R. Krishna. Manipulate- Anything: Automating Real-World Robots Using Vision-Language Models. In P. Agrawal, O. Kroemer, and W. Burgard, editors, Proceedings of The 8th Conference on Robot Learning, volume 270 of...

    2025

  9. [17]

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakr- ishnan, K. Hausman, et al. Do as I Can, Not as I Say: Grounding Language in Robotic Affordances. arXiv preprint arXiv:2204.01691, 2022. 10

    2022 arXiv

  10. [18]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  11. [19]

    B. Zhao, Z. Wang, J. Fang, C. Gao, F. Man, J. Cui, X. Wang, X. Chen, Y . Li, and W. Zhu. Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foun- dation Models via Reinforcement Learning. arXiv preprint arXiv:2504.12680, 2025

    2025 arXiv

  12. [20]

    Cobbe, V

    K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  13. [21]

    Petrov, J

    I. Petrov, J. Dekoninck, L. Baltadzhiev, M. Drencheva, K. Minchev, M. Balunovi ´c, N. Jo- vanovi´c, and M. Vechev. Proof or Bluff? Evaluating LLMS on 2025 Usa Math Olympiad. arXiv preprint arXiv:2503.21934, 2025

    2025 arXiv

  14. [22]

    Y . Yan, Y . Lu, R. Xu, and Z. Lan. Do PhD-level LLMs Truly Grasp Elementary Addi- tion? Probing Rule Learning vs. Memorization in Large Language Models. arXiv preprint arXiv:2504.05262, 2025

    2025

  15. [23]

    D. Han, M. Han, and U. team. Unsloth. https://github.com/unslothai/unsloth,

  16. [24]

    J. Pan, J. Zhang, X. Wang, L. Yuan, H. Peng, and A. Suhr. TinyZero. https://github.com/Jiayi- Pan/TinyZero, 2025. Accessed: 2025-01-24

    2025

  17. [25]

    T. P. Swaminathan, C. Silver, and T. Akilan. Benchmarking Deep Learning Models on NVIDIA Jetson Nano for Real-Time Systems: An Empirical Investigation. arXiv preprint arXiv:2406.17749, 2024

    2024 arXiv

  18. [26]

    Chen and X

    J. Chen and X. Ran. Deep Learning With Edge Computing: A Review. Proceedings of the IEEE, 107(8):1655–1674, 2019. doi:10.1109/JPROC.2019.2921977

    2019

  19. [27]

    Z. Lu, X. Li, D. Cai, R. Yi, F. Liu, X. Zhang, N. D. Lane, and M. Xu. Small Language Models: Survey, Measurements, and Insights. arXiv preprint arXiv:2409.15790, 2024

    2024 arXiv

  20. [28]

    Silver and R

    D. Silver and R. S. Sutton. Welcome to the Era of Experience. DeepMind, 2025

    2025

  21. [29]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, et al. Deepseekmath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  22. [30]

    Rudin, D

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to Walk in Minutes Using Mas- sively Parallel Deep Reinforcement Learning. In A. Faust, D. Hsu, and G. Neumann, ed- itors, Proceedings of the 5th Conference on Robot Learning , volume 164 of Proceedings of Machine Learnin...

    2022

  23. [31]

    P. R. Wurman, S. Barrett, K. Kawamoto, J. MacGlashan, K. Subramanian, T. J. Walsh, R. Capobianco, A. Devlic, F. Eckert, F. Fuchs, et al. Outracing Champion Gran Turismo Drivers With Deep Reinforcement Learning. Nature, 602(7896):223–228, 2022

    2022

  24. [32]

    Baumann, C

    N. Baumann, C. Hu, P. Sivasothilingam, H. Qin, L. Xie, M. Magno, and L. Benini. Enhancing Autonomous Driving Systems With on-Board Deployed Large Language Models, 2025. URL https://arxiv.org/abs/2504.11514

    2025 arXiv

  25. [33]

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.-T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, et al. Language to Rewards for Robotic Skill Synthesis. arXiv preprint arXiv:2306.08647, 2023. 11

    2023 arXiv

  26. [34]

    Ismail, A

    S. Ismail, A. Arbues, R. Cotterell, R. Zurbr ¨ugg, and C. A. Alonso. NARRATE: Versatile Language Architecture for Optimal Control in Robotics. arXiv preprint arXiv:2403.10762 , 2024

    2024 arXiv

  27. [35]

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y . Zhu, L. Fan, and A. Anandkumar. Eureka: Human-Level Reward Design via Coding Large Language Models. arXiv preprint arXiv:2310.12931, 2023

    2023 arXiv

  28. [36]

    Dao and D

    A. Dao and D. B. Vu. AlphaMaze: Enhancing Large Language Models’ Spatial Intelligence via GRPO. arXiv preprint arXiv:2502.14669, 2025

    2025 arXiv

  29. [37]

    Azzolini, H

    A. Azzolini, H. Brandon, P. Chattopadhyay, H. Chen, J. Chu, Y . Cui, J. Diamond, Y . Ding, F. Ferroni, R. Govindaraju, et al. Cosmos-Reason1: From Physical Common Sense to Embod- ied Reasoning. arXiv preprint arXiv:2503.15558, 2025

    2025 arXiv

  30. [38]

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, et al. Qwen Technical Report. arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  31. [39]

    Baumann, E

    N. Baumann, E. Ghignone, J. K ¨uhne, N. Bastuck, J. Becker, N. Imholz, T. Kr¨anzlin, T. Y . Lim, M. L¨otscher, L. Schwarzenbach, et al. ForzaETH Race Stack—Scaled Autonomous Head-to- Head Racing on Fully Commercial Off-the-Shelf Hardware. Journal of Field Robotics, 2024

    2024

  32. [40]

    Y . Zhai, H. Bai, Z. Lin, J. Pan, S. Tong, Y . Zhou, A. Suhr, S. Xie, Y . LeCun, Y . Ma, and S. Levine. Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Rein- forcement Learning. In The Thirty-eighth Annual Conference on Neural Information Process- ing Sy...

    2024

  33. [41]

    A. Szot, M. Schwarzer, H. Agrawal, B. Mazoure, R. Metcalf, W. Talbott, N. Mackraz, R. D. Hjelm, and A. T. Toshev. Large Language Models as Generalizable Policies for Embodied Tasks. In The Twelfth International Conference on Learning Representations , 2024. URL https://openrev...

    2024

  34. [42]

    V . S. Babu and M. Behl. F1tenth. Dev-an Open-Source Ros Based F1/10 Autonomous Racing Simulator. In 2020 IEEE 16th International Conference on Automation Science and Engineer- ing (CASE), pages 1614–1620. IEEE, 2020

    2020

  35. [43]

    Gerganov and O.-S

    G. Gerganov and O.-S. Contributors. Llama.cpp. https://github.com/ggerganov/ llama.cpp, 2023. Accessed: April 2025. Appendix A Training Curves This section presents the RLVR training curves for both the MPCxR1 and DecisionxR1 modules, shown in Figure 5 and Figure 6, respective...

    2023

  36. [2023]

    Accessed: April 2025

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.