Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Action Flow Matching for Continual Robot Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Action flow matching transforms planned actions to speed up dynamics-model realignment; task success rises by 34.2 points.

desk verdict A novel and empirically promising action-transformation trick whose central intent-recovery story is undercut by its own training data (random action targets); the gains may be explainable as stochastic exploration. read the letter →

arxiv 2504.18471 v2 pith:W6HUQPUD submitted 2025-04-25 cs.RO cs.AI

Alejandro Murillo-Gonzalez , Lantao Liu This is my paper · ORCID
classification cs.ROcs.AI
keywords continualrobotlearningflowmatchingdynamicsmodelalignmentonlineadaptationmodel-basedreinforcementactiontransformationcounterfactualdatagenerationmobilenavigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a robot with an outdated or misaligned dynamics model should not explore by executing the actions that model recommends. Instead, it should transform those planned actions through a flow-matching generative model into 'intended actions'—the actions a well-aligned model would have chosen—and execute those. The method, called Action Flow Matching (AFM), is trained on counterfactual transitions generated under the initial model, pairing a randomly sampled intended action with a different planned action. In simulated tests, AFM raised a ground vehicle's task success rate by 34.2 percentage points over the best baseline and cut a quadrotor's trajectory tracking error by 6.6%, with the same trained AFM model applied across all evaluated UGV scenarios. The motivation is that a robot should not wait to stumble onto informative states through a model that is wrong; it should correct its actions now and learn from the informative transitions those corrections produce.

What carries the argument

The central object is Action Flow Matching (AFM), a flow-matching generative model $g_\phi$ that maps a planned action $a^0_t$ to a transformed action $a^1_t$ by integrating a learned velocity field over $\tau\in[0,1]$. The central identity is the conditional flow-matching loss $L_{\mathrm{CFM}}(\phi)=\mathbb{E}_{\tau,X_0,X_1}\big[\big\|u^\phi_\tau\big((1-\tau)X_0+\tau X_1\big)-\tfrac{X_1-X_0}{1-\tau}\big\|^2\big]$, which trains the flow without access to ground-truth target velocities. AFM conditions the flow on two encoders: $E_{Z_D}$ encodes the dynamics regime from the state, the planned action, and the prediction error, while $E_{Z_T}$ encodes the action transformation over time. Training data come from counterfactual transitions generated by the initial model: the next state is computed under a randomly sampled intended action $a^1_i$ but attributed to the planned action $a^0_i$, forming the dataset $\mathcal{E}$. At deployment the flow is integrated with an explicit midpoint ODE solver, and a misalignment flag $M=\mathbf{1}(\|e_t\|_2>0)$ decides when to invoke the transformation.

What would settle it

On the UGV task, collect the prediction-error vectors $e_t$ from deployment (Eq. 20) and compare their distribution with the training errors under the initial model (Eq. 19); if AFM still outperforms the online baseline in scenarios where the two distributions barely overlap, the improvement cannot be attributed to the flow transformation, and the mechanism stated in Eq. (14) would not be supported.

Watch

Extended reading notes

Core claim

AFM's central claim is that the data a robot collects while learning are more informative when planned actions are transformed than when the misaligned model is used directly for exploration. Formally, the paper defines the planned next state $s^*_{t+1}=f_{\theta_t}(s_t,a^0_t)$ and the realized next state $s_{t+1}=f(s_t,a^0_t)$, and seeks a transformed action $a^1_t$ satisfying $\|f(s_t,a^1_t)-s^*_{t+1}\|_2 \leq \|f(s_t,a^0_t)-s^*_{t+1}\|_2$. Because the true dynamics $f$ are unknown, AFM never minimizes this deviation; instead it trains a velocity field $g_\phi$ with the conditional flow-matching loss on a dataset $\mathcal{E}=\{(s_i,a^0_i,f_{\theta_0}(s_i,a^1_i))\}$ in which the next state is computed under a randomly sampled intended action $a^1_i$ but recorded as if caused by the planned action $a^0_i$. At deployment, an encoder $E_{Z_D}$ reads the current dynamics regime from the state, planned action, and prediction error, and the flow model converts $a^0_t$ into $a^1_t$. The paper reports that this conversion accelerates online model alignment and improves task performance on a Jackal UGV and a Crazyflie quadrotor.

Load-bearing premise

The method assumes that training on made-up pairs of planned and intended actions generated randomly under the initial model teaches corrections that transfer to the real, unseen dynamics changes the robot meets at deployment, even though the training loss never directly targets the deployment deviation.

Editorial extensions

If this is right

  • A robot can keep its existing dynamics model—data-driven, physics-based, or hybrid—and still adapt to new dynamics regimes, because AFM only modifies the planned action and leaves the model's update loop unchanged.
  • Executing AFM-transformed actions collects more informative transition data, which the UGV loss curves show converge faster and recover sooner after a regime change than data from the online model's own plans.
  • When the prediction-error threshold $\delta_M$ detects a regime shift, the robot toggles from exploiting its current model to executing transformed actions until alignment is restored, providing a principled exploration-exploitation switch during non-episodic operation.
  • The same AFM weights trained once on counterfactual data generalize across all evaluated UGV shift scenarios, and perturbing the initial model's training actions by 10% (AFM+DR) adds a further 2.2 percentage points of success.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because AFM is trained entirely under the initial model, the same recipe could be applied to sim-to-real transfer: train the flow on a simulation model, then let the flow correct plans on the real robot before any real-world dynamics data are collected.
  • The dynamics-regime encoder $E_{Z_D}$ could be reused as a standalone monitoring signal for detecting latent changes such as actuator wear or surface friction, since it must learn to represent prediction-error patterns even when no action transformation is used.
  • A controlled test of the mechanism would compare AFM against an oracle that knows the true dynamics and always selects the action minimizing the deviation in Eq. (14); if random-target AFM approaches the oracle's data efficiency, the counterfactual curriculum is responsible, whereas a large gap would suggest the reported gains partly come from the online model updates themselves.
  • The method's data hunger is structural rather than incidental: flow matching needs dense coverage of action and error space, so the reported 34.2-point gain should be re-measured as the training-set size for $\mathcal{E}$ shrinks, to locate the practical data floor for deployment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes Action Flow Matching (AFM), a method for online continual learning of robot dynamics models. AFM learns a flow-matching model that transforms actions produced by a model-based planner under a misaligned dynamics model into 'intended' actions, with the stated goal of reducing the deviation between the planned next state and the realized next state (Eq. 14) and thereby collecting more informative transitions for faster model realignment. The training data are generated counterfactually from the initial model f_theta0: random planned and intended actions are sampled, and the next state is computed under the intended action but registered as if caused by the planned action (Eqs. 16-18). At deployment, a learned dynamics regime representation Z_D conditions the flow model, and actions are transformed when a misalignment flag exceeds a threshold. The method is evaluated in simulation on a Jackal UGV waypoint navigation task and a Crazyflie quadrotor tracking task, reporting a 34.2 percentage point higher average success rate than the best baseline on the UGV and a 6.6% lower tracking MSE on the quadrotor.

Significance. The idea of correcting actions rather than exploring with a misaligned model is timely and, if established, would be a useful complement to model-based continual learning. The paper's strengths include the public code release, the model-agnostic formulation, the absence of replay-buffer dependence at deployment, and the breadth of the evaluation across two very different platforms and many dynamics-shift scenarios. However, the central mechanism is not connected to the training objective, and the deployment-time error distribution is not shown to match the training-time one; the empirical claims are also less uniform than the text suggests. The contribution is potentially valuable but the core claim needs substantial additional support.

major comments (4)
  1. [IV-B, Eq. (8) and Eq. (14)] The paper's central claim is that AFM reduces the deviation in Eq. (14), but this quantity is never optimized. The dataset E in Eq. (18) is built by sampling a^0 and a^1 independently from A, and the conditional flow matching loss in Eq. (8) only trains the velocity field to reproduce the random target a^1 from the source a^0. Since a^1 is not chosen to make ||f(s,a^1)-s*|| smaller than ||f(s,a^0)-s*||, and no term in L_CFM involves f, the training objective provides no reason for the transformed action to satisfy the inequality in Eq. (14). The observed improvements could therefore be explained by the stochastic perturbation induced by AFM rather than by the 'intent mapping' mechanism described in Section IV.B.
  2. [IV-B, Eqs. (19)-(20)] The training and deployment conditioning distributions do not match. During dataset generation, the conditioning error is e_t = f_theta0(s,a^1) - f_theta0(s,a^0) (Eq. 19), computed entirely under the initial model; at deployment, e_t = s_{t+1} - f_theta_t(s,a^0) (Eq. 20), where s_{t+1} comes from the true environment dynamics and f_theta_t is continuously updated. The flow model is trained to invert the action-effect relation of f_theta0, but at deployment it is asked to compensate errors generated by a different, unseen process. The paper does not provide an argument or experiment showing that deployment-time errors lie in the support of the training-time errors; without this, the AFM correction at deployment is an extrapolation.
  3. [IV-B, Eqs. (16)-(18)] The counterfactual data generation mislabels transitions in a way that may be inconsistent with the intended physical interpretation. Eq. (16) computes s_{t+1}=f_theta0(s,a^1), and Eq. (17) registers this next state as if it followed from a^0, although a^0 and a^1 are independent samples. The resulting triple (s, a^0, f_theta0(s,a^1)) need not correspond to any achievable transition under a single dynamics model, so the dataset does not obviously simulate 'dynamics model misalignments' as claimed in Section IV.B. This also makes the learned 'dynamics regime representation Z_D' dependent on self-generated artifacts of f_theta0, and the paper provides no separate validation that Z_D tracks the actual regime changes used in Section V.
  4. [V-B, Tables I-II and V-D, Tables III-IV] The text in Section V.B that AFM 'consistently outperforms' all baselines is not supported by the per-scenario results. In Map 1, for (v_gain, omega_gain) = (2.50, 0.05), Physics achieves success 0.32 vs AFM 0.18; for (1.00, -1.00), Physics succeeds in 827 steps on average vs AFM's 1825; for (1.00, -0.50), Physics has 1.00 success vs AFM's 0.71; and for (0.10, -1.50), Physics has 0.74 vs AFM's 0.23. The aggregate 81.4% average and the headline 34.2% improvement therefore hide substantial scenario-level variability. The same issue appears in the ablations: in Table III, PE succeeds with fewer steps than AFM in scenarios (1.00, -1.00) and (2.00, 2.00), and has higher success in (0.10, -1.50). The claims should be stated per-scenario or supported by a statistical test across the full set.
minor comments (5)
  1. [Eq. (22)] The threshold condition in Eq. (22) is written with a strict inequality, leaving the case ||s_real - s_pred||_2 = delta_M undefined; also the flag M = 1(||e_t||_2 > 0) is redundant after e_t is set to zero below the threshold.
  2. [Eqs. (16)-(17)] The notation in Eqs. (16)-(17) uses s_{t+1} for two different quantities (the state under a^1 and the registered state), which is confusing; different symbols should be used for the two quantities.
  3. [Section IV.B] The qualitative claim that AFM balances exploration and exploitation is not measured; no metric of informativeness or model alignment is reported beyond task success, steps, and loss curves.
  4. [Figure 7] The quadrotor improvement of 6.6% is reported only as a mean over five experiments in Fig. 7; include per-speed breakdown and confidence intervals, since the headline improvement is small.
  5. [Abstract] The abstract's 'record 34.2% higher task success rate' should be qualified as an average over simulated UGV scenarios, given the per-scenario variability documented in Tables I-II.

Circularity Check

1 steps flagged · score 2.0 of 10

No load-bearing circularity: the reported gains are external benchmark results, but the paper labels uniformly random training targets as 'intended actions,' making the intent-mapping claim definitional rather than derived.

  1. self definitional [Section IV-B, Eqs. (16)-(18) and the training-data description.]
    "given an initial state s_t∼S, a planned action a^0_t∼A, and an intended action a^1_t∼A, we compute the next state s_{t+1} under the intended action: s_{t+1}=f_{θ0}(s_t,a^1_t), but register the transition as if it resulted from the planned action: s_{t+1}∼f_{θ0}(s_t,a^0_t). ... E := {(s_i,a^0_i,f_{θ0}(s_i,a^1_i)) : s_i∼S, a^0_i,a^1_i∼A}_{i=0}^N"

    The paper defines an 'intended action' as one that would have been chosen under an aligned dynamics model, but the training target a^1 in Eq. (18) is sampled uniformly from the action space A. The conditional flow-matching loss of Eq. (8) trains g_phi to map planned actions a^0 to these random a^1 values given an error computed from f_theta0. Consequently, the model's output is, by construction, a sample from the random-target conditional distribution; calling it an 'intent-corrected' action is a label attached in Eq. (18), not a quantity derived from Eq. (14) or from any aligned-model data.

full rationale

The paper's central performance claims—34.2 percentage points higher success rate on the UGV and 6.6% lower tracking error on the quadrotor—are established by simulation experiments against external baselines (stream-x PE, Online-KNODE-MPC, and physics-based models), not by a formal derivation. The AFM training objective (Eq. 8) is a standard conditional flow-matching loss, and the training data (Eq. 18) are generated from the initial model f_theta0. No term in the loss is equal to the reported task success rate or tracking error, so the empirical results are not forced by construction. The one genuinely circular element is terminological: a^1_t in Eq. (18) is drawn uniformly from A and called an 'intended action,' despite the paper's own definition of intent as the action chosen under a well-aligned model. The flow model therefore learns a random-target conditional mapping; labeling its output as 'intent' is a definitional choice, not a recovered quantity. Because the reported improvements are externally measured and the method is evaluated against multiple baselines and ablations, this naming issue is not load-bearing for the empirical outcome, though it does inflate the interpretation of what the flow model has learned.

Assumptions & free parameters 2 free parameters · 3 assumptions · 2 invented entities

The method rests on two self-generated constructs: the random 'intended action' target and the latent dynamics-regime encoder. The only external inputs are the initial dynamics model f_theta_0 (trained on Dubins data for the UGV) and the online transitions used to update f_theta. The threshold delta_M and training budgets are hand-chosen. The core learning signal is internally generated, so the method is self-contained but also self-referential.

free parameters (2)
  • Misalignment threshold delta_M = 1 (UGV experiments)
    Predefined threshold in Eq. (22) that decides when to apply AFM; its value affects how often transformed actions are used and is not tuned or justified beyond a single setting.
  • AFM training budget = 75,000 iterations (UGV), 15M samples (quadrotor)
    Data-intensive training budget for the flow model; chosen by hand, not justified by convergence analysis.
assumptions (3)
  • standard math Flow matching with conditional optimal transport (Eqs. 6-8) yields a valid velocity field for the action transformation.
    The paper relies on the flow matching framework of Lipman et al., including the gradient equivalence del_phi L_FM = del_phi L_CFM (Eq. 9).
  • domain assumption The error signal at deployment (Eq. 20) is distributionally comparable to the training error generated with the initial model f_theta_0 (Eq. 19).
    The flow model is trained on e = f_theta_0(s,a^1) - f_theta_0(s,a^0) but used with e = s_real - f_theta_t(s,a^0) under the updated model and true environment; if these distributions differ, the transformed actions will not be appropriate.
  • domain assumption The planner's cost function (goal, waypoints) is not part of AFM training; the transformed action is sampled without explicit goal-conditioning, yet the robot is expected to still complete goal-directed tasks.
    In Section IV-B, the dataset E samples actions from A without conditioning on the goal G; the empirical results assume that informative exploratory actions remain compatible with waypoint completion.
invented entities (2)
  • Intended action a^1 (the action a well-aligned model would choose)
    purpose: Target distribution for AFM; the paper claims AFM maps planned actions to these intended actions.
    In the training dataset E (Eq. 18), a^1 is sampled uniformly at random from A. No well-aligned model or external definition of intent is used, so the entity is defined by the method itself.
  • Dynamics regime representation Z_D
    purpose: Latent encoding of the current dynamics regime, used to condition the flow model and detect regime changes.
    Learned end-to-end from self-generated counterfactual data; the paper does not validate that the latent aligns with physically meaningful regimes beyond the simulator's gain parameters.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Action Flow Matching for Continual Robot Learning." pith.science (2026). https://pith.science/paper/W6HUQPUD

@misc{pith2026250418471,
  author       = {Pith},
  title        = {Pith review of: Action Flow Matching for Continual Robot Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W6HUQPUD}},
  note         = {Machine review of arXiv:2504.18471}
}
read the original abstract

Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks, mirroring human adaptability. A key challenge is refining dynamics models, essential for planning and control, while addressing issues such as safe adaptation, catastrophic forgetting, outlier management, data efficiency, and balancing exploration with exploitation -- all within task and onboard resource constraints. Towards this goal, we introduce a generative framework leveraging flow matching for online robot dynamics model alignment. Rather than executing actions based on a misaligned model, our approach refines planned actions to better match with those the robot would take if its model was well aligned. We find that by transforming the actions themselves rather than exploring with a misaligned model -- as is traditionally done -- the robot collects informative data more efficiently, thereby accelerating learning. Moreover, we validate that the method can handle an evolving and possibly imperfect model while reducing, if desired, the dependency on replay buffers or legacy model snapshots. We validate our approach using two platforms: an unmanned ground vehicle and a quadrotor. The results highlight the method's adaptability and efficiency, with a record 34.2\% higher task success rate, demonstrating its potential towards enabling continual robot learning. Code: https://github.com/AlejandroMllo/action_flow_matching.

Figures

Figures reproduced from arXiv: 2504.18471 by the authors.

Figure 1
Figure 1. Action Flow Matching for efficient continual learning. The planner uses the latest version of the dynamics model and the desired goal to select the next action a 0 t . AFM transforms a 0 t into a 1 t whenever the misalignment between the model and the environment is unacceptable. With a 1 t , the agent is expected to execute more informative actions leading to faster model re-alignment. model inaccuracies, thus incr… view at source ↗
Figure 2
Figure 2. AFM Pipeline. During training we leverage counterfactual-inspired transitions by simulating the state evolution with one randomly sampled action but registering a different random action as the cause. At deployment time, the agent tries to identify the current dynamics regime and based on that transforms the planned actions a 0 t to a corrected or intended action a 1 t that AFM considers has a higher chance of accom… view at source ↗
Figure 3
Figure 3. Feasible Dynam￾ics Space Intuition. Blue: State-Action space. White: Dynamically feasible re￾gion. Orange: Approxima￾tion of the feasible region using the dynamics model. To address Q1, one naive approach would involve randomly generating tuples (st, a 1 t , st+1) alongside corresponding planned actions a 0 t and using them to train the flow matching model. However, this would inefficiently cover the entire state-ac… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Platforms employed for validation of our method and comparison against the baselines in simulation. (Left) Jackal UGV [34]. (Right) Crazyflie quadrotor [2]. A. Baselines We are interested in comparing with methods that achieve dynamics model adaptation by learning from…
Figure 5
Figure 5. Figure 5: ETHZ Maps [25] where the online dynamics learning tasks took place. The UGV has to reach a series of sparse waypoints (blue dots) to complete the lap. At certain points the dynamics change and the robot needs to adapt to the new and unexpected world characteristics. • …
Figure 7
Figure 7. Figure 7: Quadrotor trajectory tracking error under evolv￾ing dynamics. Mean ± stan￾dard deviation over five exper￾iments. Results [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: UGV Lifelong Learning Loss Comparison. We present the results on Map 1 (left) and Map 2 (right). Vertical lines indicate the average step at which a change in the environmental dynamics occurred, thus presenting a new scenario in which the robot needs to learn to opera…
Figure 9
Figure 9. Figure 9: Trajectory heatmaps with continual dynamics learning in Map 1 (top) and Map 2 (bottom) under multiple interventions. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10 [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 36 canonical work pages

  1. [1]

    Courier Corporation, 2007

    Brian DO Anderson and John B Moore.Optimal control: linear quadratic methods. Courier Corporation, 2007

    work page 2007

  2. [2]

    Crazyflie simulation

    Bitcraze. Crazyflie simulation. https://github.com/ bitcraze/crazyflie-simulation, 2024. Accessed: 2024-12- 11

    work page 2024

  3. [3]

    The cross-entropy method for optimization

    Zdravko I Botev, Dirk P Kroese, Reuven Y Rubinstein, and Pierre L’Ecuyer. The cross-entropy method for optimization. InHandbook of statistics, volume 31, pages 35–59. Elsevier, 2013

    work page 2013

  4. [4]

    Prov- able lifelong learning of representations

    Xinyuan Cao, Weiyang Liu, and Santosh Vempala. Prov- able lifelong learning of representations. InInternational Conference on Artificial Intelligence and Statistics, pages 6334–6356. PMLR, 2022

    work page 2022

  5. [5]

    Knode-mpc: A knowledge-based data-driven predictive control framework for aerial robots.IEEE Robotics and Automation Letters, 7(2):2819–2826, 2022

    Kong Yao Chee, Tom Z Jiahao, and M Ani Hsieh. Knode-mpc: A knowledge-based data-driven predictive control framework for aerial robots.IEEE Robotics and Automation Letters, 7(2):2819–2826, 2022

    work page 2022

  6. [6]

    Adaptive neural trajectory tracking control for flexible-joint robots with online learning

    Shuyang Chen and John T Wen. Adaptive neural trajectory tracking control for flexible-joint robots with online learning. In2020 IEEE International Conference on Robotics and Automation (ICRA), pages 2358–2364. IEEE, 2020

    work page 2020

  7. [7]

    Deep reinforcement learning in a handful of trials using probabilistic dynamics models

    Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems, 31, 2018

    2018

  8. [8]

    Univer- sity of Houston, 1983

    Charles Ray Cutler.Dynamic matrix control: an optimal multivariable control algorithm with constraints. Univer- sity of Houston, 1983

    work page 1983

Show all 46 references
  1. [9]

    A continual learning survey: Defying forgetting in classification tasks.IEEE transactions on pattern analysis and machine intelligence, 44(7):3366– 3385, 2021

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale ˇs Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks.IEEE transactions on pattern analysis and machine intelligence, 44(7):3366– 3...

    2021

  2. [10]

    L. E. Dubins. On curves of minimal length with a con- straint on average curvature, and with prescribed initial and terminal positions and tangents.American Journal of Mathematics, 79(3):497–516, 1957. ISSN 00029327, 10806377. URL http://www.jstor.org/stable/2372560

    1957

  3. [11]

    Streaming deep reinforcement learning finally works.arXiv preprint arXiv:2410.14606, 2024

    Mohamed Elsayed, Gautham Vasan, and A Rupam Mah- mood. Streaming deep reinforcement learning finally works.arXiv preprint arXiv:2410.14606, 2024

    2024 arXiv

  4. [12]

    Robust model predictive path integral control: Analysis and perfor- mance guarantees.IEEE Robotics and Automation Let- ters, 6(2):1423–1430, 2021

    Manan S Gandhi, Bogdan Vlahov, Jason Gibson, Grady Williams, and Evangelos A Theodorou. Robust model predictive path integral control: Analysis and perfor- mance guarantees.IEEE Robotics and Automation Let- ters, 6(2):1423–1430, 2021

    2021

  5. [13]

    Pro- totyping design and learning in outdoor mobile robots operating in unstructured outdoor environments.IEEE international robotics and automation magazine, 8(3): 53–69, 2001

    Hani Hagras, Victor Callaghan, and Martin Colley. Pro- totyping design and learning in outdoor mobile robots operating in unstructured outdoor environments.IEEE international robotics and automation magazine, 8(3): 53–69, 2001

    2001

  6. [14]

    Online learning and adaptation of autonomous mobile robots for sustainable agriculture

    Hani Hagras, Martin Colley, Victor Callaghan, and Mal- colm Carr-West. Online learning and adaptation of autonomous mobile robots for sustainable agriculture. Autonomous Robots, 13:37–52, 2002

    2002

  7. [15]

    Td-mpc2: Scalable, robust world models for continuous control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control. arXiv preprint arXiv:2310.16828, 2023

    2023 arXiv

  8. [16]

    Online learning of humanoid robot kinematics under switching tools contexts

    Lorenzo Jamone, Bruno Damas, Jos ´e Santos-Victor, and Atsuo Takanishi. Online learning of humanoid robot kinematics under switching tools contexts. In2013 IEEE international conference on robotics and automation, pages 4811–4817. IEEE, 2013

    2013

  9. [17]

    Online dynamics learning for predictive control with an application to aerial robots

    Tom Z Jiahao, Kong Yao Chee, and M Ani Hsieh. Online dynamics learning for predictive control with an application to aerial robots. InConference on Robot Learning, pages 2251–2261. PMLR, 2023

    2023

  10. [18]

    Contributions to the theory of optimal control.Bol

    Rudolf Emil Kalman et al. Contributions to the theory of optimal control.Bol. soc. mat. mexicana, 5(2):102–119, 1960

    1960

  11. [19]

    Towards continual reinforcement learning: A review and perspectives.Journal of Artificial Intelli- gence Research, 75:1401–1476, 2022

    Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup. Towards continual reinforcement learning: A review and perspectives.Journal of Artificial Intelli- gence Research, 75:1401–1476, 2022

    2022

  12. [20]

    Rma: Rapid motor adaptation for legged robots

    Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021

    2021 arXiv

  13. [21]

    Adapting rapid motor adaptation for bipedal robots

    Ashish Kumar, Zhongyu Li, Jun Zeng, Deepak Pathak, Koushil Sreenath, and Jitendra Malik. Adapting rapid motor adaptation for bipedal robots. In2022 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS), pages 1161–1168. IEEE, 2022

    2022

  14. [22]

    Context-aware dynamics model for generalization in model-based reinforcement learning

    Kimin Lee, Younggyo Seo, Seunghyun Lee, Honglak Lee, and Jinwoo Shin. Context-aware dynamics model for generalization in model-based reinforcement learning. InInternational Conference on Machine Learning, pages 5757–5766. PMLR, 2020

    2020

  15. [23]

    Continual learning for robotics: Definition, framework, learning strategies, opportunities and chal- lenges.Information fusion, 58:52–68, 2020

    Timoth ´ee Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, and Natalia D ´ıaz- Rodr´ıguez. Continual learning for robotics: Definition, framework, learning strategies, opportunities and chal- lenges.Information fusion, 58:52–68, 2020

    2020

  16. [24]

    Rapid motor adaptation for robotic manipulator arms

    Yichao Liang, Kevin Ellis, and Jo ˜ao Henriques. Rapid motor adaptation for robotic manipulator arms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16404–16413, 2024

    2024

  17. [25]

    Optimization-based autonomous racing of 1:43 scale rc cars.Optimal Control Applications and Methods, 36(5):628–647, 2015

    Alexander Liniger, Alexander Domahidi, and Manfred Morari. Optimization-based autonomous racing of 1:43 scale rc cars.Optimal Control Applications and Methods, 36(5):628–647, 2015

    2015

  18. [26]

    Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q. Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code, 2024. URL https://arxiv.org/abs/2412. 06264

    2024

  19. [27]

    Towards robust online inverse dynamics learning

    Franziska Meier, Daniel Kappler, Nathan Ratliff, and Stefan Schaal. Towards robust online inverse dynamics learning. In2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4034–4039. IEEE, 2016

    2016

  20. [28]

    Ihab S Mohamed, Kai Yin, and Lantao Liu. Autonomous navigation of agvs in unknown cluttered environments: log-mppi control strategy.IEEE Robotics and Automa- Accepted to Robotics: Science and Systems (RSS) 2025 tion Letters, 7(4):10240–10247, 2022

    2025

  21. [29]

    Deep online learning via meta-learning: Continual adaptation for model-based rl.arXiv preprint arXiv:1812.07671, 2018

    Anusha Nagabandi, Chelsea Finn, and Sergey Levine. Deep online learning via meta-learning: Continual adaptation for model-based rl.arXiv preprint arXiv:1812.07671, 2018

    2018 arXiv

  22. [30]

    Incremental online sparsification for model learning in real-time robot con- trol.Neurocomputing, 74(11):1859–1867, 2011

    Duy Nguyen-Tuong and Jan Peters. Incremental online sparsification for model learning in real-time robot con- trol.Neurocomputing, 74(11):1859–1867, 2011

    2011

  23. [31]

    Online movement adaptation based on previous sensor experiences

    Peter Pastor, Ludovic Righetti, Mrinal Kalakrishnan, and Stefan Schaal. Online movement adaptation based on previous sensor experiences. In2011 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems, pages 365–371. IEEE, 2011

    2011

  24. [32]

    In-hand object rotation via rapid motor adaptation

    Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, and Jitendra Malik. In-hand object rotation via rapid motor adaptation. InConference on Robot Learning, pages 1722–1732. PMLR, 2023

    2023

  25. [33]

    Model predictive heuristic control.Automatica (journal of IFAC), 14(5):413–428, 1978

    Jacques Richalet, Andr ´e Rault, JL Testud, and J Papon. Model predictive heuristic control.Automatica (journal of IFAC), 14(5):413–428, 1978

    1978

  26. [34]

    Jackal ugv - small weatherproof robot - clearpath., 2025

    Clearpath Robotics. Jackal ugv - small weatherproof robot - clearpath., 2025. URL https://clearpathrobotics. com/jackal-small-unmanned-ground-vehicle/

    2025

  27. [35]

    Ella: An efficient lifelong learning algorithm

    Paul Ruvolo and Eric Eaton. Ella: An efficient lifelong learning algorithm. InInternational conference on ma- chine learning, pages 507–515. PMLR, 2013

    2013

  28. [36]

    John Wiley & Sons, 2016

    Dale E Seborg, Thomas F Edgar, Duncan A Mellichamp, and Francis J Doyle III.Process dynamics and control. John Wiley & Sons, 2016

    2016

  29. [37]

    Lifelong machine learning systems: Beyond learning algorithms

    Daniel L Silver, Qiang Yang, and Lianghao Li. Lifelong machine learning systems: Beyond learning algorithms. In2013 AAAI spring symposium series, 2013

    2013

  30. [38]

    Lifelong autonomous improvement of navigation foun- dation models in the wild

    Kyle Stachowicz, Lydia Ignatova, and Sergey Levine. Lifelong autonomous improvement of navigation foun- dation models in the wild. In8th Annual Conference on Robot Learning, 2024

    2024

  31. [39]

    Cambridge university press, 2003

    Endre S ¨uli and David F Mayers.An introduction to numerical analysis. Cambridge university press, 2003

    2003

  32. [40]

    Online learning of unknown dynamics for model- based controllers in legged locomotion.IEEE Robotics and Automation Letters, 6(4):8442–8449, 2021

    Yu Sun, Wyatt L Ubellacker, Wen-Loong Ma, Xiang Zhang, Changhao Wang, Noel V Csomay-Shanklin, Masayoshi Tomizuka, Koushil Sreenath, and Aaron D Ames. Online learning of unknown dynamics for model- based controllers in legged locomotion.IEEE Robotics and Automation Letters, 6(4...

    2021

  33. [41]

    MIT press, 2018

    Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction. MIT press, 2018

    2018

  34. [42]

    Error-based learning mechanism for fast online adaptation in robot motor control.IEEE transactions on neural networks and learning systems, 31(6):2042–2051, 2019

    Mathias Thor and Poramate Manoonpong. Error-based learning mechanism for fast online adaptation in robot motor control.IEEE transactions on neural networks and learning systems, 31(6):2042–2051, 2019

    2019

  35. [43]

    Lifelong learning algorithms

    Sebastian Thrun. Lifelong learning algorithms. In Learning to learn, pages 181–209. Springer, 1998. APPENDIX A. Streaming Model-Free Reinforcement Learning Streaming reinforcement learning methods aim to process data one sample at a time without storing previous samples, there...

    1998

  36. [44]

    •Eligibility Traces:Accumulating traces facilitate credit assignment over time

    Sample Efficiency:To enhance sample efficiency, the method employs sparse initialization and eligibility traces: •Sparse Initialization:Most weights are initialized to zero while retaining a sparse set of non-zero weights, reducing interference and improving learning robustnes...

  37. [45]

    An efficient step-size control mechanism mitigates computational overhead associated with traditional backtracking

    Update Stability:Step sizes are stabilized using a mod- ified backtracking line search that bounds updates, avoiding overshooting errors in single-sample learning. An efficient step-size control mechanism mitigates computational overhead associated with traditional backtracking

  38. [46]

    The normalization scale and bias parameters are not learned

    Normalization and Scaling:Layer normalization is ap- plied across network layers to maintain stable activation distri- butions under non-stationary data. The normalization scale and bias parameters are not learned. Additionally, observations and rewards are scaled dynamically ...

    2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.