Pith. sign in

REVIEW 2 major objections 4 minor 52 references

Sample-Efficient Online Control Policy Learning with Real-Time Recursive Model Updates

T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Recursive Koopman Learning: one recursive least squares update equals full EDMD retraining, enabling real-time sample-efficient control.

desk verdict The RLS-EDMD equivalence is old but the batch consistency theorem is new; the stress-test is right that the online loop is not covered by the Markov-chain SLLN, but the empirical pipeline is still worth a look. read the letter →

arxiv 2509.08241 v1 pith:EIMPAC2X submitted 2025-09-10 cs.RO cs.SYeess.SY

Zixin Zhang , James Avtges , Todd D. Murphey This is my paper · ORCID
classification cs.ROcs.SYeess.SY
keywords KoopmanoperatorrecursiveleastsquaresonlinemodelupdateextendeddynamicmodedecompositionMarkovchainconvergencesample-efficientcontrolpredictivesoftrobotics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Recursive Koopman Learning (RKL) claims that an online controller can update its learned dynamics model at every time step—at negligible computational cost—without ever losing the quality of a full batch refit. The central analytical result is that one recursive least squares update produces exactly the same Koopman matrix as retraining the entire extended dynamic mode decomposition model on all data collected so far, so the online model is always the globally optimal least-squares fit. Under specified ergodicity conditions on the data stream, the paper proves the estimated Koopman matrix converges almost surely to the optimal linear Koopman operator for the chosen observables. If true, this means a robot can learn a control policy while it is running, from a small initial dataset, and improve continuously in real time. The paper demonstrates the payoff on a simulated two-link arm and on a soft-actuated hardware platform, where RKL reaches working controllers with a small fraction of the data and training time required by reinforcement learning baselines.

What carries the argument

The central object is the finite-dimensional Koopman matrix K, which maps lifted observables (state plus control input) one step forward in the lifted space. The load-bearing identity is the recursive least squares update: with α_k the current lifted snapshot, β_k the next lifted snapshot, and P_k the inverse data covariance, the updates P_{k+1} = P_k − γ_k P_k α_k α_k^T P_k and K_{k+1} = K_k + γ_k (β_k − K_k α_k) α_k^T P_k, with γ_k = 1/(1 + α_k^T P_k α_k), reproduce the batch EDMD solution exactly. This Sherman–Morrison rank-one update carries the entire argument: it makes online updating exact, keeps per-step cost at O(n^2), and lets the convergence proof of EDMD apply to the online estim

What would settle it

Take a fixed recorded data stream, compute the batch EDMD Koopman matrix once, then feed the same snapshots one by one through the RLS updates and compare the final matrices; the paper's identity predicts equality up to floating-point roundoff. A concrete way to observe failure: if the inverse covariance P becomes ill-conditioned after many similar states, recursive and batch estimates will diverge. Separately, during an RKL run on hardware, record the empirical distribution of (state, action) pairs over time; if that distribution keeps shifting as the policy updates rather than settling down,

Watch

Extended reading notes

Core claim

The paper establishes an identity and a convergence theorem. The identity: for a Koopman model estimated via EDMD, appending one new state-action snapshot and applying the recursive least squares update to K and P gives exactly the same K as recomputing the closed-form EDMD solution from the whole dataset. The theorem: if the snapshot sequence is an irreducible, aperiodic, positive-recurrent Markov chain with invariant measure μ, the observables are μ-square-integrable, and the covariance matrices stay full-rank, then the EDMD/RLS estimate converges almost surely to K* = E[φ(f(s))φ(s)^T] (E[φ(s)φ(s)^T])^{-1}, the optimal least-squares Koopman operator in the observation space. The paper also

Load-bearing premise

The data stream must behave like a random process with a fixed long-run distribution over states and actions; in the actual online loop the controller changes after every model update, so this stationarity is not guaranteed, and if it fails the convergence proof does not cover the RKL process.

Editorial extensions

If this is right

  • Online model updates are exact, so no information is ever discarded: the model at every step is the least-squares optimal fit to all accumulated data, not an approximation.
  • Per-update cost is O(n^2) and independent of dataset size, so model updates can continue at control frequency even in long-horizon, high-rate deployments.
  • Under the stated ergodic sampling conditions, the learned model converges with probability one to the best possible Koopman operator in the chosen observable space, giving the pipeline a formal consistency guarantee.
  • Data collected while the controller is attempting the goal is highly informative: the weak ACG hypothesis is empirically supported, and RKL matches or beats RL baselines with under 10% of their data and far less wall-clock time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The exact-equivalence identity presupposes that the inverse covariance P stays full-rank. In real deployments, repeated similar states (e.g. a puck lingering in one spot) can make P ill-conditioned, so recursive and batch estimates may drift apart numerically; a practical safeguard would be to periodically re-run batch EDMD and compare.
  • The convergence proof assumes the data stream is a stationary ergodic Markov chain. In RKL the policy changes at every model update, so the closed-loop transition kernel is time-varying; the paper approximates the trajectories as Markov chains, but a fully rigorous online convergence result would need a law of large numbers for time-inhomogeneous chains.
  • The same rank-one update argument transfers to any online linear least-squares model, not just Koopman observables; the O(n^2) complexity and exact-equivalence identity should hold for generic linear state-space model updates.
  • Ergodicity or coverage metrics could be used to actively shape exploration during task execution, steering data collection toward under-sampled regions and avoiding the stagnation failure mode the paper reports; this is a testable extension suggested by the convergence conditions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper presents Recursive Koopman Learning (RKL), an online control-policy learning pipeline that combines an EDMD-initialized Koopman model with RLS-based recursive model updates and MPC. The central claims are: (i) one RLS update is algebraically equivalent to retraining the full EDMD model on the appended dataset; (ii) under ergodic Markov-chain conditions, the EDMD/RLS estimate converges almost surely to the optimal finite-dimensional Koopman estimate; (iii) the per-step update cost is O(n^2) and independent of dataset size; and (iv) on a simulated planar two-link arm and a soft Stewart platform, RKL achieves substantially higher sample efficiency than RL and NN-MPPI baselines. The paper also introduces a 'weak ACG hypothesis' to explain why data collected while attempting the control task are informative for model learning.

Significance. The standalone batch-EDMD consistency argument in Appendix D is a correct application of the Markov-chain SLLN under the stated assumptions, and the RLS equivalence proof in Appendix B is elegant. The complexity analysis is straightforward and the open-source C++ implementation, together with the hardware experiments, is a useful empirical contribution. If the online convergence guarantee were valid, this would be a significant result for sample-efficient Koopman-based control. However, the main theoretical claim—convergence of the online RKL loop—is not established by the supplied analysis because the closed-loop data are not a time-homogeneous Markov chain; the paper proves a batch-consistency result for a fixed system, not convergence of the adaptive online algorithm.

major comments (2)
  1. [Appendix D.2–D.3, Eqs. (30)–(39) and Algorithm 1] The convergence proof applies the Markov-chain SLLN (Eq. (34)) to the data sequence {s_k}. This requires a fixed irreducible, aperiodic, positive-recurrent chain with a single invariant measure μ. In Algorithm 1, s_k=(x_k,u_k) and u_{k+1}=MPC(K_{k+1}, z_{k+1}, \bar{z}), with K_{k+1} updated at every step by Eq. (7). Therefore the transition kernel of {s_k} is time-varying and there is no fixed invariant measure to which the empirical averages in Eqs. (30)–(31) converge. The statement in D.2 that trajectories 'can be approximated as Markov chains' is an assertion, not a derived sufficient condition. Moreover, the initial dataset is generated by a PD controller or random actions (Sec. E.1.3), while online data use the MPC policy, so the concatenated dataset is generated by more than one transition kernel. The argument establishes batch-EDMD consistency for a fixed system, not convergence o
  2. [Appendix D.2] The assumptions of irreducibility, aperiodicity, and positive recurrence are not justified by the experiments. The text argues these follow from controllability of the open-loop system, but the chain is generated by the closed-loop MPC policy, which is typically stabilizing and may have an attractor. A deterministic closed-loop map is not an irreducible Markov chain on the full state-action space unless stochastic exploration is explicitly injected. If exploration noise is present, its distribution and variance must be specified in the model; if not, the SLLN hypotheses fail even for a fixed policy.
minor comments (4)
  1. [Appendix B and Algorithm 3] The notation \bar{z} is used inconsistently: in Algorithm 1 it denotes the control goal, while in Algorithm 3 and the surrounding text it appears to denote the next observation z_{k+1}. This makes the RLS update description confusing and should be clarified.
  2. [Section 3.3 and Eq. (6)] The recursive update of P assumes P_k is invertible, not merely a pseudoinverse. The paper states full rank as an assumption, but this should be stated clearly before Eq. (6), and the exact-arithmetic caveat for the claimed 'same result as EDMD' should be mentioned.
  3. [Section 4.2, Table 1] Results are reported as means over 50 trials without variance or confidence intervals. Several adjacent rows in Table 1 have very close RMSE values (e.g., RKL-SAC 3,000+500 vs 5,000+500), so statistical significance of the ranking is unclear.
  4. [Appendix D] The convergence result is presented as an informal derivation rather than a formal theorem. Stating it as a theorem with explicitly numbered hypotheses and a proof would improve verifiability and make the gap between the batch result and the online loop more visible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RLS-EDMD equivalence and Koopman convergence are self-contained algebraic/SLLN results; the ACG hypothesis is empirically tested, not assumed.

full rationale

The paper's central formal claims are (i) the RLS update equals exact EDMD retraining, proved in Appx. B by direct substitution and the Sherman-Morrison identity; this is an algebraic equivalence, not a reduction to an input. (ii) Under explicitly stated sufficient conditions (irreducible, aperiodic, positive recurrent Markov chain, square-integrable observables, and full rank of the covariance matrices), the EDMD/RLS estimate converges almost surely to the population least-squares Koopman operator K*, defined independently via expectations under the invariant measure. This is a conditional consistency theorem proven by applying Breiman's external Strong Law of Large Numbers for Markov chains to the empirical covariances; the target K* is not defined as the data limit, and no equation is defined in terms of another to force the claimed result. The convergence theorem itself is therefore self-contained. The online applicability of the Markov-chain assumption is stated as an approximation ('can be approximated as Markov chains'), not derived from the conclusion; the skeptic's concern about the time-varying transition kernel in the closed-loop RKL process is a correctness/assumption risk, not a circularity. The 'weak ACG hypothesis' is an empirical hypothesis the paper tests: the experimental success is presented as evidence supporting it, not as a logical consequence of it. The self-citations (MPC-SAC, Koopman-based control, hardware) supply background methods and baselines, but none are load-bearing: the RLS-EDMD proof and the convergence analysis do not rely on any prior work by the authors for their validity. The limitations section candidly notes cases where the ergodicity/coverage assumptions fail, further showing the analysis is not rigged to guarantee the empirical outcome. No circular step can be exhibited as Eq. X = Eq. Y by construction or a fitted parameter renamed as a prediction, so the paper does not meet the threshold for circularity under the stated rules.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central theoretical result rests on standard SLLN and linear algebra, plus the domain assumption that online data is an ergodic Markov chain; the latter is not verified and is the main fragility. No invented physical entities are introduced; the ACG hypothesis is a conceptual framing, not an entity.

free parameters (3)
  • MPC-SAC control weight \bar{R} and LQR weights = Various, e.g., 1e10 for planar arm MPC-SAC, 0.002 for SSP polynomial case
    Chosen by hand for each experiment; listed in Tables 3-10. They affect control performance but not the theoretical claims.
  • Observation function choice and dimension = 17-dim polynomial/trig for arm; 28-dim polynomial or 117-dim RBF for SSP (epsilon and centers via k-means++)
    Selected manually; RBF centers are fit to the initial dataset, and the polynomial degrees are chosen by the authors.
  • Moving average filter window size for state feedback = 65 ms for position, 120 ms for control actions (SSP)
    Chosen empirically to smooth noisy vision-based position feedback.
assumptions (4)
  • standard math Breiman's Strong Law of Large Numbers for irreducible, aperiodic, positive recurrent Markov chains
    Invoked in Appx D.3 to obtain entrywise almost-sure convergence of empirical correlation matrices.
  • domain assumption The online data (states and controls) forms a time-homogeneous ergodic Markov chain
    Sec 3.3 and Appx D.2 state the data 'can be approximated as Markov chains', but the closed-loop transition kernel changes as the model and policy update online.
  • domain assumption The observation function phi is square-integrable w.r.t. the invariant measure, and the covariance matrices P and P_N are full rank
    Appx D.2 requires these for the SLLN and for continuity of the matrix inverse at the limit.
  • domain assumption A finite set of observables approximately spans a Koopman-invariant subspace, giving a linear model phi(x_{k+1}) approx K phi(x_k) + K_g psi(u_k)
    Background Sec 2.1; this is the standard finite-dimensional Koopman approximation used throughout the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sample-Efficient Online Control Policy Learning with Real-Time Recursive Model Updates." pith.science (2026). https://pith.science/paper/EIMPAC2X

@misc{pith2026250908241,
  author       = {Pith},
  title        = {Pith review of: Sample-Efficient Online Control Policy Learning with Real-Time Recursive Model Updates},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EIMPAC2X}},
  note         = {Machine review of arXiv:2509.08241}
}
read the original abstract

Data-driven control methods need to be sample-efficient and lightweight, especially when data acquisition and computational resources are limited -- such as during learning on hardware. Most modern data-driven methods require large datasets and struggle with real-time updates of models, limiting their performance in dynamic environments. Koopman theory formally represents nonlinear systems as linear models over observables, and Koopman representations can be determined from data in an optimization-friendly setting with potentially rapid model updates. In this paper, we present a highly sample-efficient, Koopman-based learning pipeline: Recursive Koopman Learning (RKL). We identify sufficient conditions for model convergence and provide formal algorithmic analysis supporting our claim that RKL is lightweight and fast, with complexity independent of dataset size. We validate our method on a simulated planar two-link arm and a hybrid nonlinear hardware system with soft actuators, showing that real-time recursive Koopman model updates improve the sample efficiency and stability of data-driven controller synthesis -- requiring only <10% of the data compared to benchmarks. The high-performance C++ codebase is open-sourced. Website: https://www.zixinatom990.com/home/robotics/corl-2025-recursive-koopman-learning.

Figures

Figures reproduced from arXiv: 2509.08241 by the authors.

Figure 1
Figure 1. Recursive Koopman Learning Pipeline Designing controllers for nonlinear, dynamic robotic systems is challenging. Conventional model-based approaches rely on first principles to derive system models and then compute con￾trol inputs by solving constrained optimal con￾trol problems, often through trajectory optimiza￾tion. While these methods are mathematically rigorous and have been successfully deployed on robotic sys… view at source ↗
Figure 2
Figure 2. KL-SAC vs. RKL-SAC vs. RL-SAC (five random seeds) in puck balancing. The whiskers [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. The mean and standard deviation of the absolute errors (scaled by log base 10) computed over the last 5 seconds of each 20-second RKL￾SAC trial. This controller uses a 4-minute-long initial dataset and a 28-dimensional polynomial observation function. The distance between each goal point and its six adjacent goal points is 2 cm. We also evaluate RKL-SAC on never-before￾seen states by removing all data with positions… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: The Soft Stewart Platform The Soft Stewart Platform (SSP) is a six-DoF parallel manipulator inspired by the traditional Stew￾art platform design ( [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: The distribution of data in the cropped 4-minute initial dataset, as well as the mean and [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: Tracking a reference trajectory containing contacts. The start, end and both corners of the [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: The results of tracking the reference tip position trajectory on the simulated planar two [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]
Figure 8
Figure 8. Figure 8: The results of tracking the reference tip position trajectory on the simulated planar two [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]
Figure 9
Figure 9. Figure 9: The results of tracking the reference tip position trajectory on the simulated planar two [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]
Figure 10
Figure 10. Figure 10: The results of tracking the reference tip position trajectory on the simulated planar two [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: The distribution of the puck’s positions (m) in the initial dataset. [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: The distribution of the puck’s velocities (m/s) in the initial dataset. [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 36 canonical work pages

  1. [1]

    Kleff, A

    S. Kleff, A. Meduri, R. Budhiraja, N. Mansard, and L. Righetti. High-frequency nonlinear model predictive control of a manipulator. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 7330–7336, 2021. doi:10.1109/ICRA48506.2021.9560990

    arXiv 2021

  2. [2]

    Grandia, F

    R. Grandia, F. Jenelten, S. Yang, F. Farshidian, and M. Hutter. Perceptive locomotion through nonlinear model-predictive control.IEEE Transactions on Robotics, 39(5):3402–3421, 2023. doi:10.1109/TRO.2023.3275384

    arXiv 2023

  3. [3]

    Meduri, P

    A. Meduri, P. Shah, J. Viereck, M. Khadiv, I. Havoutis, and L. Righetti. BiConMP: A nonlinear model predictive control framework for whole body motion planning.IEEE Transactions on Robotics, 39(2):905–922, 2023. doi:10.1109/TRO.2022.3228390

    work page arXiv 2023

  4. [4]

    Le Cleac’h, T

    S. Le Cleac’h, T. A. Howell, S. Yang, C.-Y . Lee, J. Zhang, A. Bishop, M. Schwager, and Z. Manchester. Fast contact-implicit model predictive control.IEEE Transactions on Robotics, 40:1617–1629, 2024. doi:10.1109/TRO.2024.3351554

    work page arXiv 2024

  5. [5]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimiza- tion algorithms.CoRR, abs/1707.06347, 2017. URLhttp://arxiv.org/abs/1707. 06347

    arXiv 2017

  6. [6]

    Fujimoto, H

    S. Fujimoto, H. van Hoof, and D. Meger. Addressing function approximation error in actor- critic methods. In J. Dy and A. Krause, editors,Proceedings of the 35th International Confer- ence on Machine Learning, volume 80 ofProceedings of Machine Learning Research, pages 1587–1596. PMLR, 10–15 Jul 2018. URLhttps://proceedings.mlr.press/v80/ fujimoto18a.html

    work page 2018

  7. [7]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. InProceedings of the 35th International Conference on Machine Learning, volume 80 ofProceedings of Machine Learning Research, pages 1861–1870. PMLR, 10–15 Jul 2018. URLhttps://proceedings.mlr.press/ v80/haarnoj...

    work page 2018

  8. [8]

    Jitosho, T

    R. Jitosho, T. G. W. Lum, A. Okamura, and K. Liu. Reinforcement learning enables real- time planning and control of agile maneuvers for soft robot arms. InProceedings of The 7th Conference on Robot Learning, volume 229 ofProceedings of Machine Learning Research, pages 1131–1153. PMLR, 06–09 Nov 2023

    work page 2023

Show all 52 references
  1. [9]

    Rajeswaran, V

    A. Rajeswaran, V . Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstra- tions. InProceedings of Robotics: Science and Systems, Pittsburgh, Pennsylvania, June 2018. doi:10.15...

    2018 doi

  2. [10]

    Y . Chen, T. Wu, S. Wang, X. Feng, J. Jiang, Z. Lu, S. McAleer, H. Dong, S.-C. Zhu, and Y . Yang. Towards human-level bimanual dexterous manipulation with reinforcement learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors,Advances in Neural I...

    2022

  3. [11]

    Williams, N

    G. Williams, N. Wagener, B. Goldfain, P. Drews, J. M. Rehg, B. Boots, and E. A. Theodorou. Information theoretic mpc for model-based reinforcement learning. In2017 IEEE Inter- national Conference on Robotics and Automation (ICRA), pages 1714–1721, 2017. doi: 10.1109/ICRA.2017....

    2017

  4. [12]

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter. Learning quadrupedal lo- comotion over challenging terrain.Science Robotics, 5(47):eabc5986, 2020. doi:10.1126/ scirobotics.abc5986

    2020

  5. [13]

    J. W. Mock and S. S. Muknahallipatna. A comparison of PPO, TD3 and SAC reinforcement algorithms for quadruped walking gait generation.Journal of Intelligent Learning Systems and Applications, 15(1):36–56, 2023. doi:10.4236/jilsa.2023.151003

    2023

  6. [14]

    J. H. Tu.Dynamic mode decomposition: Theory and applications. PhD thesis, Princeton University, 2013. URLhttp://turing.library.northwestern. edu/login?url=https://www.proquest.com/dissertations-theses/ dynamic-mode-decomposition-theory-applications/docview/ 1458341928/se-2

    2013

  7. [15]

    M. O. Williams, I. G. Kevrekidis, and C. W. Rowley. A data–driven approximation of the Koopman operator: Extending dynamic mode decomposition.Journal of Nonlinear Science, 25:1307–1346, 2015. doi:10.1007/s00332-015-9258-5

    2015 doi

  8. [16]

    C. W. Rowley and S. T. Dawson. Model reduction for flow analysis and control.An- nual Review of Fluid Mechanics, 49(49):387–417, 2017. ISSN 1545-4479. doi:10.1146/ annurev-fluid-010816-060042

    2017

  9. [17]

    Geneva and N

    N. Geneva and N. Zabaras. Transformers for modeling physical systems.Neural Networks, 146:272–289, 2022. ISSN 0893-6080. doi:10.1016/j.neunet.2021.11.022

    2022 doi

  10. [18]

    Susuki and I

    Y . Susuki and I. Mezi ´c. Nonlinear Koopman modes of coupled swing dynamics and co- herency identification. InIEEE PES General Meeting, pages 1–8, 2010. doi:10.1109/PES. 2010.5589363

    2010

  11. [19]

    Bruder, B

    D. Bruder, B. Gillespie, C. D. Remy, and R. Vasudevan. Modeling and control of soft robots using the Koopman operator and model predictive control. InProceedings of Robotics: Science and Systems, Freiburg im Breisgau, Germany, June 2019. doi:10.15607/RSS.2019.XV .060

    2019 doi

  12. [20]

    Bruder, D

    D. Bruder, D. Bombara, and R. J. Wood. A Koopman-based residual modeling approach for the control of a soft robot arm.The International Journal of Robotics Research, 0(0), 2024. doi:10.1177/02783649241272114

    2024 doi

  13. [21]

    Abraham and T

    I. Abraham and T. D. Murphey. Active learning of dynamics for data-driven control using Koopman operators.IEEE Transactions on Robotics, 35(5):1071–1083, 2019. doi:10.1109/ TRO.2019.2923880

    2019

  14. [22]

    Mamakoukas, M

    G. Mamakoukas, M. L. Casta ˜no, X. Tan, and T. D. Murphey. Derivative-based Koopman operators for real-time control of robotic systems.IEEE Transactions on Robotics, 37(6): 2173–2192, 2021. doi:10.1109/TRO.2021.3076581

    2021

  15. [23]

    F. Li, A. Abuduweili, Y . Sun, R. Chen, W. Zhao, and C. Liu. Continual learning and lifting of Koopman dynamics for linear control of legged robots. InProceedings of the 7th An- nual Learning for Dynamics and Control Conference, volume 283 ofProceedings of Machine Learning Res...

    2025

  16. [24]

    D. A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In D. Touretzky, editor,Advances in Neural Information Processing Systems, volume 1. Morgan-Kaufmann,

  17. [25]

    M. H. Hayes. Chapter 9.4 - recursive least squares. InStatistical Digital Signal Processing and Modeling. John Wiley & Sons, 1996. doi:10.1080/00401706.1997.10485128. 11

    1996

  18. [26]

    Nagabandi, K

    A. Nagabandi, K. Konolige, S. Levine, and V . Kumar. Deep dynamics models for learn- ing dexterous manipulation. In2020 Conference on Robot Learning (CoRL), pages 1101–

  19. [27]

    Zhang, C

    H. Zhang, C. W. Rowley, E. A. Deem, and L. N. Cattafesta. Online dynamic mode decompo- sition for time-varying systems.SIAM Journal on Applied Dynamical Systems, 18(3):1586– 1609, 2019. doi:10.1137/18M1192329

    2019 doi

  20. [28]

    H. M. Calder ´on, E. Schulz, T. Oehlschl ¨agel, and H. Werner. Koopman operator-based model predictive control with recursive online update. In2021 European Control Conference (ECC), pages 1543–1549, 2021. doi:10.23919/ECC54610.2021.9655220

    2021

  21. [29]

    N ¨uske, S

    F. N ¨uske, S. Peitz, F. Philipp, M. Schaller, and K. Worthmann. Finite-data error bounds for Koopman-based prediction and control.Journal of Nonlinear Science, 33(1):14, 2023. doi: 10.1007/s00332-022-09862-1

    2023 doi

  22. [30]

    Zhang and E

    C. Zhang and E. Zuazua. A quantitative analysis of Koopman operator methods for system identification and predictions.Comptes Rendus. M ´ecanique, 351(S1):1–31, 2023. doi:10. 5802/crmeca.138

    2023

  23. [31]

    F. M. Philipp, M. Schaller, S. Boshoff, S. Peitz, F. N ¨uske, and K. Worthmann. Extended dynamic mode decomposition: Sharp bounds on the sample efficiency.arXiv preprint arXiv:2402.02494, 2024. URLhttps://arxiv.org/abs/2402.02494

    2024 arXiv

  24. [32]

    L. Breiman. The strong law of large numbers for a class of Markov chains.The Annals of Math- ematical Statistics, 31(3):801–803, 1960. URLhttps://www.jstor.org/stable/ 2237593

    1960

  25. [33]

    B. O. Koopman. Hamiltonian systems and transformation in hilbert space.Proceedings of the National Academy of Sciences, 17(5):315–318, 1931. doi:10.1073/pnas.17.5.315

    1931 doi

  26. [34]

    H. H. Asada. Global, unified representation of heterogenous robot dynamics using composition operators: A Koopman direct encoding method.IEEE/ASME Transactions on Mechatronics, 28(5):2633–2644, 2023. doi:10.1109/TMECH.2023.3253599

    2023

  27. [35]

    J. L. Proctor, S. L. Brunton, and J. N. Kutz. Dynamic mode decomposition with control.SIAM Journal on Applied Dynamical Systems, 15(1):142–161, 2016. doi:10.1137/15M1013857

    2016 doi

  28. [36]

    Sherman and W

    J. Sherman and W. J. Morrison. Adjustment of an inverse matrix corresponding to a change in one element of a given matrix.The Annals of Mathematical Statistics, 21(1):124–127, 1950. ISSN 00034851. URLhttp://www.jstor.org/stable/2236561

    1950

  29. [37]

    A. R. Ansari and T. D. Murphey. Sequential action control: Closed-form optimal control for nonlinear and nonsmooth systems.IEEE Transactions on Robotics, 32(5):1196–1214, 2016. doi:10.1109/TRO.2016.2596768

    2016

  30. [38]

    Nishimura and M

    H. Nishimura and M. Schwager. SACBP: Belief space planning for continuous-time dynam- ical systems via stochastic sequential action control.The International Journal of Robotics Research, 40(10-11):1167–1195, 2021. doi:10.1177/02783649211037697

    2021 doi

  31. [39]

    Ketchum, J

    J. Ketchum, J. Avtges, M. Schlafly, H. Young, T. Kim, R. L. Truby, and T. D. Murphey. Force and speed in a soft stewart platform, 2025. URLhttps://arxiv.org/abs/2504. 13127

    2025

  32. [40]

    T. Kim, P. Kaarthik, and R. L. Truby. A flexible, architected soft robotic actuator for motorized extensional motion.Advanced Intelligent Systems, 6(11):2300866, 2024. doi:10.1002/aisy. 202300866. 12

    2024 doi

  33. [41]

    Abraham, G

    I. Abraham, G. de la Torre, and T. Murphey. Model-based control using Koopman operators. InProceedings of Robotics: Science and Systems, Cambridge, Massachusetts, July 2017. doi: 10.15607/RSS.2017.XIII.052

    2017 doi

  34. [42]

    X. Chen, C. Wang, Z. Zhou, and K. W. Ross. Randomized ensembled double q-learning: Learning fast without a model. InInternational Conference on Learning Representations,

  35. [43]

    Avtges, J

    J. Avtges, J. Ketchum, M. Schlafly, H. Young, T. Kim, A. Pinosky, R. L. Truby, and T. D. Murphey. Real-time reinforcement learning for dynamic tasks with a parallel soft robot. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025

    2025

  36. [44]

    Boyd and L

    S. Boyd and L. Vandenberghe. Part III least squares. InIntroduction to applied linear algebra: vectors, matrices, and least squares. Cambridge university press, 2018. URLhttps:// web.stanford.edu/˜boyd/vmls/vmls.pdf

    2018

  37. [45]

    G. W. Stewart. On the continuity of the generalized inverse.SIAM Journal on Applied Mathe- matics, 17(1):33–45, 1969. ISSN 00361399. URLhttp://www.jstor.org/stable/ 2099241

    1969

  38. [46]

    Foundation

    F. Foundation. Reacher - gymnasium documentation.https://gymnasium.farama. org/environments/mujoco/reacher/

  39. [47]

    T. A. Berrueta, A. Pinosky, and T. D. Murphey. Maximum diffusion reinforcement learning. Nature Machine Intelligence, 6(5):504–514, 2024

    2024

  40. [48]

    Raffin, A

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann. Stable-baselines3: Reliable reinforcement learning implementations.Journal of Machine Learning Research, 22 (268):1–8, 2021. URLhttp://jmlr.org/papers/v22/20-1364.html

    2021

  41. [49]

    Williams, P

    G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou. Aggressive driving with model predictive path integral control. In2016 IEEE International Conference on Robotics and Automation (ICRA), pages 1433–1440, 2016. doi:10.1109/ICRA.2016.7487277. A Sequential Actio...

    2016

  42. [1112]

    URLhttps://proceedings.mlr.press/v100/ nagabandi20a.html

    PMLR, 30 Oct–01 Nov 2020. URLhttps://proceedings.mlr.press/v100/ nagabandi20a.html

    2020

  43. [1988]

    URLhttps://proceedings.neurips.cc/paper_files/paper/1988/ file/812b4ba287f5ee0bc9d43bbf5bbe87fb-Paper.pdf

    1988

  44. [2021]

    URLhttps://openreview.net/forum?id=AY8zfZm0tDd

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.