Pith. sign in

REVIEW 4 major objections 5 minor 42 references

CARoL: Context-aware Adaptation for Robot Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A robot can rank prior tasks by how well their learned state-transition models predict a new task's dynamics, then weight prior policies accordingly.

desk verdict CARoL is a clean context-aware knowledge-reuse trick, but its headline claim of 'learning a new task' overstates what the reward-free Algorithm 1 actually does; worth refereeing with a demand for re-framing and better baseline coverage. read the letter →

arxiv 2506.07006 v1 pith:3PMNNSR5 submitted 2025-06-08 cs.RO cs.AI

Zechen Hu , Tong Xu , Xuesu Xiao , Xuan Wang This is my paper
classification cs.ROcs.AI
keywords reinforcementlearningrobotknowledgetransferstatetransitioncontextawarenesspolicyadaptationoff-roadnavigationmulti-task
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the right way to decide which previously learned robotic skill matters for a new task is to compare how the tasks move: the state transitions in the system's dynamics. It proposes CARoL, which learns a separate transition-prediction function for each source task, then scores a source task by how well that function predicts the target task's observed trajectories. Those scores become weights that bias the target policy toward the most relevant source policies or value functions. The authors show the weighting scheme fits policy-based, value-based, and actor-critic reinforcement learning, and they report faster adaptation in two simulated control tasks and on a physical ground vehicle navigating rocky terrain.

What carries the argument

The central object is the state-transition prediction function $f_i(s,a \mid \phi_i)$, a learned map from a state-action pair to the next state, trained separately on each source task's data. It serves as the context representation because it approximates the transition probability $P_i$, which the paper identifies as the factor that determines what knowledge an MDP needs. CARoL's similarity measure is the prediction error $Y_i$ of $f_i$ on the target task's trajectories, and the softmax of negative errors produces the weights $w_i$ that reweight source knowledge in the losses $L_P$, $L_Q$, and $L_{AC}$. The method's key move is to measure task relatedness through dynamics rather than through environment or robot encodings.

What would settle it

Take three tasks with identical low-dimensional state representation but a hidden dynamics parameter that only affects high-dimensional observations. If two source tasks show nearly equal transition-prediction errors on target trajectories yet measurably different transfer performance, the proxy fails; a concrete check would withhold wheel-slip information from $f_i$ while letting the policy observe camera images, then test whether target tasks with different slip regimes receive incorrect weights.

Watch

Extended reading notes

Core claim

The central claim is that in a Markov decision process the transition probability is the contextual marker that ties prior knowledge to a new task: source tasks whose state transitions resemble the target's are the ones whose policies or value functions should be reused. CARoL operationalizes this by training a small state-transition model $f_i(s,a)$ for each source task and computing $Y_i$, the sum of squared prediction errors of that model on trajectories sampled from the target task. Low error means high similarity, and the errors are turned into softmax weights $w_i$. Those weights enter three adaptation losses: a KL-divergence loss pulling the target policy toward weighted source policies, a Bellman target built from weighted source $Q$-functions, and an actor-critic loss that combines both. The paper reports that the resulting weights track the true task configurations, that CARoL converges faster than learning from scratch or fusing all source knowledge equally, and that it transfers policies from simulation to a real off-road vehicle using only a small set of manually collected trajectories.

Load-bearing premise

The load-bearing premise is that a source task's transition-prediction error on the target's trajectories is a faithful proxy for how useful that source's policy or value function will be on the target; if the learned dynamics miss control-relevant features, the weights can rank knowledge incorrectly even though the rest of the pipeline is sound.

Editorial extensions

If this is right

  • If the similarity weights are correct, a robot can rank its prior knowledge without training a transition model on the new task, using only a small set of target trajectories.
  • The same weighting scheme can be inserted into policy-based, value-based, or actor-critic algorithms, so existing RL pipelines can absorb it without changing their core learning mechanism.
  • When the target task matches a source task, the method assigns near-unit weight to that source, giving the framework a built-in sanity check during adaptation.
  • Combining the weighted prior-knowledge loss with a standard RL loss, as the paper's CARoL+ variant does, preserves fast convergence while recovering in cases where no source task is close.
  • If the weights are computed from a source task that is itself poorly learned, the quality of the source knowledge, not the similarity measure, becomes the limiting factor for the final target policy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not develop is online reweighting: if target trajectories arrive incrementally during deployment, the same $Y_i$ computation could update the weights continuously, turning CARoL from a one-shot pre-adaptation step into a real-time task detector.
  • The ranking claim suggests a sharp test not reported in the paper: the ordering of similarity weights should predict the ordering of transfer performance of the source policies on held-out target tasks.
  • Because the context model in the car experiment uses only speed and gyroscope readings while the policy consumes stacked camera images, the method implicitly bets that control-relevant dynamics are visible in the low-dimensional state; testing on a representation that hides a control-relevant variable would bound the method's scope.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. CARoL proposes to accelerate RL adaptation to a new task by (i) learning a transition function for each source task, (ii) computing softmax similarity weights from the prediction error of these transition functions on target trajectories, and (iii) using those weights to adapt source policies, value functions, or actor-critic pairs to the target. The paper presents three algorithmic variants (policy-based, value-based, actor-critic), validates them in CarRacing and LunarLander, and reports a physical ground-vehicle experiment in which a policy adapted from two simulated off-road skills achieves 5/5 successful traversals of a rocky testbed. The central claim is that state-transition prediction error is a useful contextual marker for ranking source knowledge, and that the weighted adaptation yields faster convergence and higher reward than indiscriminate distillation, meta-learning, or learning from scratch.

Significance. The core idea is simple and interpretable: use the prediction error of learned source transition models on target trajectories as a task-similarity measure. This is a sensible alternative to latent environment encodings, and the weights are computed from independent target samples rather than fitted to target performance, so the core mechanism is not circular. The paper also provides a useful comparison against uniform policy distillation, and the real-world off-road deployment is a valuable practicality check. However, the central claim that CARoL enables efficient RL learning of a new task is not currently supported: the policy-based and actor-critic variants never use the target reward, and the main simulation results are single-curve comparisons without statistical grounding. The contribution would be significant if reframed as context-aware knowledge fusion or if the reward-based CARoL+ variant were made the primary method, but as written the evidence does not match the strength of the claims.

major comments (4)
  1. [Section IV.B.1, Eq. (5); Section IV.B.3, Eq. (9); Algorithms 1 and 3] Among the three proposed algorithms, only the value-based variant (Eq. (8)) includes the target reward. Algorithm 1 minimizes only the weighted KL divergence in Eq. (5), and Algorithm 3 adds Eq. (9), which evaluates the target actor with source critics; neither term depends on the target reward. Remark 1 explicitly states that standard RL losses are omitted and defers reward-based learning to the CARoL+ variant. Since the CarRacing experiments (Section V.A.2) and the real-world experiment (Section V.C.2) use Algorithm 1, the reported "faster convergence and higher rewards" measure reward-free policy distillation, not RL-based learning of a new task. With no target reward, there is no mechanism to discover behavior that outperforms the best achievable weighted combination of source behaviors. The abstract's claim that CARoL "demonstrates faster convergence and higher rewards when learning policies for new tasks" therefore overstates what is shown. Please either reframe the contribution as context-aware policy fusion/adaptation without reward-based learning, or make CARoL+ (with target reward) the main method and report its performance on all targets.
  2. [Section V.A.2 and Eq. (3)] In CarRacing, the transition function used for context is trained on speed and gyroscope readings only, while the policy consumes stacked camera images. The similarity measure Y_i in Eq. (3) therefore ranks source tasks based on a low-dimensional projection of the dynamics that may omit control-relevant features such as track curvature or visual friction cues. The central premise of Section IV is that transition probabilities determine the knowledge needed to solve an MDP; this premise requires f_i to approximate the dynamics that actually dictate policy choice. Please provide evidence that Y_i correctly ranks source tasks by their usefulness on the target, for example by plotting Y_i against the measured return of each source policy on each target task, or by training the context model on the full observation and comparing the resulting weights. Without such validation, the similarity-ranking claim is not fully supported.
  3. [Section V.A.3, Section V.B.3, Figures 4 and 6] The main experimental claims rest on single learning curves with no seeds, confidence intervals, or significance tests. Statements such as "CARoL demonstrates a faster convergence speed compared to other methods" and "CARoL generally achieves the best performance" are not substantiated by any measure of variability. Please report mean and variance over at least five random seeds for the simulation experiments, and specify how many evaluation episodes are used per curve point. This is particularly important because the reward-free objective gives no a priori reason for CARoL to beat every source policy, so the observed advantages need statistical support.
  4. [Section V.B.3 and Figure 5] The paper acknowledges that on target task (i) and on some trials of (j), Learning from Scratch (LfS) outperforms CARoL, and that only CARoL+ recovers the gap. This is exactly the failure mode one would expect from an adaptation objective that contains no target reward. The abstract and conclusion should scope the claims accordingly: CARoL can effectively blend source knowledge for target configurations inside or near the source envelope, but it cannot discover new behavior for targets far outside it. At present, this limitation appears only as a caveat in the experimental analysis, which conflicts with the paper's general claim that CARoL "enables efficient learning of a new task."
minor comments (5)
  1. [Section IV.B, Eq. (6)] Equation (6) contains unbalanced parentheses and the KL divergence is not written with explicit sums over actions. Please rewrite it as D(pi_i(s), pi_g(s|theta_g)) = sum_a Phi(pi_i(s)/T)_a * ln( Phi(pi_i(s)/T)_a / Phi(pi_g(s|theta_g))_a ).
  2. [Algorithm 2, line 4] The description "Greedy method to choose action a with explorations or argmax_a(Q_g(s,a|psi_g))" is ambiguous. Please specify the epsilon-greedy schedule or the exploration mechanism used during data collection.
  3. [Figures 4 and 6 captions] The captions state that dashed lines show source knowledge (SK) rewards, but they do not identify which dash style corresponds to SK1, SK2, SK3, and SK4, nor do they map subfigures (a)–(h) or (a)–(j) to the target configurations listed in the text. Adding this information would make the figures much easier to interpret.
  4. [Section I and Code Availability] The footnote "Code will be publicly available on/github" is an incomplete sentence. Please either provide a complete repository URL or state that code will be released upon publication.
  5. [Tables I and II] The table headers T1, T2, T3, T4 refer to source tasks whose configurations are described only in the text or in Figure 5. Adding the source-task parameters directly in the table headers would improve readability.

Circularity Check

1 steps flagged · score 2.0 of 10

CARoL's core mechanism is not circular; only the diagonal sanity check is tautological, and self-citations are not load-bearing.

  1. self definitional [Section V.A.3, Experiment Results and Analysis (CarRacing), and analogous passage in Section V.B.3 (LunarLander)]
    "In particular, the tasks with µ=0.5, µ=1, and µ=2 correspond to scenarios where the target task is the same as one of the source tasks. These serve as a sanity check for the algorithm's ability to correctly identify contextual similarity. The results confirm the effectiveness of our approach, as the corresponding source task consistently receives the highest similarity weight."

    The similarity metric Y_i in Eq. (3) is the L2 prediction error of the source transition function f_i on target samples, and Eq. (2) trains f_i by minimizing the same L2 loss on samples from source task T_i. When the target task is literally the same as T_i, the matching f_i must have low error relative to other f_j by construction, so the corresponding weight w_i being highest is a direct consequence of the training objective rather than independent evidence that transition similarity predicts transferable knowledge. This is a tautological validation of the metric, but it is not load-bearing for the central adaptation claim, which is evaluated on held-out target rewards.

full rationale

CARoL's core pipeline is not circular. The contextual weights in Eq. (4) are computed from pre-trained source transition models and target trajectory samples, not from target rewards, and the adapted policy or value function is then optimized via weighted source knowledge (Eqs. (5), (7), (8), (9)). No equation in the derivation equates the final adapted policy's target performance with the input weights by construction; the weights are an approximate relevance proxy, and the target-task reward is an independent evaluation signal. The paper itself honestly notes that CARoL alone does not learn from target rewards and that CARoL+ is needed when source knowledge is insufficient, which further shows the core method is not a self-fulfilling prediction. The only mild self-definitional element is the diagonal sanity check in Tables I and II, where a target identical to a source task mechanically yields the highest weight for that source because f_i was trained on that task's transitions; this is a minor validation artifact rather than a load-bearing derivation step. The paper's self-citations (e.g., [15], [36], [38]) are confined to related-work examples and do not justify a central premise, so they do not raise the circularity score. Overall, the central claim retains independent empirical content and the paper is only mildly circular in its sanity-check presentation.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The ledger captures the main load-bearing premises. No new physical entities are introduced; the free parameters are algorithm hyperparameters (beta and temperature) that affect the adaptation losses but are not central constants. The method's validity depends on the transition-similarity proxy, the shared-MDP assumption, and the quality of the source knowledge.

free parameters (2)
  • beta (actor-critic weighting) = 1
    In LunarLander, L_AC = L_P + beta * L_C is used with beta = 1 (Section V.B.2). The paper notes beta is adjustable, but no sweep or sensitivity analysis is given.
  • temperature T in softmax KL divergence = not reported
    Equation (6) defines D(pi_i, pi_g) with a softmax temperature T, which controls how sharply source policy distributions are compared; T is never specified, so its effect on all policy and actor-critic results is unknown.
assumptions (5)
  • domain assumption Source tasks and the target task share the same state space, action space, and reward function, and differ only in transition probabilities (Section III.B).
    This shared-MDP assumption makes knowledge transfer well-defined, but it is only approximate in the real sim-to-real experiment and limits the method to tasks with identical reward design.
  • domain assumption Similarity in transition probabilities implies similarity in the knowledge (policy or value function) needed to solve the task (Section IV).
    The paper argues this from the central role of P in an MDP, but offers no theorem or counterexample analysis. It is the main theoretical premise for using transition prediction error as a relevance weight.
  • ad hoc to paper The learned transition functions f_i are accurate enough to rank source tasks on target data (Section IV, Equation 3).
    The transition models are small MLPs trained on a reduced state representation, and the paper provides no validation of their accuracy or of how errors affect the weights.
  • domain assumption Source tasks are sufficiently trained so that their knowledge is worth transferring (Section III.B).
    The Limitations section admits CARoL depends heavily on source knowledge quality; no mechanism evaluates source reliability.
  • ad hoc to paper The 20 manually collected real-world trajectories are representative of the states the adapted policy will visit (Section V.C.2).
    Similarity weights in the physical experiment are computed before adaptation from human-driven trials; if those trials miss states encountered by the learned policy, the weights can be wrong.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CARoL: Context-aware Adaptation for Robot Learning." pith.science (2026). https://pith.science/paper/3PMNNSR5

@misc{pith2026250607006,
  author       = {Pith},
  title        = {Pith review of: CARoL: Context-aware Adaptation for Robot Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3PMNNSR5}},
  note         = {Machine review of arXiv:2506.07006}
}
read the original abstract

Using Reinforcement Learning (RL) to learn new robotic tasks from scratch is often inefficient. Leveraging prior knowledge has the potential to significantly enhance learning efficiency, which, however, raises two critical challenges: how to determine the relevancy of existing knowledge and how to adaptively integrate them into learning a new task. In this paper, we propose Context-aware Adaptation for Robot Learning (CARoL), a novel framework to efficiently learn a similar but distinct new task from prior knowledge. CARoL incorporates context awareness by analyzing state transitions in system dynamics to identify similarities between the new task and prior knowledge. It then utilizes these identified similarities to prioritize and adapt specific knowledge pieces for the new task. Additionally, CARoL has a broad applicability spanning policy-based, value-based, and actor-critic RL algorithms. We validate the efficiency and generalizability of CARoL on both simulated robotic platforms and physical ground vehicles. The simulations include CarRacing and LunarLander environments, where CARoL demonstrates faster convergence and higher rewards when learning policies for new tasks. In real-world experiments, we show that CARoL enables a ground vehicle to quickly and efficiently adapt policies learned in simulation to smoothly traverse real-world off-road terrain.

Figures

Figures reproduced from arXiv: 2506.07006 by the authors.

Figure 1
Figure 1. An example of autonomous off-road navigation involves a vehicle [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of Context-aware Adaptation for Robot Learning [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Two OpenAI simulation environments for validating our algorithm: [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of cumulative reward for CARoL and baselines under different CarRacing tasks. The x-axis represents the running steps and the y-axis [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Task configurations in Lunarlander. In the plot, circles represent the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of cumulative reward for CARoL and baselines under different LunarLander tasks. The x-axis represents the running episodes and the [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 23 canonical work pages

  1. [1]

    Deep reinforce- ment learning: A brief survey.IEEE Signal Processing Magazine, 34(6):26–38, 2017

    Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. Deep reinforce- ment learning: A brief survey.IEEE Signal Processing Magazine, 34(6):26–38, 2017

    work page 2017

  2. [2]

    Flexible attention-based multi-policy fusion for efficient deep reinforcement learning.Advances in Neural Information Processing Systems, 36, 2024

    Zih-Yun Chiu, Yi-Lin Tuan, William Yang Wang, and Michael Yip. Flexible attention-based multi-policy fusion for efficient deep reinforcement learning.Advances in Neural Information Processing Systems, 36, 2024

    work page 2024

  3. [3]

    Model- agnostic meta-learning for fast adaptation of deep net- works

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep net- works. InInternational conference on machine learning, pages 1126–1135. PMLR, 2017

    2017

  4. [4]

    Address- ing function approximation error in actor-critic methods

    Scott Fujimoto, Herke Hoof, and David Meger. Address- ing function approximation error in actor-critic methods. InInternational conference on machine learning, pages 1587–1596. PMLR, 2018

    2018

  5. [5]

    Contextual markov decision processes.arXiv preprint arXiv:1502.02259, 2015

    Assaf Hallak, Dotan Di Castro, and Shie Mannor. Contextual markov decision processes.arXiv preprint arXiv:1502.02259, 2015

    arXiv 2015

  6. [6]

    Learning an embedding space for transferable robot skills

    Karol Hausman, Jost Tobias Springenberg, Ziyu Wang, Nicolas Heess, and Martin Riedmiller. Learning an embedding space for transferable robot skills. InInter- national Conference on Learning Representations, 2018

    work page 2018

  7. [7]

    Mt-opt: Continuous multi-task robotic reinforcement learning at scale.arXiv preprint arXiv:2104.08212, 2021

    Dmitry Kalashnikov, Jacob Varley, Yevgen Chebotar, Benjamin Swanson, Rico Jonschkowski, Chelsea Finn, Sergey Levine, and Karol Hausman. Mt-opt: Continuous multi-task robotic reinforcement learning at scale.arXiv preprint arXiv:2104.08212, 2021

    arXiv 2021

  8. [8]

    Fast online adaptation in robotics through meta-learning embeddings of simulated priors

    Rituraj Kaushik, Timoth ´ee Anne, and Jean-Baptiste Mouret. Fast online adaptation in robotics through meta-learning embeddings of simulated priors. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5269–5276. IEEE, 2020

    work page 2020

Show all 42 references
  1. [9]

    Sliced-wasserstein autoencoder: An embarrassingly simple generative model.arXiv preprint arXiv:1804.01947, 2018

    Soheil Kolouri, Phillip E Pope, Charles E Martin, and Gustavo K Rohde. Sliced-wasserstein autoencoder: An embarrassingly simple generative model.arXiv preprint arXiv:1804.01947, 2018

    2018 arXiv

  2. [10]

    Rma: Rapid motor adaptation for legged robots

    Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021

    2021 arXiv

  3. [11]

    Learning quadrupedal locomotion over challenging terrain.Science robotics, 5 (47):eabc5986, 2020

    Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. Learning quadrupedal locomotion over challenging terrain.Science robotics, 5 (47):eabc5986, 2020

    2020

  4. [12]

    Context-aware dynamics model for generalization in model-based reinforcement learning

    Kimin Lee, Younggyo Seo, Seunghyun Lee, Honglak Lee, and Jinwoo Shin. Context-aware dynamics model for generalization in model-based reinforcement learning. InInternational Conference on Machine Learning, pages 5757–5766. PMLR, 2020

    2020

  5. [13]

    Context-aware policy reuse.arXiv preprint arXiv:1806.03793, 2018

    Siyuan Li, Fangda Gu, Guangxiang Zhu, and Chongjie Zhang. Context-aware policy reuse.arXiv preprint arXiv:1806.03793, 2018

    2018 arXiv

  6. [14]

    Context-aware deep reinforcement learning for autonomous robotic navigation in unknown area

    Jingsong Liang, Zhichen Wang, Yuhong Cao, Jimmy Chiun, Mengqi Zhang, and Guillaume Adrien Sar- toretti. Context-aware deep reinforcement learning for autonomous robotic navigation in unknown area. InCon- ference on Robot Learning, pages 1425–1436. PMLR, 2023

    2023

  7. [15]

    A lifelong learning approach to mobile robot navigation.IEEE Robotics and Automation Letters, 6(2):1090–1096, 2021

    Bo Liu, Xuesu Xiao, and Peter Stone. A lifelong learning approach to mobile robot navigation.IEEE Robotics and Automation Letters, 6(2):1090–1096, 2021

    2021

  8. [16]

    Lifelong feder- ated reinforcement learning: a learning architecture for navigation in cloud robotic systems.IEEE Robotics and Automation Letters, 4(4):4555–4562, 2019

    Boyi Liu, Lujia Wang, and Ming Liu. Lifelong feder- ated reinforcement learning: a learning architecture for navigation in cloud robotic systems.IEEE Robotics and Automation Letters, 4(4):4555–4562, 2019

    2019

  9. [17]

    Learning then, learning now, and every second in be- tween: lifelong learning with a simulated humanoid robot.Frontiers in Neurorobotics, 15:669534, 2021

    Aleksej Logacjov, Matthias Kerzel, and Stefan Wermter. Learning then, learning now, and every second in be- tween: lifelong learning with a simulated humanoid robot.Frontiers in Neurorobotics, 15:669534, 2021

    2021

  10. [18]

    Gradient episodic memory for continual learning.Advances in neural information processing systems, 30, 2017

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning.Advances in neural information processing systems, 30, 2017

    2017

  11. [19]

    Lifelong robot learning with human assisted language planners

    Meenal Parakh, Alisha Fong, Anthony Simeonov, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. Lifelong robot learning with human assisted language planners. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 523–529. IEEE, 2024

    2024

  12. [20]

    Actor-mimic: Deep multitask and transfer reinforce- ment learning.arXiv preprint arXiv:1511.06342, 2015

    Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdi- nov. Actor-mimic: Deep multitask and transfer reinforce- ment learning.arXiv preprint arXiv:1511.06342, 2015

    2015 arXiv

  13. [21]

    Deepmimic: Example-guided deep re- inforcement learning of physics-based character skills

    Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel Van de Panne. Deepmimic: Example-guided deep re- inforcement learning of physics-based character skills. ACM Transactions On Graphics (TOG), 37(4):1–14, 2018

    2018

  14. [22]

    Learning agile robotic locomotion skills by imitating animals.arXiv preprint arXiv:2004.00784, 2020

    Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang- Wei Lee, Jie Tan, and Sergey Levine. Learning agile robotic locomotion skills by imitating animals.arXiv preprint arXiv:2004.00784, 2020

    2004 arXiv

  15. [23]

    Model-based reinforcement learning variable impedance control for human-robot collabora- tion.Journal of Intelligent & Robotic Systems, 100(2): 417–433, 2020

    Loris Roveda, Jeyhoon Maskani, Paolo Franceschi, Arash Abdi, Francesco Braghin, Lorenzo Molinari Tosatti, and Nicola Pedrocchi. Model-based reinforcement learning variable impedance control for human-robot collabora- tion.Journal of Intelligent & Robotic Systems, 100(2): 417–433, 2020

    2020

  16. [24]

    Knowledge adaptation: Teaching to adapt.arXiv preprint arXiv:1702.02052, 2017

    Sebastian Ruder, Parsa Ghaffari, and John G Breslin. Knowledge adaptation: Teaching to adapt.arXiv preprint arXiv:1702.02052, 2017

    2017 arXiv

  17. [25]

    Policy distillation.arXiv preprint arXiv:1511.06295, 2015

    Andrei A Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Raz- van Pascanu, V olodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell. Policy distillation.arXiv preprint arXiv:1511.06295, 2015

    2015 arXiv

  18. [26]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  19. [27]

    Rapidly adaptable legged robots via evolutionary meta- learning

    Xingyou Song, Yuxiang Yang, Krzysztof Choromanski, Ken Caluwaerts, Wenbo Gao, Chelsea Finn, and Jie Tan. Rapidly adaptable legged robots via evolutionary meta- learning. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3769–3776. IEEE, 2020

    2020

  20. [28]

    Fully autonomous real-world reinforce- ment learning with applications to mobile manipula- tion

    Charles Sun, Jedrzej Orbik, Coline Manon Devin, Brian H Yang, Abhishek Gupta, Glen Berseth, and Sergey Levine. Fully autonomous real-world reinforce- ment learning with applications to mobile manipula- tion. InConference on Robot Learning, pages 308–319. PMLR, 2022

    2022

  21. [29]

    Chrono: An open source multi-physics dynamics engine

    Alessandro Tasora, Radu Serban, Hammad Mazhar, Ar- man Pazouki, Daniel Melanz, Jonathan Fleischmann, Michael Taylor, Hiroyuki Sugiyama, and Dan Negrut. Chrono: An open source multi-physics dynamics engine. InHigh Performance Computing in Science and Engi- neering: Second Inter...

    2015

  22. [30]

    Distral: Robust multitask rein- forcement learning.Advances in neural information processing systems, 30, 2017

    Yee Teh, Victor Bapst, Wojciech M Czarnecki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, and Razvan Pascanu. Distral: Robust multitask rein- forcement learning.Advances in neural information processing systems, 30, 2017

    2017

  23. [31]

    Deep reinforcement learning with double q-learning

    Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q-learning. InPro- ceedings of the AAAI conference on artificial intelligence, volume 30, 2016

    2016

  24. [32]

    Selective knowledge distillation for neural ma- chine translation.arXiv preprint arXiv:2105.12967, 2021

    Fusheng Wang, Jianhao Yan, Fandong Meng, and Jie Zhou. Selective knowledge distillation for neural ma- chine translation.arXiv preprint arXiv:2105.12967, 2021

    2021 arXiv

  25. [33]

    Deep reinforcement learning: A survey.IEEE Transactions on Neural Networks and Learning Systems, 35(4):5064– 5078, 2022

    Xu Wang, Sen Wang, Xingxing Liang, Dawei Zhao, Jincai Huang, Xin Xu, Bin Dai, and Qiguang Miao. Deep reinforcement learning: A survey.IEEE Transactions on Neural Networks and Learning Systems, 35(4):5064– 5078, 2022

    2022

  26. [34]

    Multi-task reinforcement learning: a hierar- chical bayesian approach

    Aaron Wilson, Alan Fern, Soumya Ray, and Prasad Tadepalli. Multi-task reinforcement learning: a hierar- chical bayesian approach. InProceedings of the 24th international conference on Machine learning, pages 1015–1022, 2007

    2007

  27. [35]

    Safe deep policy adaptation

    Wenli Xiao, Tairan He, John Dolan, and Guanya Shi. Safe deep policy adaptation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 17286–17292. IEEE, 2024

    2024

  28. [36]

    Motion planning and control for mobile robot navigation using machine learning: a survey.Autonomous Robots, 46(5):569–597, 2022

    Xuesu Xiao, Bo Liu, Garrett Warnell, and Peter Stone. Motion planning and control for mobile robot navigation using machine learning: a survey.Autonomous Robots, 46(5):569–597, 2022

    2022

  29. [37]

    Multi-task reinforcement learning for quadrotors.IEEE Robotics and Automation Letters, 2024

    Jiaxu Xing, Ismail Geles, Yunlong Song, Elie Aljal- bout, and Davide Scaramuzza. Multi-task reinforcement learning for quadrotors.IEEE Robotics and Automation Letters, 2024

    2024

  30. [38]

    Benchmarking reinforcement learning techniques for autonomous navigation

    Zifan Xu, Bo Liu, Xuesu Xiao, Anirudh Nair, and Peter Stone. Benchmarking reinforcement learning techniques for autonomous navigation. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9224–9230. IEEE, 2023

    2023

  31. [39]

    Multi-expert learning of adaptive legged locomotion.Science Robotics, 5(49):eabb2174, 2020

    Chuanyu Yang, Kai Yuan, Qiuguo Zhu, Wanming Yu, and Zhibin Li. Multi-expert learning of adaptive legged locomotion.Science Robotics, 5(49):eabb2174, 2020

    2020

  32. [40]

    Preparing for the unknown: Learning a universal policy with online system identification.Robotics: Science and Systems Proceedings, 2017

    Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk. Preparing for the unknown: Learning a universal policy with online system identification.Robotics: Science and Systems Proceedings, 2017

    2017

  33. [41]

    A survey on multi-task learning.IEEE transactions on knowledge and data engineering, 34(12):5586–5609, 2021

    Yu Zhang and Qiang Yang. A survey on multi-task learning.IEEE transactions on knowledge and data engineering, 34(12):5586–5609, 2021

    2021

  34. [42]

    The ingredients of real-world robotic re- inforcement learning.arXiv preprint arXiv:2004.12570, 2020

    Henry Zhu, Justin Yu, Abhishek Gupta, Dhruv Shah, Kristian Hartikainen, Avi Singh, Vikash Kumar, and Sergey Levine. The ingredients of real-world robotic re- inforcement learning.arXiv preprint arXiv:2004.12570, 2020

    2004 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.