Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulation

Xinrui Wang; Yan Jin

REVIEW 5 major objections 6 minor 29 references

Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulation

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read On a simulated two-block stacking task, a three-stage curriculum with a redesigned reward reaches success in about 1,500 episodes, a 40 percent reduction from direct learning with a roughly 10 percent higher final success rate.

desk verdict A plausible reward-redesign plus curriculum case study on CausalWorld stacking, but the headline 40%/10% numbers are not yet supported because there is no defined convergence criterion, no variance reporting, and the winning curriculum was picked post hoc. read the letter →

arxiv 2505.10522 v1 pith:JBIM2IET submitted 2025-05-15 cs.RO cs.AIcs.LG

Xinrui Wang , Yan Jin This is my paper · ORCID

classification cs.ROcs.AIcs.LG

keywords reinforcementlearningcurriculumknowledgetransferroboticmanipulationrewardfunctiondesigntwo-blockstackingSoftActor-Critic

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the reason agents struggle on a simulated two-block stacking task is not the learning algorithm but the reward function and the absence of a structured knowledge-transfer plan. It proposes KCAC, a knowledge-capture, adaptation, and composition framework that replaces the benchmark's conditional, staged reward with an unconditional sum of dense and sparse terms, and that pre-trains the agent on a grasping task and then a picking task before the full stacking task, choosing when to switch and what learning rate to use from a cosine-similarity measure between reward-component vectors. The central claim is that this combination finishes training in about 1,500 episodes versus 2,500 for direct learning, a 40 percent reduction, while raising the final fractional success by about 10 percent. If correct, reward engineering plus cross-task curricula can substitute for more algorithm-level intervention on complex manipulation tasks.

What carries the argument

The load-bearing object is a binary reward-component vector representing each task: a task's compound reward is written as a sum of eight named terms, and a task is coded by which terms are present (1) or absent (0). Cosine similarity between two such vectors, $\operatorname{Sim}(V_i,V_j)=V_i\cdot V_j/(\|V_i\|\,\|V_j\|)$, is used to choose when to transfer weights and what learning rate to use: low similarity (grasping to stacking, 0.4) recommends early transition and high learning rate, while high similarity (picking to stacking, 0.8) recommends late transition and low learning rate. The second half of the mechanism is the redesigned stacking reward of Equation (3), which removes the indicator-function gates of the benchmark reward so that dense and sparse terms can all be optimized concurrently. These two pieces, unconditional reward plus similarity-guided curriculum, carry the argument.

What would settle it

Run the low-similarity grasping-to-stacking curriculum at learning rate 1e-5: the paper's similarity rule predicts slower convergence than at 1e-4, because low similarity is said to need a high learning rate; a faster convergence at the low rate would falsify the M function. Also, construct two tasks with identical binary reward vectors but very different state and contact dynamics; if transfer between them is poor, the binary vector misses what actually transfers.

Watch

Extended reading notes

Core claim

The paper's central claim is that the KCAC curriculum, with 60 episodes of grasping pretraining, 900 episodes of picking pretraining, and a learning rate of 1e-5 for the stacking stage, learns the two-block stacking task in about 1,500 episodes total, compared with about 2,500 episodes for direct learning on the redesigned reward, a 40 percent reduction in training time with about a 10 percent higher final fractional success. The argument has two parts. First, the benchmark reward, which gates each reward component on conditions that force a fixed ordering of sub-skills, is replaced by an unconditional compound reward that lets the agent accumulate dense and sparse rewards simultaneously; in the paper's experiments this alone activates learning of the top block that the baseline never achieves. Second, knowledge from simpler tasks is transferred through weight initialization, with the transition point and learning rate set according to task similarity: low-similarity grasping-to-stacking benefits from early transition and high learning rate, while high-similarity picking-to-stacking benefits from long pretraining and low learning rate. A three-stage curriculum combines these, cutting the picking pretraining needed from 1,800 to 900 episodes while matching or exceeding the two-stage result.

Load-bearing premise

Everything hinges on the assumption that the presence or absence of shared reward terms, measured by cosine similarity over binary vectors, tells you how much skill transfers between two tasks; if reward magnitudes, state-space overlap, or contact dynamics matter more, the recommended transition timing and learning rates have no reliable basis.

Editorial extensions

If this is right

Reward functions that gate learning on a fixed order of sub-skills can be the main obstacle to solving a manipulation task; removing the gates can activate learning of previously ignored components.
Pairwise task similarity, measured from reward structure alone, can be used to set two curriculum knobs, transition timing and learning rate, without exhaustive sweeps.
A three-stage curriculum can replace a long high-similarity pretraining stage with a chain of shorter low- and medium-similarity stages, reducing total episodes while keeping final success.
The same reward-decomposition-and-transfer recipe applies to other tasks whose reward is a sum of identifiable movement and goal components.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An immediate test of the similarity measure would be to recompute it with reward weights instead of binary flags; if the recommended curriculum order changes, the binary simplification is not neutral.
Since the reported metric is fractional overlap, a stricter binary success threshold might erase part of the 10 percent gain; the 40 percent time saving should be re-checked under a thresholded success definition.
The paper's M and G functions are conceptual, not fitted; with more sub-task pairs and similarity scores, they could become explicit regression models that predict transition timing and learning rate, which the paper itself lists as future work.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The thing to know: this paper is a case study on CausalWorld two-block stacking where the authors redesign the reward function and run a 3-stage grasp-pick-stack curriculum. The specific empirical result—about 1500 episodes vs 2500 for direct learning—is new in that exact form, but it is not a new method; it is a sensible combination of reward shaping, curriculum learning, and transfer. The paper does a few things well. The diagnosis that the original CausalWorld reward forces a strict learning order and that this hurts is plausible, and the redesigned reward (removing conditional gates) is clearly described. The binary reward-component similarity vector is a simple, transparent way to compare tasks, and the qualitative guidance (low similarity -> early transition/high LR, high similarity -> late transition/low LR) is consistent with the reported trends.

The soft spots are real. First, the headline 40%/10% claim is not testable as reported. 'Convergence time' is never defined, no standard deviation or confidence interval is given for the five-seed averages, and the winning curriculum (60 grasp + 900 pick + 540 stack) was selected after scanning many variants, so selection bias is a live threat. The direct-learning baseline is compared without specifying which learning rate was used, and it is compared against modified-reward direct learning, not the original benchmark reward. Second, the M and G functions are defined retroactively from the same experiments that are used to demonstrate them; the similarity measure is based on binary presence of reward terms and ignores reward magnitudes, dynamics, and state-space overlap, so the guidance it produces is a heuristic, not an established principle. Third, no code or data are provided, only 'upon reasonable request,' which limits independent checking.

These are not fatal to the core idea. The trends across the figures (described in the text) are consistent with the claimed effect, and the reward redesign seems genuinely useful for this benchmark. But the paper overstates what is established: it presents heuristics as a 'framework' and a single post-hoc best run as a quantitative result.

Who should read it: researchers working on CausalWorld or on curriculum design for robotic manipulation will find the reward breakdown and the similarity-based timing suggestions worth a look, mainly as inspiration rather than as a citable result. I would not cite the 40%/10% numbers in my own work.

Recommendation: send it to peer review, but with a clear request for error bars, a defined success threshold, a pre-specified or at least fully disclosed curriculum search, and a baseline comparison against the original reward. With those fixes it could be a solid case study; without them, the headline claim should be downgraded.

Referee Report

5 major / 6 minor

Summary. The paper proposes the Knowledge Capture, Adaptation, and Composition (KCAC) framework for cross-task curriculum learning in robotic manipulation, evaluated on a two-block stacking task in CausalWorld. The authors redesign the benchmark reward to remove conditional sequencing, introduce two hand-designed subtasks (grasping and picking), and represent tasks as binary reward-component vectors with cosine similarity. They systematically vary transition timing and learning rate in two-stage curricula and then construct a three-stage curriculum. The central claim is that the best three-stage curriculum (60 episodes grasping, 900 episodes picking, then stacking) completes stacking in about 1,500 episodes versus 2,500 episodes for direct learning with the redesigned reward, a 40% reduction in training time with a 10% higher final fractional success.

Significance. If substantiated, the results would provide a useful demonstration that a compound reward function without hard sequencing, combined with a staged curriculum, can accelerate SAC training on a contact-rich manipulation task. The paper also makes a design-oriented contribution by cataloguing how transition timing and learning rate interact with task similarity. However, the evidence as presented is not sufficient: the headline 40%/10% gains are based on five-seed averages with no variance, an undefined convergence criterion, a post hoc selected curriculum, and an unspecified baseline learning rate. The similarity measure in Eq. (7) is asserted rather than validated. The reward-redesign observation itself is interesting but is confounded by an initialization change. The work is therefore promising but currently falls short of a solid quantitative claim.

major comments (5)

[5.3; Fig. 15] The headline claim of a 40% training-time reduction rests on the assertion that the best 3-stage curriculum 'completes training in 1500 episodes' while direct stacking 'requires 2500 episodes.' No convergence threshold is defined, and all curves are averages over five seeds with no standard deviation or confidence band. Because the winning curriculum was selected after inspecting many two- and three-stage configurations with three learning rates, the comparison is vulnerable to selection bias. Please report per-seed curves or variance, define the success-rate threshold used to determine convergence, and state whether the baseline learning rate was the most favorable one.
[3.1; Eq. (3); Figs. 4-5] The redesigned reward is evaluated after initializing block_1 at its goal position, which deactivates the first condition of the original CausalWorld reward. This changes the task itself, so the large improvement over the baseline in Figures 4-5 may reflect a simpler task rather than better knowledge capture. Please compare against the unmodified benchmark task, or clearly present the modified task as a separate benchmark and temper the comparison.
[5.3] For the direct-stacking baseline in Figure 15, the learning rate is not stated. Section 5.2 shows that learning rate strongly affects curriculum performance; without knowing which setting was used for direct learning, the 40% improvement cannot be attributed to the curriculum. Please report the baseline learning rate and, ideally, the best direct-learning result across the three settings.
[4.2; Eq. (7)] The guidance function M is based on binary cosine similarity between reward-component vectors. The manuscript asserts that relative component contributions are comparable when tasks are implemented appropriately, but no evidence is given. Since Eq. (7) drives the transition-timing and learning-rate recommendations in Section 5.2 (e.g., low similarity -> early transition and high learning rate), this assumption is load-bearing. Please validate it with transfer experiments or present M as a descriptive heuristic rather than a predictive principle.
[3.2; Table 1] The KCAC procedure is presented as a framework, but functions G and M are only specified after the results are known, and no algorithm for constructing them is given. As presented, the framework does not offer a predictive rule for new tasks; Section 6 itself states that future work is needed for mathematical formulation of M. The paper should clarify whether KCAC is a conceptual framework or an implementable method.

minor comments (6)

[Title] The title typo 'ADAPTAION' should be corrected to 'ADAPTATION'.
[Fig. 15 caption] The caption word 'Comparision' should be 'Comparison'.
[4.3] The phrase 'In this session' should be 'In this section'.
[References] References [29] and [31] duplicate the same Cooper 1999 citation, and references [30] and [33] duplicate the same Amini et al. citation; please consolidate them.
[Eq. (1)] Equation (1) contains garbled symbols (e.g., '$&’!()*(+!<,’&-’' and '𝟏') that render incorrectly; please ensure all mathematical expressions are typeset cleanly.
[Figures 6-14] The curve labels such as 'stack2' and 'grasp2_60_curri_lr_1e-5' are not defined in the captions; please explain all legend entries explicitly.

Circularity Check

1 steps flagged · score 4.0 of 10

The KCAC guidance functions M and G are defined post hoc from the same 2-/3-stage experiments used to demonstrate them; the headline 40% empirical comparison itself is not circular.

other [Section 4, KCAC framework setup, paragraph beginning 'Based on these findings...']
"Based on these findings, the function 𝑀(<𝑆𝑁>) is conceptually defined to determine optimal transition timing and learning parameter selection. 3-stage curriculums are developed using prior conclusions to further improve the learning efficiency. The results are compared with 2-stage curriculum candidates, leading to the conceptual definition of function 𝐺(𝑆), which contributes to curriculum generation."

The framework's two core functions are not derived from an independent theory; they are fitted to the same empirical curves in Figures 6–14 that are then offered as the evidence for the framework. The guidance attributed to M — low similarity implies early transition/high learning rate, high similarity implies late transition/low learning rate, medium similarity implies an intermediate timing — is a verbal restatement of the observed slopes and saturation points of those curves. Likewise, G is 'conceptually defined' only after comparing 2-stage and 3-stage candidates and selecting the 60+900 grasping/picking schedule post hoc. The derivation chain is experiments → M/G → conclusions, so the framework's recommendations reduce to the data used to build them.

full rationale

The paper's central empirical comparison (1500 vs. 2500 episodes, a 40% reduction, with roughly 10% higher final fractional success) is a direct measurement on CausalWorld, not the output of a derived formula, so it is not circular by construction. The reward redesign in Eq. (3), the sub-task rewards in Eqs. (4)–(5), and the cosine-similarity values in Eq. (7) are modeling and descriptive choices; none of these definitions alone forces the observed learning curves. The one genuine circularity concern is the status of M(<S_N>) and G(S): Section 4 states that M is 'conceptually defined' based on the 2-stage findings and that G is defined after comparing curriculum candidates. Thus the framework's transition-timing and learning-rate guidance is a summary of the same curves in Figures 6–14 rather than an independent prediction; the 'key design parameters' reproduce their own empirical inputs. This is partial because the 40%/10% result would still stand or fall on the plotted curves regardless of the M/G naming. Self-citations [3], [6], [10], and [11] are not scored as load-bearing here: the current experiments independently exhibit the transition-timing effects, and [3] is prior empirical work on a different task. Missing variance and an undefined convergence criterion are correctness and statistical concerns, not circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The framework rests on hand-engineered reward weights, experimentally selected transition timings and learning rates, and the assumption that binary reward-vector similarity predicts transferability. No new physical entities are postulated; KCAC is a label for existing reward shaping plus curriculum transfer components.

free parameters (5)

Dense reward scale coefficients in Equation 3 (750, 250, 125, 0.005) = 750, 250, 125, 0.005
Hand-adjusted weights for end-effector distance, block distance, and velocity penalty; the paper states reward engineering was used to emphasize block_2 manipulation.
Sparse reward weights in Equation 3 (0.5 for R_sparse1, 1 for R_sparse2) = 0.5, 1
Adjusted to guide the agent toward block_2 manipulation; no principled derivation is given.
Grasping subtask reward design choices in Equation 4 = hand-designed min(1, distance ratio) term
The paper says the grasping task uses a slightly different reward function based on experimental results, so multiple terms are hand-engineered for that subtask.
Transition timing values (60, 300, 360, 900, 1200, 1800 episodes) = varies by curriculum
Chosen from scanning a grid of pretraining durations; the headline three-stage curriculum uses 60 and 900 episodes. These are empirical selections, not predicted values.
Learning rate settings (1e-4, 5e-5, 1e-5) and associated SAC hyperparameters = Table 2 settings
Selected as experimental factors; the paper concludes the optimal learning rate depends on task similarity, making the learning rate part of the method rather than a derived constant.

assumptions (5)

domain assumption CausalWorld simulator is a valid proxy for real robotic manipulation.
All claims are evaluated only in simulation; transfer to physical robots is not demonstrated.
domain assumption Soft Actor-Critic with Stable-Baselines3 is a sound underlying RL implementation.
The paper relies on this implementation without verifying its correctness or comparing against other RL backbones.
ad hoc to paper Binary reward-component vectors and cosine similarity capture task transferability.
Introduced in Section 4.2; no independent validation that reward magnitudes or environment dynamics are irrelevant.
ad hoc to paper Hand-designed grasping and picking subtasks are useful prerequisites for stacking and compose into an efficient curriculum.
Justified by experiments after the fact; no generative principle for G(S) is provided.
domain assumption Conclusions from the authors' prior single-block curriculum study [3] transfer to the two-block stacking task.
Used to motivate transition-timing choices; self-cited without independent replication.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulation." pith.science (2026). https://pith.science/paper/JBIM2IET

@misc{pith2026250510522,
  author       = {Pith},
  title        = {Pith review of: Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JBIM2IET}},
  note         = {Machine review of arXiv:2505.10522}
}

read the original abstract

Reinforcement learning (RL) has demonstrated remarkable potential in robotic manipulation but faces challenges in sample inefficiency and lack of interpretability, limiting its applicability in real world scenarios. Enabling the agent to gain a deeper understanding and adapt more efficiently to diverse working scenarios is crucial, and strategic knowledge utilization is a key factor in this process. This paper proposes a Knowledge Capture, Adaptation, and Composition (KCAC) framework to systematically integrate knowledge transfer into RL through cross-task curriculum learning. KCAC is evaluated using a two block stacking task in the CausalWorld benchmark, a complex robotic manipulation environment. To our knowledge, existing RL approaches fail to solve this task effectively, reflecting deficiencies in knowledge capture. In this work, we redesign the benchmark reward function by removing rigid constraints and strict ordering, allowing the agent to maximize total rewards concurrently and enabling flexible task completion. Furthermore, we define two self-designed sub-tasks and implement a structured cross-task curriculum to facilitate efficient learning. As a result, our KCAC approach achieves a 40 percent reduction in training time while improving task success rates by 10 percent compared to traditional RL methods. Through extensive evaluation, we identify key curriculum design parameters subtask selection, transition timing, and learning rate that optimize learning efficiency and provide conceptual guidance for curriculum based RL frameworks. This work offers valuable insights into curriculum design in RL and robotic learning.

Figures

Figures reproduced from arXiv: 2505.10522 by the authors.

**Figure 6.** FIGURE 6 [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

**Figure 7.** FIGURE 7 [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗

Figures from the paper (1 more)

**Figure 9.** Figure 9: FIGURE 9 [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 23 canonical work pages

[1]

of Aerospace & Mechanical Engineering University of Southern California Los Angeles, USA xinruiw@usc.edu Yan Jin* Dept

1 © 2025 by ASME KNOWLEDGE CAPTURE, ADAPTAION AND COMPOSITION (KCAC): A FRAMEWORK FOR CROSS-TASK CURRICULUM LEARNING IN ROBOTIC MANIPULATION Xinrui Wang Dept. of Aerospace & Mechanical Engineering University of Southern California Los Angeles, USA xinruiw@usc.edu Yan Jin* Dept. of Aerospace & Mechanical Engineering University of Southern California Los An...

work page 2025
[2]

In the domain of robotic manipulation, machine learning has significantly advanced its capabilities in handling objects and executing complex tasks

INTRODUCTION As science and technology advance, the landscape of the engineering work process is continuously extended and revolutionized, which can be initiated and carried out by robotic systems. In the domain of robotic manipulation, machine learning has significantly advanced its capabilities in handling objects and executing complex tasks. Machine le...

work page 2025
[3]

Reinforcement learning is one of the solutions, allowing agents to understand and optimize the process through continuous interaction with their environment [14-16]

RELATED WORK 2.1 Reinforcement Learning Engineering design is a multidisciplinary topic that requires careful optimization to balance performance and efficiency across various constraints [8, 9]. Reinforcement learning is one of the solutions, allowing agents to understand and optimize the process through continuous interaction with their environment [14-...

work page 2025
[4]

#$%=𝟏&!((

is defined as follows. 𝑅!"#$%=𝟏&!((",*),-.-/(−750∆"(𝑜0,𝑒)−250∆"(𝑜0,𝑔0))+𝟏&!((",*)1-.-/2−750∆"(𝑜/,𝑒)−25034𝑜/,2"−𝑔/,2"4−4𝑜/,2"30−𝑔/,2"45−𝟏(#,%!34#,%!,-125∆"3𝑜/,5,6,𝑔/,5,657+ 𝑅4(#7"+ 𝑅4(#7#+0.005∗‖𝑣"−𝑣"30‖ (1) 4 © 2025 by ASME where 𝑒, 𝑜, 𝑔, and 𝑣 represent end-effector position, block position, goal block position, and joint velocity, respectively, and 𝑑" r...

work page 2025
[5]

(𝑜/,𝑒)−250∆

The objective of the picking task is to grasp block_2 and lift it vertically without any horizontal movement while avoiding collisions with block_1. As shown in Figure 3, block_2 should fully overlap with goal_block_2 at the correct height in the stacking task. The reward functions for these two sub-tasks are derived from the stacking task reward function...

work page 2025
[6]

The default learning rate of 1e-4, as provided by the baseline, is used for all pre-training phases in the curriculum

For clarity, we denote these settings as lr_1e-4, lr_5e-5, and lr_1e-5, respectively. The default learning rate of 1e-4, as provided by the baseline, is used for all pre-training phases in the curriculum. 7 © 2025 by ASME Table

work page 2025
[7]

Our model significantly improves the top block's fractional success regardless of the conditions and causing learning sequence

FIGURE 4: Fractional successful rate of top block FIGURE 5: Fractional successful rate of two blocks From Figure 4, we can see that the top block has not been activated in the baseline, which is the main reason for the poor performance of the baseline model. Our model significantly improves the top block's fractional success regardless of the conditions a...

work page 2025
[8]

Note that only the last stage learning (stacking task) is shown in the figures since the fractional success of target tasks cannot evaluate previous stage tasks. FIGURE 6: Grasping-stacking curriculum with lr_1e-5 learning process FIGURE 7: Grasping-stacking curriculum with lr_5e-5 learning process FIGURE 8: Grasping-stacking curriculum with lr_1e-4 learn...

work page 2025

Show all 29 references

[9]

This task requires the agent to differentiate between blocks and operate with higher precision to avoid collisions while successfully constructing the assembly

CONCLUSION AND FUTURE WORK In this research, we use the two-block stacking task as a case study, extending our investigation from a single-block task to a more complex two-block assembly task. This task requires the agent to differentiate between blocks and operate with higher...

2025
[10]

Causalworld: A robotic manipulation benchmark for causal structure and transfer learning

Ahmed, Ossama, et al. "Causalworld: A robotic manipulation benchmark for causal structure and transfer learning." arXiv preprint arXiv:2010.04296 (2020)

2020 arXiv
[11]

Robust multi-agent reinforcement learning with state uncertainty

He, Sihong, et al. "Robust multi-agent reinforcement learning with state uncertainty." arXiv preprint arXiv:2307.16212 (2023)

2023 arXiv
[12]

Enhancing Efficiency in Collision Avoidance: A Study on Transfer Reinforcement Learning in Autonomous Ships’ Navigation

Wang, Xinrui, and Yan Jin. "Enhancing Efficiency in Collision Avoidance: A Study on Transfer Reinforcement Learning in Autonomous Ships’ Navigation." ASME Open Journal of Engineering 3 (2024)

2024
[13]

Simultaneous Thermal-Electrical Cloak and Camouflage via Level-Set-Based Topology Optimization

Xu, Xiaoqiang, and Shikui Chen. "Simultaneous Thermal-Electrical Cloak and Camouflage via Level-Set-Based Topology Optimization." Journal of Mechanical Design (2025): 1-16

2025
[14]

Topology optimization of multimaterial thermoelectric structures

Xu, Xiaoqiang, et al. "Topology optimization of multimaterial thermoelectric structures." Journal of Mechanical Design 143.1 (2021): 011705

2021
[15]

Explainable reinforcement learning: A survey and comparative review

Milani, Stephanie, et al. "Explainable reinforcement learning: A survey and comparative review." ACM Computing Surveys 56.7 (2024): 1-36

2024
[16]

Continuous control with deep reinforcement learning

Lillicrap, Timothy P., et al. "Continuous control with deep reinforcement learning." arXiv preprint arXiv:1509.02971 (2015). 12 © 2025 by ASME

2015 arXiv
[17]

Human-level control through deep reinforcement learning

Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." nature 518.7540 (2015): 529-533

2015
[18]

Data-driven distributionally robust optimization for vehicle balancing of mobility-on-demand systems

Miao, Fei, et al. "Data-driven distributionally robust optimization for vehicle balancing of mobility-on-demand systems." ACM Transactions on Cyber-Physical Systems 5.2 (2021): 1-27

2021
[19]

Explicable reward design for reinforcement learning agents

Devidze, Rati, et al. "Explicable reward design for reinforcement learning agents." Advances in neural information processing systems 34 (2021): 20118-20131

2021
[20]

Curriculum learning for reinforcement learning domains: A framework and survey

Narvekar, Sanmit, et al. "Curriculum learning for reinforcement learning domains: A framework and survey." Journal of Machine Learning Research 21.181 (2020): 1-50

2020
[21]

Source task creation for curriculum learning

Narvekar, Sanmit, et al. "Source task creation for curriculum learning." Proceedings of the 2016 international conference on autonomous agents & multiagent systems

2016
[22]

Curricullm: Automatic task curricula design for learning complex robot skills using large language models

Ryu, Kanghyun, et al. "Curricullm: Automatic task curricula design for learning complex robot skills using large language models." arXiv preprint arXiv:2409.18382 (2024)

2024 arXiv
[23]

Proximal curriculum for reinforcement learning agents

Tzannetos, Georgios, et al. "Proximal curriculum for reinforcement learning agents." arXiv preprint arXiv:2304.12877 (2023)

2023 arXiv
[24]

Residual learning from demonstration: Adapting dmps for contact-rich manipulation

Davchev, Todor, et al. "Residual learning from demonstration: Adapting dmps for contact-rich manipulation." IEEE Robotics and Automation Letters 7.2 (2022): 4488-4495

2022
[25]

Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models

Chen, Annie S., et al. "Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models." arXiv preprint arXiv:2407.02666 (2024)

2024 arXiv
[26]

$\mathrm {R}^{3} $: On-Device Real-Time Deep Reinforcement Learning for Autonomous Robotics

Li, Zexin, et al. "$\mathrm {R}^{3} $: On-Device Real-Time Deep Reinforcement Learning for Autonomous Robotics." 2023 IEEE Real-Time Systems Symposium (RTSS). IEEE,

2023
[27]

A critical review of Knowledge-Based Engineering: An identification of research challenges

Verhagen, Wim JC, et al. "A critical review of Knowledge-Based Engineering: An identification of research challenges." Advanced Engineering Informatics 26.1 (2012): 5-15

2012
[33]

Machine learning model towards evaluating data gathering methods in manufacturing and mechanical engineering

Amini, Mahyar, Koosha Sharifani, and Ali Rahmani. "Machine learning model towards evaluating data gathering methods in manufacturing and mechanical engineering." International Journal of Applied Science and Engineering Research 15.2023 (2023): 349-362

2023
[35]

A recurrent neural network architecture for failure prediction in deep drawing sensory time series data

Meyes, Richard, et al. "A recurrent neural network architecture for failure prediction in deep drawing sensory time series data." Procedia Manufacturing 34 (2019): 789-797

2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

of Aerospace & Mechanical Engineering University of Southern California Los Angeles, USA xinruiw@usc.edu Yan Jin* Dept

1 © 2025 by ASME KNOWLEDGE CAPTURE, ADAPTAION AND COMPOSITION (KCAC): A FRAMEWORK FOR CROSS-TASK CURRICULUM LEARNING IN ROBOTIC MANIPULATION Xinrui Wang Dept. of Aerospace & Mechanical Engineering University of Southern California Los Angeles, USA xinruiw@usc.edu Yan Jin* Dept. of Aerospace & Mechanical Engineering University of Southern California Los An...

work page 2025

[2] [2]

In the domain of robotic manipulation, machine learning has significantly advanced its capabilities in handling objects and executing complex tasks

INTRODUCTION As science and technology advance, the landscape of the engineering work process is continuously extended and revolutionized, which can be initiated and carried out by robotic systems. In the domain of robotic manipulation, machine learning has significantly advanced its capabilities in handling objects and executing complex tasks. Machine le...

work page 2025

[3] [3]

Reinforcement learning is one of the solutions, allowing agents to understand and optimize the process through continuous interaction with their environment [14-16]

RELATED WORK 2.1 Reinforcement Learning Engineering design is a multidisciplinary topic that requires careful optimization to balance performance and efficiency across various constraints [8, 9]. Reinforcement learning is one of the solutions, allowing agents to understand and optimize the process through continuous interaction with their environment [14-...

work page 2025

[4] [4]

#$%=𝟏&!((

is defined as follows. 𝑅!"#$%=𝟏&!((",*),-.-/(−750∆"(𝑜0,𝑒)−250∆"(𝑜0,𝑔0))+𝟏&!((",*)1-.-/2−750∆"(𝑜/,𝑒)−25034𝑜/,2"−𝑔/,2"4−4𝑜/,2"30−𝑔/,2"45−𝟏(#,%!34#,%!,-125∆"3𝑜/,5,6,𝑔/,5,657+ 𝑅4(#7"+ 𝑅4(#7#+0.005∗‖𝑣"−𝑣"30‖ (1) 4 © 2025 by ASME where 𝑒, 𝑜, 𝑔, and 𝑣 represent end-effector position, block position, goal block position, and joint velocity, respectively, and 𝑑" r...

work page 2025

[5] [5]

(𝑜/,𝑒)−250∆

The objective of the picking task is to grasp block_2 and lift it vertically without any horizontal movement while avoiding collisions with block_1. As shown in Figure 3, block_2 should fully overlap with goal_block_2 at the correct height in the stacking task. The reward functions for these two sub-tasks are derived from the stacking task reward function...

work page 2025

[6] [6]

The default learning rate of 1e-4, as provided by the baseline, is used for all pre-training phases in the curriculum

For clarity, we denote these settings as lr_1e-4, lr_5e-5, and lr_1e-5, respectively. The default learning rate of 1e-4, as provided by the baseline, is used for all pre-training phases in the curriculum. 7 © 2025 by ASME Table

work page 2025

[7] [7]

Our model significantly improves the top block's fractional success regardless of the conditions and causing learning sequence

FIGURE 4: Fractional successful rate of top block FIGURE 5: Fractional successful rate of two blocks From Figure 4, we can see that the top block has not been activated in the baseline, which is the main reason for the poor performance of the baseline model. Our model significantly improves the top block's fractional success regardless of the conditions a...

work page 2025

[8] [8]

Note that only the last stage learning (stacking task) is shown in the figures since the fractional success of target tasks cannot evaluate previous stage tasks. FIGURE 6: Grasping-stacking curriculum with lr_1e-5 learning process FIGURE 7: Grasping-stacking curriculum with lr_5e-5 learning process FIGURE 8: Grasping-stacking curriculum with lr_1e-4 learn...

work page 2025