Pith. sign in

REVIEW 4 major objections 5 minor 39 references

Unsupervised Meta-Testing with Conditional Neural Processes for Hybrid Meta-Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UMCNP adapts to an unseen test task from a single real rollout plus 24 CNP-generated rollouts, matching an oracle that uses 25 real rollouts, without reward signals during meta-testing.

desk verdict Solid incremental combination worth a referee; the headline sample-efficiency claim holds in two benchmarks but the Walker evidence is too weak to carry the load. read the letter →

arxiv 2506.04399 v1 pith:WASPV3B6 submitted 2025-06-04 cs.LG cs.AIcs.RO

Suzan Ece Ada , Emre Ugur This is my paper
classification cs.LGcs.AIcs.RO
keywords meta-reinforcementlearningconditionalneuralprocessesunsupervisedmeta-testingtaskinferencesampleefficiencymodel-basedreinforcementno-rewardmeta
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes UMCNP, a meta-reinforcement-learning method for settings where the reward signal is unavailable during meta-testing and the test task's transition dynamics are hidden. UMCNP separates task inference from policy-gradient meta-training: it reuses previously collected meta-training transitions, offline, to train a conditional neural process that predicts next states from state-action queries conditioned on a latent summary of a task. At meta-test time, a single real rollout from the unseen task is encoded into that latent summary, and the CNP generates additional rollouts by interacting with the learned dynamics model instead of the environment. The paper claims this achieves adaptation performance close to an oracle that collects 25 real rollouts, using only one real rollout plus 24 generated rollouts, in point-agent, cartpole-with-sensor-bias, and walker benchmarks.

What carries the argument

The central object is the conditional neural process dynamics model, an encoder-decoder network. Transition tuples $(s, a, s')$ from a task are encoded by a shared encoder and averaged into a permutation-invariant latent vector $r = \frac{1}{k}\sum_k g_{\theta_E}(s_k, a_k, s'_k)$; the decoder maps the state-action query concatenated with $r$ to the mean and variance of the next-state distribution, $\mu_q, \sigma_q = f_{\theta_D}([s_q, a_q] \oplus r)$. At test time $r$ is computed from one real rollout, the meta-policy queries the decoder autoregressively to produce rollouts, and a NORML pseudo-advantage function $A_\psi(s, a, s')$ guides the policy-gradient inner-loop update using the combined real and generated transitions.

What would settle it

For the 2D point agent, construct a pair of rotation parameters $\omega_1$ and $\omega_2$ whose trajectories under the meta-policy coincide for the first episode; if UMCNP's latent representation is insensitive to $\omega$ and its post-update reward on both tasks is no better than no adaptation, then the claim that one rollout suffices is refuted.

Watch

Extended reading notes

Core claim

The central claim is that a latent vector computed by averaging CNP encodings of the transition tuples in one rollout is enough to represent the unseen task's dynamics, and that rollouts generated from this model can replace almost all online interaction during meta-testing. On the 2D point agent, adaptation with 24 generated rollouts plus one real rollout reaches -1.02 ± 0.09 post-update reward, versus ORACLE's -1.20 ± 0.12 with 25 real rollouts, while NORML with one real rollout reaches -11.49 ± 1.44. The same pattern holds in cartpole with a hidden angle-sensor bias and in a Walker-2D agent with randomized dynamics, where UMCNP also uses a single real rollout to match or beat the 25-rollout oracle.

Load-bearing premise

The load-bearing premise is that one rollout from the unseen task contains enough information for the averaged CNP latent vector to pin down the task's transition dynamics, so the rollouts it generates point the policy update in a useful direction.

Editorial extensions

If this is right

  • An agent can adapt to a new task with a single real rollout rather than 25, reducing costly and potentially dangerous online interaction in no-reward settings.
  • Task inference is decoupled from policy-gradient meta-training, so the dynamics model can be trained offline from already-collected transitions without extra meta-training samples and without access to task parameters or rewards.
  • Because the CNP is permutation invariant and agnostic to the number of encoded transitions, the same trained model supports adaptation with 5, 10, or 50 transitions without retraining.
  • The generated rollouts can be combined with the pseudo-advantage function to perform a policy-gradient inner-loop update, so no reward signal or environment model parameter is needed at test time.
  • On the evaluated benchmarks, UMCNP not only matches the oracle's performance but sometimes exceeds it, suggesting that model-generated samples can be less noisy than real samples for meta-adaptation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the single-rollout latent is identifying the hidden dynamics only up to the transitions seen, then an active query-selection extension could choose the most informative state-action pairs to include in the rollout, further reducing the real samples needed; the paper does not test this.
  • The same offline CNP module could likely be attached post hoc to any policy-gradient meta-RL method that has a stored set of transitions, since the decoupling means no meta-training retraining is required; this follows from the architecture but is not demonstrated.
  • The observation that UMCNP sometimes beats the oracle suggests a testable hypothesis: model-generated rollouts act as a variance-reducing regularizer for the meta-update. An ablation that replaces generated rollouts one-for-one with real oracle rollouts, holding the update rule fixed, would separate the effect of sample volume from the effect of generation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes UMCNP, a meta-RL method for unsupervised meta-testing where the reward is unavailable. A Conditional Neural Process is trained offline on transitions collected during NORML meta-training. At meta-test time, a single real rollout is encoded into a latent vector r (Eq. 5); the decoder then autoregressively generates rollouts, which are combined with the real rollout and used to adapt the meta-policy through NORML's pseudo-advantage function. Experiments are reported on a 2D point agent with unknown force field, a cartpole with sensor bias, and a Walker-2D with randomized dynamics. The central claim is that UMCNP matches the performance of an ORACLE that uses 25 real rollouts while only requiring 1 real rollout, e.g., -1.02 ± 0.09 vs -1.20 ± 0.12 on the point agent (Table I).

Significance. If the central claim holds, the paper offers a practical sample-efficiency improvement for meta-RL in no-reward settings, reducing online interaction from 25 rollouts to a single rollout by reusing offline meta-training data. The approach of training a CNP on PPG-RL trajectories and using it to generate adaptation rollouts is conceptually clean and, unlike some prior task-inference methods, does not require reward or task parameters at meta-test time. The paper also reports experiments with 5 seeds and 95% confidence intervals, which is a strength. However, the significance is contingent on evidence that the learned latent actually identifies the test task and that the observed gains are due to generated-rollout quality rather than to confounding factors.

major comments (4)
  1. [Section IV-B, Eq. (5)] The method's central assumption is that the mean latent vector r, computed by averaging encodings of a single test rollout's transitions, is sufficient to identify the unseen transition function so that decoder rollouts are on-task. The paper provides no direct evidence for this: it never reports whether r correlates with the hidden task parameter (e.g., ω in the point agent), whether the decoder's next-state predictions are accurate on true test transitions, or whether the generated rollouts resemble true dynamics under the meta-policy. Without such evidence, the Table I result could be obtained even if the model ignores r and generates task-averaged or generic rollouts. Please add an analysis or ablation that directly validates the latent/decoder, such as measuring prediction error of generated rollouts against ground-truth transitions, comparing against a model with r replaced by a constant or by a task-averaged encoding, and checking if r from different test tasks separates in latent space.
  2. [Section V-A, Table I] The headline comparison confounds the effect of generated rollouts with the total number of rollouts used in the policy-gradient update. UMCNP uses 24 generated + 1 real rollout, while NORML uses only 1 real rollout; ORACLE uses 25 real rollouts. The improvement over NORML could be due solely to having more trajectories (or more gradient steps) rather than to the task-specific quality of the generated data. Please include a control where the same adaptation procedure is run with 25 real rollouts (e.g., ORACLE is one such control, but report the comparison in the same table and also test a version of UMCNP with 24 rollouts generated from a mismatched or random dynamics model). This would isolate whether the CNP's task inference contributes beyond added data volume.
  3. [Section V-C, Table II (Walker-2D)] The Walker-2D results do not support the claim of improved performance. UMCNP (1844.62 ± 338.80) and NORML (1608.97 ± 316.83) have heavily overlapping 95% confidence intervals, as do 10N UMCNP (1684.56 ± 355.71) and 10N NORML (1566.38 ± 308.84). The conclusion in Section VI that UMCNP shows 'better performance in walker environments' is not supported by the reported statistics. Please report per-task results or effect sizes, and either temper the claim to 'comparable' or provide additional evidence (e.g., paired analyses across the 10 test tasks).
  4. [Section IV-B, adaptation step] The generated rollouts are fed into the NORML pseudo-advantage network Aψ, which was trained on real transitions. If the generated transitions are off-distribution, Aψ may output unreliable advantage estimates that steer the policy update incorrectly. The paper does not analyze this distribution shift, for example by comparing advantage magnitudes on generated versus real rollouts or by ablating the pseudo-advantage (e.g., using a simple reward-free surrogate). Adding such an analysis would strengthen the claim that the adaptation mechanism is sound.
minor comments (5)
  1. [Abstract and Contributions] The abstract states 'requires only a single rollout from the unseen test task compared to 25 rollouts used in prior works,' but in the experiments NORML is evaluated with a single rollout; the 25-rollout comparison is against the ORACLE version of NORML. Please clarify this wording to avoid implying that prior NORML papers require 25 rollouts.
  2. [Section V-A] UMCNP-ADV is introduced in the point-agent section and in Table I, but the reader is not told how it is trained until later in the paragraph; please define it explicitly in the experimental setup before presenting results.
  3. [Section V-C] The labels '10N UMCNP' and '10N NORML' are ambiguous: does '10N' mean 10 transitions from a single rollout, 10 rollouts, or 10 transitions total? Please state the exact number of rollout trajectories and transitions used in the Walker experiments.
  4. [Appendix] The architecture description says '128-dimensional representation and two hidden layers, each with 128 units'; please clarify whether the 128-dimensional representation is the latent r dimension or the encoder/decoder hidden size, as this affects reproducibility.
  5. [Figure 6(b)] The caption for Figure 6(b) is incomplete: 'Overall performance compared with baselines with 95% Confidence Intervals' does not explain what the bars or x-axis represent. Please reference the task-parameter axis or describe the aggregation used.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: UMCNP's sample-efficiency result is an empirical claim supported by an independently trained dynamics model; self-citations are not load-bearing.

full rationale

The paper's claimed derivation is empirical rather than definitional. UMCNP first trains the CNP encoder/decoder offline on transitions collected during NORML meta-training (Eq. 5-6), then at meta-test encodes one real rollout into r, autoregressively generates 24 rollouts through the learned dynamics model, and uses the pre-trained NORML pseudo-advantage Aψ to perform one policy-gradient update. The reported result (Table I: UMCNP -1.02 ± 0.09 vs ORACLE -1.20 ± 0.12) is not equal to any input by construction: the CNP is not fitted to the test-task post-update reward, Aψ is not fitted to test-task transitions, and the evaluation reward is measured on the true environment. The paper's self-citations ([27], [28], [29], [32]) appear only in related-work or definitional context and are not used to justify the central sample-efficiency claim. The latent-summary assumption (one rollout must identify the hidden task for Eq. 5 to generate on-task rollouts) is a legitimate empirical risk, but a missing-evidence concern is not circularity. No equation or fitted parameter is renamed as a prediction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method contributes no new physical entities. The central empirical claim rests on four domain assumptions about task distribution, latent sufficiency, pseudo-advantage transfer, and compounding-error tolerance, all unmeasured in the paper. The free parameters are mostly design choices without sensitivity analysis.

free parameters (4)
  • Number of CNP-generated rollouts used for adaptation = 24 (point agent), 5 or 50 transitions (cartpole), 10 transitions (Walker)
    Chosen per environment; the paper reports no sweep or ablation, and the sample-efficiency gain is directly tied to this number.
  • CNP training data selection = last 10% of meta-training trajectories
    Stated in the Appendix without justification; this selection could materially affect the learned dynamics model.
  • CNP architecture and training hyperparameters = latent dim 128, hidden units 128, batch size 32, learning rate 0.0001, 500k iterations
    Standard choices, but no sensitivity analysis is provided; central results could depend on these settings.
  • Context set size for task inference = one rollout (point, Walker), 5 or 50 transitions (cartpole)
    The method's core claim is that a single rollout suffices; this is a design choice, not derived.
assumptions (5)
  • domain assumption Test tasks are drawn from the same task distribution p(T) used in meta-training.
    Invoked in Section III-B and needed for the CNP latent representation to transfer to unseen tasks.
  • domain assumption The mean of encoded context transitions is a sufficient statistic for the hidden transition dynamics parameter.
    Eq. 5 computes r as the average of encoder outputs; the paper does not prove or measure that this average disambiguates the task.
  • domain assumption The NORML pseudo-advantage function Apsi, trained on meta-training data, provides a useful gradient signal when applied to tuples generated by the CNP model.
    Adaptation in Section IV-B scores both real and generated rollouts with the same Apsi; miscalibration on generated states would corrupt the policy update.
  • domain assumption Autoregressive rollout through the learned dynamics model does not accumulate enough error to mislead adaptation.
    Fig. 2 constructs full trajectories by feeding predicted next states back into the decoder; no error analysis is provided.
  • domain assumption The meta-policy's stochasticity is sufficient to generate diverse and informative rollouts from the learned dynamics model.
    The paper relies on PPO stochasticity for exploration; no diversity or coverage measure is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unsupervised Meta-Testing with Conditional Neural Processes for Hybrid Meta-Reinforcement Learning." pith.science (2026). https://pith.science/paper/WASPV3B6

@misc{pith2026250604399,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Meta-Testing with Conditional Neural Processes for Hybrid Meta-Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WASPV3B6}},
  note         = {Machine review of arXiv:2506.04399}
}
read the original abstract

We introduce Unsupervised Meta-Testing with Conditional Neural Processes (UMCNP), a novel hybrid few-shot meta-reinforcement learning (meta-RL) method that uniquely combines, yet distinctly separates, parameterized policy gradient-based (PPG) and task inference-based few-shot meta-RL. Tailored for settings where the reward signal is missing during meta-testing, our method increases sample efficiency without requiring additional samples in meta-training. UMCNP leverages the efficiency and scalability of Conditional Neural Processes (CNPs) to reduce the number of online interactions required in meta-testing. During meta-training, samples previously collected through PPG meta-RL are efficiently reused for learning task inference in an offline manner. UMCNP infers the latent representation of the transition dynamics model from a single test task rollout with unknown parameters. This approach allows us to generate rollouts for self-adaptation by interacting with the learned dynamics model. We demonstrate our method can adapt to an unseen test task using significantly fewer samples during meta-testing than the baselines in 2D-Point Agent and continuous control meta-RL benchmarks, namely, cartpole with unknown angle sensor bias, walker agent with randomized dynamics parameters.

Figures

Figures reproduced from arXiv: 2506.04399 by the authors.

Figure 2
Figure 2. Meta-testing procedure of our proposed method UMCNP. The meta [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Test task indices 2000, 2500, 3000, 3500, 4000, and 4500 correspond to task parameter [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Point agent plot shows the post-update cumulative reward (sum of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Visualization of Cartpole with Unknown Angle Sensor Bias En [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: (a) Post-update cumulative Reward (y-axis) in test tasks compared [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Violin plots of post-update cumulative reward from unseen 10 [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 28 canonical work pages

  1. [1]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    arXiv 2017

  2. [2]

    Playing atari with deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wier- stra, and M. A. Riedmiller, “Playing atari with deep reinforcement learning,” CoRR, vol. abs/1312.5602, 2013

    arXiv 2013

  3. [3]

    A survey of meta-reinforcement learning,

    J. Beck, R. Vuorio, E. Z. Liu, Z. Xiong, L. Zintgraf, C. Finn, and S. Whiteson, “A survey of meta-reinforcement learning,” arXiv preprint arXiv:2301.08028, 2023

    arXiv 2023

  4. [4]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in ICML, 2017, pp. 1126–1135

    work page 2017

  5. [5]

    A survey on offline reinforcement learning: Taxonomy, review, and open problems,

    R. F. Prudencio, M. R. O. A. Maximo, and E. L. Colombini, “A survey on offline reinforcement learning: Taxonomy, review, and open problems,” 2022

    work page 2022

  6. [6]

    NoRML: No-Reward Meta Learning

    Y . Yang, K. Caluwaerts, A. Iscen, J. Tan, and C. Finn, “Norml: No- reward meta learning,” arXiv preprint arXiv:1903.01063 , 2019

    work page Pith review arXiv 1903

  7. [7]

    Learning to adapt: Meta-learning for model-based control,

    I. Clavera, A. Nagabandi, R. S. Fearing, P. Abbeel, S. Levine, and C. Finn, “Learning to adapt: Meta-learning for model-based control,” arXiv preprint arXiv:1803.11347 , 2018

    arXiv 2018

  8. [8]

    Context-aware dynamics model for generalization in model-based reinforcement learning,

    K. Lee, Y . Seo, S. Lee, H. Lee, and J. Shin, “Context-aware dynamics model for generalization in model-based reinforcement learning,” in ICML, 2020, pp. 5757–5766

    work page 2020

Show all 39 references
  1. [9]

    Rma: Rapid motor adaptation for legged robots,

    A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,” in RSS, 2021

    2021

  2. [10]

    Efficient off- policy meta-reinforcement learning via probabilistic context variables,

    K. Rakelly, A. Zhou, C. Finn, S. Levine, and D. Quillen, “Efficient off- policy meta-reinforcement learning via probabilistic context variables,” in ICML. PMLR, 2019, pp. 5331–5340

    2019

  3. [11]

    Varibad: A very good method for bayes-adaptive deep RL via meta-learning,

    L. M. Zintgraf, K. Shiarlis, M. Igl, S. Schulze, Y . Gal, K. Hofmann, and S. Whiteson, “Varibad: A very good method for bayes-adaptive deep RL via meta-learning,” CoRR, vol. abs/1910.08348, 2019

    1910 arXiv

  4. [12]

    Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. Institut f ¨ur Informatik, Technische Universit¨at M ¨unchen,

    J. Schmidhuber, “Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. Institut f ¨ur Informatik, Technische Universit¨at M ¨unchen,” 1987

    1987

  5. [13]

    Promp: Proximal meta-policy search,

    J. Rothfuss, D. Lee, I. Clavera, T. Asfour, and P. Abbeel, “Promp: Proximal meta-policy search,” in ICLR, 2018

    2018

  6. [14]

    A theoretical understanding of gradient bias in meta-reinforcement learning,

    B. Liu, X. Feng, J. Ren, L. Mai, R. Zhu, H. Zhang, J. Wang, and Y . Yang, “A theoretical understanding of gradient bias in meta-reinforcement learning,” in NeurIPS, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022

    2022

  7. [15]

    A simple neural attentive meta-learner

    N. Mishra, M. Rohaninejad, X. Chen, and P. Abbeel, “A simple neural attentive meta-learner.” in ICLR (Poster). OpenReview.net, 2018

    2018

  8. [16]

    One-shot imitation learning

    Y . Duan, M. Andrychowicz, B. C. Stadie, J. Ho, J. Schneider, I. Sutskever, P. Abbeel, and W. Zaremba, “One-shot imitation learning.” in NIPS, 2017

    2017

  9. [17]

    Meta- reinforcement learning of structured exploration strategies,

    A. Gupta, R. Mendonca, Y . Liu, P. Abbeel, and S. Levine, “Meta- reinforcement learning of structured exploration strategies,” NeurIPS, vol. 31, 2018

    2018

  10. [18]

    Meta-sgd: Learning to learn quickly for few-shot learning,

    Z. Li, F. Zhou, F. Chen, and H. Li, “Meta-sgd: Learning to learn quickly for few-shot learning,” arXiv preprint arXiv:1707.09835 , 2017

    2017 arXiv

  11. [19]

    Multimodal model-agnostic meta-learning via task-aware modulation,

    R. Vuorio, S.-H. Sun, H. Hu, and J. J. Lim, “Multimodal model-agnostic meta-learning via task-aware modulation,” NeurIPS, vol. 32, 2019

    2019

  12. [20]

    Conditional neural processes,

    M. Garnelo, D. Rosenbaum, C. J. Maddison, T. Ramalho, D. Saxton, M. Shanahan, Y . W. Teh, D. J. Rezende, and S. M. A. Eslami, “Conditional neural processes,” CoRR, vol. abs/1807.01613, 2018

    2018 arXiv

  13. [21]

    Focal: Efficient fully-offline meta- reinforcement learning via distance metric learning and behavior reg- ularization,

    L. Li, R. Yang, and D. Luo, “Focal: Efficient fully-offline meta- reinforcement learning via distance metric learning and behavior reg- ularization,” in ICLR, 2020

    2020

  14. [22]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” 2013

    2013

  15. [23]

    Spatial multi-attention conditional neural processes,

    L.-L. Bao, J.-S. Zhang, and C.-X. Zhang, “Spatial multi-attention conditional neural processes,” Neural Networks , vol. 173, p. 106201, 2024

    2024

  16. [24]

    Autoregressive conditional neural processes,

    W. Bruinsma, S. Markou, J. Requeima, A. Y . K. Foong, T. Andersson, A. Vaughan, A. Buonomo, S. Hosking, and R. E. Turner, “Autoregressive conditional neural processes,” in ICLR, 2023

    2023

  17. [25]

    Attentive neural processes,

    H. Kim, A. Mnih, J. Schwarz, M. Garnelo, A. Eslami, D. Rosenbaum, O. Vinyals, and Y . W. Teh, “Attentive neural processes,” inICLR, 2019

    2019

  18. [26]

    Contrastive conditional neural processes,

    Z. Ye and L. Yao, “Contrastive conditional neural processes,” in CVPR, 2022, pp. 9687–9696

    2022

  19. [27]

    Conditional neural movement primitives

    M. Y . Seker, M. Imre, J. H. Piater, and E. Ugur, “Conditional neural movement primitives.” in Robotics: Science and Systems , 2019

    2019

  20. [28]

    Imitation and mirror systems in robots through deep modality blending networks,

    M. Y . Seker, A. Ahmetoglu, Y . Nagai, M. Asada, E. Oztop, and E. Ugur, “Imitation and mirror systems in robots through deep modality blending networks,” Neural Networks, vol. 146, p. 22, 2022

    2022

  21. [29]

    GAN-CNMP: an interactive generative drawing tool,

    S. E. Ada, M. Y . Seker, and P. Yanardag, “GAN-CNMP: an interactive generative drawing tool,” CoRR, vol. abs/2111.14934, 2021

    2021 arXiv

  22. [30]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT Press, 2018

    2018

  23. [31]

    High- dimensional continuous control using generalized advantage estimation,

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estimation,” arXiv preprint arXiv:1506.02438 , 2015

    2015 arXiv

  24. [32]

    Generalization in transfer learning: robust control of robot locomotion,

    S. E. Ada, E. Ugur, and H. L. Akin, “Generalization in transfer learning: robust control of robot locomotion,” Robotica, vol. 40, no. 11, p. 3811–3836, 2022

    2022

  25. [33]

    Adam: Amethod for stochastic optimiza- tion,

    D. P. Kingma and J. L. Ba, “Adam: Amethod for stochastic optimiza- tion,” in Proc. 3rd Int. Conf. Learn. Representations , 2014

    2014

  26. [34]

    Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,

    K. Chua, R. Calandra, R. McAllister, and S. Levine, “Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,” NIPS, vol. 31, 2018

    2018

  27. [35]

    Mujoco: A physics engine for model- based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model- based control,” in IROS, 2012, pp. 5026–5033

    2012

  28. [36]

    Openai gym,

    G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schul- man, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540, 2016

    2016 arXiv

  29. [37]

    CaDM Code Repository ,

    Y . Seo, “ CaDM Code Repository ,” 2020, https://github.com/ younggyoseo/CaDM, Accessed in 2024

    2020

  30. [38]

    Model predictive control: Theory and practice—a survey,

    C. E. Garcia, D. M. Prett, and M. Morari, “Model predictive control: Theory and practice—a survey,”Automatica, vol. 25, no. 3, pp. 335–348, 1989

    1989

  31. [39]

    The cross- entropy method for optimization,

    Z. I. Botev, D. P. Kroese, R. Y . Rubinstein, and P. L’Ecuyer, “The cross- entropy method for optimization,” in Handbook of statistics . Elsevier, 2013, vol. 31, pp. 35–59

    2013

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.