Pith. sign in

REVIEW 4 major objections 6 minor 40 references

URSA makes a real quadruped build its own diverse skill repertoire and use it to recover from leg damage.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

URSA extends quality-diversity actor-critic with learned skill spaces, safety constraints, and world-model training, enabling real-world unsupervised skill discovery on a quadruped.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Real-hardware unsupervised QD is a genuine step forward, but the abstract's damage-adaptation numbers outrun what the experiments actually support. the 4 major comments →

arxiv 2508.19172 v3 pith:KIFCSASS submitted 2025-08-26 cs.RO cs.AIcs.LG

From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity

Luca Grillotti , Lisa Coiffard , Oscar Pang , Maxence Faldor , Antoine Cully (AIRL , Imperial College London) This is my paper
classification cs.RO cs.AIcs.LG
keywords unsupervised skill discoveryquality-diversityreinforcement learningworld modelsquadruped locomotiondamage adaptationsafety constraintslatent skill space
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

URSA is a training scheme that lets a real quadruped robot build a library of diverse locomotion skills from scratch, without a simulator and without a hand-specified set of gaits. Instead of optimizing for one best walking policy, the robot maintains a repertoire of distinct behaviors sampled from a learned latent space, kept diverse by an entropy-based replacement rule and filtered by safety constraints. On a Unitree A1 quadruped, the paper reports, this works in simulation and in the real world; when a leg is damaged, selecting a compensating skill from the repertoire with a few trials of Bayesian search beats a single-skill learner on most of the damage scenarios tested. The broader claim is that skill discovery on hardware can be unsupervised, sample-efficient, and continuous, so robots can adapt without being re-engineered for each new situation.

Core claim

URSA's central claim: a robot can grow a skill repertoire on its own body, defining a skill as the expected feature vector of a policy's stationary distribution. Features are hand-chosen velocities or, unsupervised, a VAE latent encoding. New skills are sampled from a KDE over the repertoire and trained under two Lagrangian constraints: stay close to the target skill in successor-feature space, and keep the cost critic non-positive to remain upright. Greedy deletion maximizes a lower bound on KDE entropy, spreading skills uniformly. Demonstrated: diverse real quadruped gaits, and a Bayesian search selects a compensating skill in a few trials after leg damage.

What carries the argument

The load-bearing mechanism is the diversity-preserving repertoire R. Every safe state feature is added, then the skill with the smallest nearest-neighbor Mahalanobis distance is removed, a rule derived from a lower bound on the entropy of a KDE fitted to R; this converts raw observations into an evenly spread set of behaviors without a predefined descriptor. Around R, a skill-conditioned actor-critic trained by imagined rollouts in a world model enforces two constraints via Lagrangian critics: the successor-feature constraint (the policy really executes the skill) and the safety constraint (the robot stays upright and inside the safe state set).

Load-bearing premise

The repertoire's diversity guarantee rests on the approximation that replacing the closest-neighbor skill leaves the covariance of the skill set essentially unchanged; if that approximation fails, the entropy bound that drives uniform coverage of the reachable skill space no longer holds.

What would settle it

During a simulated training run, record the true entropy of the KDE fitted to the repertoire and the nearest-neighbor lower bound after every repertoire update; if the two diverge over time, or if the covariance shifts by more than a small amount after a single swap, the greedy replacement is not delivering the uniformity it claims. A cheaper check is to run the same deletion rule on a fixed random skill set, where the true entropy-maximizing deletion can be found by exhaustive search, and compare outcomes.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A robot can acquire a broad skill set on hardware with only about five hours of interaction, because most policy learning happens inside an imagined world model.
  • Damage recovery becomes a search problem over an existing repertoire: selecting the best compensating skill takes a few trials rather than retraining the policy.
  • Unsupervised discovery can be steered by the choice of feature space: with hand-chosen velocity features, the same mechanism yields controllable velocity-tracking skills.
  • Skill diversity itself can help performance, since the repertoire-based learner also beats the single-skill learner in the undamaged setting.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The repertoire/search split suggests a natural interface for lifelong learning: a planner or user could query the latent space for a behavior that satisfies a desired outcome, which the paper only demonstrates through velocity targets and ITE.
  • Because the skill space is learned by a VAE, the method is in principle morphology-agnostic, but the paper's own limitation notes that the choice of which state dimensions feed the encoder is decisive; ablating those inputs would make that dependency explicit.
  • The entropy lower bound could be tightened by updating the covariance with each replacement, removing the paper's 'negligible change' assumption and likely changing which skills get deleted early in training.
  • A direct test of the diversity mechanism would compare the greedy nearest-neighbor deletion against an exact entropy-maximizing deletion in simulation, isolating whether the uniformity gains come from the update rule or from the KDE sampling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes URSA, an extension of QDAC for unsupervised quality-diversity skill discovery directly on real-world robots. URSA combines a learned VAE-based skill space, a KDE-based sampling distribution over a fixed-size repertoire, safety constraints, and a DayDreamer world-model backbone. The authors evaluate URSA on a Unitree A1 quadruped in simulation and in the real world, claiming diverse locomotion skill discovery and downstream damage adaptation. The abstract further claims that URSA outperforms all baselines in 5/9 simulated and 3/5 real-world damage scenarios.

Significance. If the claims are supported, this would be a meaningful step toward real-world unsupervised QD, with potential impact on robot damage adaptation and continuous skill learning. The paper includes a correct and reasonably transparent mathematical derivation in Appendix C, and the experimental setup is nontrivial: real-world learning on a quadruped without sim-to-real transfer, with safety constraints and an asynchronous world-model framework. The main weakness is that the headline quantitative claims are not consistently supported by the paper's own experimental text and figures, which limits the reliability of the central message.

major comments (4)
  1. [Abstract; Section 5.3; Appendix A; Figure 5] The abstract's claim of '5 out of 9 simulated and 3 out of 5 real-world damage scenarios' is not supported by the reported results. Section 5.3 states that URSA+ITE outperforms both DOMiNiC and DayDreamer only in upper-leg joint damage, and that DOMiNiC's near-optimality constraint is more beneficial for severe full-leg failures. Appendix A explicitly says 'URSA does not consistently outperform all baselines across individual damage scenarios.' In the real-world comparison, Figure 5 shows only URSA+ITE versus DayDreamer; DOMiNiC was not run, so 'outperforms all baselines in 3 out of 5 real-world scenarios' cannot be evaluated. Moreover, the text in Section 5.3 says URSA outperforms DayDreamer in all damage scenarios except back-right leg failures, which is 4 out of 5, not 3 out of 5. These discrepancies must be resolved and the abstract revised to match the evidence.
  2. [Section 5.3, Figure 5] The real-world results are based on two independent runs per condition, and the figure reports median and IQR across those two runs. With n=2, the per-scenario 'win' count is not statistically meaningful, and the paper should either provide individual run values, confidence intervals that account for small n, or weaken the per-scenario comparison to a qualitative trend. This is load-bearing for RQ2 because the damage-adaptation advantage is the main downstream-utility claim.
  3. [Section 5.3, Figure 4] The comparison underlying the '5 out of 9' claim is not clearly defined. Figure 4 shows two URSA variants: a hatched 'best return for URSA' bar and a URSA+ITE bar. The hatched bar appears to be an oracle upper bound over all skills in the repertoire rather than a deployable policy. If the 5/9 count is based on this oracle, it is not an apples-to-apples comparison with the baselines. The paper should report a precise win convention (which variant, which baseline, which scenarios) and provide per-scenario tables rather than only a CCDF.
  4. [Section 4.2, Eq. (1)] The repertoire update rule removes the skill with the smallest nearest-neighbor distance, justified by maximizing the entropy lower bound in Eq. (1). However, the paper states 'Assuming the impact of those updates on Σ is negligible' without providing evidence. Since Σ is recomputed from the repertoire and changes every time a skill is added or removed, this assumption is nontrivial. The paper should either justify why the covariance update is negligible or provide an ablation comparing the proposed update against random replacement or a direct entropy estimate. This is important because the diversity mechanism is central to the method's claimed advantage.
minor comments (6)
  1. [Figure 4 caption] The caption says 'The best return for URSA is shown in hatched bars' but the legend lists both 'URSA' and 'URSA + ITE'. It would be clearer to explicitly label the hatched bars as an upper bound or 'best over repertoire' and to state how this differs from the ITE variant.
  2. [Section 5.3] The text says 'URSA outperforms DayDreamer in all damage scenarios except for back-right leg failures', but the real-world panel in Figure 5 includes a 'No Damage' condition. Clarify whether the 5/9 and 3/5 counts include the no-damage condition, since the abstract says 'damage scenarios'.
  3. [Section 5.2, Figure 3] The claim of '4 times greater coverage compared to the baselines' is not supported by a precise definition of coverage. Please specify how the joint-angle space is discretized and how coverage is computed.
  4. [Appendix D.4, Table 1] The hyperparameter table would benefit from also listing the safety threshold and reward weights, which are currently given only in the text of D.1. This would make the setup easier to reproduce.
  5. [Section 4.2, Equation (1)] The notation 'k' for the constant term is defined only in the derivation of Appendix C. Consider defining it inline or referring explicitly to Eq. (C.1) to avoid confusion.
  6. [Figure 5, middle panel] The middle panel caption says 'evolution of attempted skills' but the x-axis is 'ITE iteration' and the y-axis is 'Total Return'. Clarify what is plotted, e.g., the skill selected by ITE and its evaluated return.

Circularity Check

0 steps flagged

No circularity: the derivation is self-contained; the overstated damage-adaptation claim is a robustness issue, not a circular reduction.

full rationale

The paper's derivation chain is not circular. The entropy lower bound (Eq. 1, Appendix C) is a standard log-sum-exp inequality, and the repertoire update is a greedy heuristic for maximizing that bound under an explicit 'impact of those updates on Sigma is negligible' assumption; whether that assumption holds is a correctness concern, not circularity. The central diversity claim is evaluated externally via joint-angle coverage (Fig. 3), a metric independent of the latent-space entropy objective, so the discovery claim is not self-referential. Damage-adaptation results are empirical comparisons against external baselines (DayDreamer, DOMiNiC); the abstract's '3 out of 5 real-world' claim is weakened because DOMiNiC was not run and n=2, but that is an overstatement/statistical robustness issue, not a reduction of the prediction to the training objective. The paper cites prior work by the same authors (QDAC, AURORA), but these are published, peer-reviewed components used as building blocks, not invoked as proof of URSA's own efficacy. No equation equals its input by construction, and no fitted parameter is renamed as a prediction. The stated limitation that the system 'is limited to exploring skills that have been previously encountered during training' further confirms that the repertoire update is a heuristic with acknowledged gaps, not a tautology. Score 0.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The method introduces no new physical entities. The main load-bearing choices are the hand-designed reward and safety cost, the VAE latent space, and the KDE/repertoire update assumption of negligible covariance change. These are all domain assumptions or ad hoc choices rather than fitted parameters in the statistical sense, but they are essential to the system's operation.

free parameters (6)
  • Nz (target distinct skills) = 512
    Sets the adaptive constraint threshold delta; chosen by user, not derived from data.
  • NR (repertoire size) = 4096
    Maximum number of skills in repertoire; fixed hyperparameter.
  • VAE latent dimension = 2
    Dimensionality of the learned skill space; chosen by hand.
  • Safe-state threshold rupr = 0.7
    Defines Ssafe; hand-chosen cutoff for upright posture.
  • Reward weights (5, -0.5, -0.5) = 5, -0.5, -0.5
    Shaping coefficients for forward, lateral, yaw terms in unsupervised reward.
  • ITE length scale = 0.1
    Gaussian process length scale for Bayesian optimization in damage adaptation.
axioms (6)
  • domain assumption The DayDreamer world model (RSSM) provides accurate enough latent dynamics for real-world skill learning.
    URSA relies on imagination-based training; if the world model is inaccurate, learned skills may not transfer to hardware.
  • domain assumption The reachable skill space is well-approximated by a Gaussian KDE with a fixed covariance.
    Section 4.2 fits a KDE on the repertoire and samples from it; this assumes the skill distribution is smooth and Gaussian-like.
  • domain assumption The hand-defined safe state set captures all unsafe behaviors.
    Safety relies on user-provided cost function; unsafe states outside this set could be committed to the repertoire.
  • ad hoc to paper The entropy lower-bound maximization via nearest-neighbor removal yields a uniform repertoire.
    Section 4.2 ignores the impact of repertoire updates on the covariance Sigma, an approximation that may not hold.
  • domain assumption The VAE latent space is a meaningful behavior descriptor for diversity.
    The learned features define what counts as diverse; if the VAE ignores relevant dimensions, skills will be deceptively similar.
  • domain assumption The reward function is an adequate proxy for downstream task utility.
    Skills are optimized for forward velocity and stability; this may not generalize to arbitrary downstream tasks.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity." pith.science (2026). https://pith.science/paper/KIFCSASS

@misc{pith2026250819172,
  author       = {Pith},
  title        = {Pith review of: From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KIFCSASS}},
  note         = {Machine review of arXiv:2508.19172}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Autonomous skill discovery aims to enable robots to acquire diverse behaviors without explicit supervision. Learning such behaviors directly on physical hardware remains challenging due to safety and data efficiency constraints. Existing methods, including Quality-Diversity Actor-Critic (QDAC), require manually defined skill spaces and carefully tuned heuristics, limiting real-world applicability. We propose Unsupervised Real-world Skill Acquisition (URSA), an extension of QDAC that enables robots to autonomously discover and master diverse, high-performing skills directly in the real world. We demonstrate that URSA successfully discovers diverse locomotion skills on a Unitree A1 quadruped in both simulation and the real world. Our approach supports both heuristic-driven skill discovery and fully unsupervised settings. We also show that the learned skill repertoire can be reused for downstream tasks such as real-world damage adaptation, where URSA outperforms all baselines in 5 out of 9 simulated and 3 out of 5 real-world damage scenarios. Our results establish a new framework for real-world robot learning that enables continuous skill discovery with limited human intervention, representing a significant step toward more autonomous and adaptable robotic systems. Demonstration videos are available at https://adaptive-intelligent-robotics.github.io/URSA.

Figures

Figures reproduced from arXiv: 2508.19172 by Antoine Cully (AIRL, Imperial College London), Lisa Coiffard, Luca Grillotti, Maxence Faldor, Oscar Pang.

Figure 1
Figure 1. Figure 1: We propose Unsupervised Real-world Skill Acquisition (URSA), a framework for unsuper [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of URSA: The system checks if the state [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Average joint angles across the skill repertoire in URSA, DayDreamer, and DOMiNiC. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of returns across joint damage scenarios in simulation. The best return for [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Left: Comparison of returns across damage scenarios in the real world, showing median return and IQR across 2 independent runs. Middle: Evolution of attempted skills for a single run of ITE with FL leg damage. Right: Average reward during training (shaded area as standard deviation of 15-minute segments). evaluate URSA combined with Iterative Trial and Error (ITE) [3] (URSA + ITE), which uses Bayesian opti… view at source ↗
Figure 6
Figure 6. Figure 6: Velocity tracking errors during skill execution, evaluating the robot’s accuracy in following [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Complementary cumulative distribution functions (CCDF) of the median performance [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Illustration of URSA’s imagination-based training loop in the world model [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 31 canonical work pages · 3 internal anchors

  1. [1]

    J. Luo, Z. Hu, C. Xu, Y . L. Tan, J. Berg, A. Sharma, S. Schaal, C. Finn, A. Gupta, and S. Levine. Serl: A software suite for sample-efficient robotic reinforcement learning. 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 16961–16969, 2024. URL https://api.semanticscholar.org/CorpusID:267311834

    work page 2024

  2. [2]

    Kostrikov, L

    I. Kostrikov, L. M. Smith, and S. Levine. Demonstrating A walk in the park: Learning to walk in 20 minutes with model-free reinforcement learning. In K. E. Bekris, K. Hauser, S. L. Herbert, and J. Yu, editors, Robotics: Science and Systems XIX, Daegu, Republic of Korea, July 10-14, 2023, 2023. doi:10.15607/RSS.2023.XIX.056. URL https://doi.org/10. 15607/R...

    work page doi:10.15607/rss.2023.xix.056 2023

  3. [3]

    Cully, J

    A. Cully, J. Clune, D. Tarapore, and J.-B. Mouret. Robots that can adapt like animals. Nature, 521(7553):503–507, May 2015. ISSN 1476-4687. doi:10.1038/nature14422. URL https: //doi.org/10.1038/nature14422

    doi:10.1038/nature14422 2015

  4. [4]

    Chatzilygeroudis, V

    K. Chatzilygeroudis, V . Vassiliades, and J.-B. Mouret. Reset-free trial-and-error learning for robot damage recovery. Robotics and Autonomous Systems , 100:236–250, 2018

    work page 2018

  5. [5]

    Kaushik, P

    R. Kaushik, P. Desreumaux, and J.-B. Mouret. Adaptive prior selection for repertoire-based online adaptation in robotics. Frontiers in Robotics and AI, 6:151, 2020. 9

    work page 2020

  6. [6]

    Sharma, M

    A. Sharma, M. Ahn, S. Levine, V . Kumar, K. Hausman, and S. Gu. Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning. In Robotics: Science and Systems XVI. Robotics: Science and Systems Foundation, July 2020. ISBN 978-0-9923747-6-1. doi:10.15607/RSS.2020.XVI.053. URL http://www.roboticsproceedings.org/ rss16/p053.pdf

    work page doi:10.15607/rss.2020.xvi.053 2020

  7. [7]

    S. C. Smith, B. Lim, H. Janmohamed, and A. Cully. Quality-diversity optimisation on a physical robot through dynamics-aware and reset-free learning. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation , GECCO ’23 Companion, page 171–174, New York, NY , USA, 2023. Association for Computing Machinery. ISBN 9798400701207. doi: ...

    work page arXiv 2023

  8. [8]

    J. K. Pugh, L. B. Soros, and K. O. Stanley. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:40, 2016

    work page 2016

  9. [9]

    Cully and Y

    A. Cully and Y . Demiris. Quality and diversity optimization: A unifying modular framework. IEEE Transactions on Evolutionary Computation , 22(2):245–259, 2017

    work page 2017

  10. [10]

    A. Cully. Autonomous skill discovery with quality-diversity and unsupervised descriptors. In Proceedings of the Genetic and Evolutionary Computation Conference , pages 81–89, 2019

    work page 2019

  11. [11]

    Grillotti and A

    L. Grillotti and A. Cully. Relevance-guided unsupervised discovery of abilities with quality- diversity algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference , pages 77–85, 2022

    work page 2022

  12. [12]

    Coiffard, P

    L. Coiffard, P. Templier, and A. Cully. Overcoming Deceptiveness in Fitness Optimization with Unsupervised Quality-Diversity, page 122–130. Association for Computing Machinery, New York, NY , USA, 2025. ISBN 9798400714658. URLhttps://doi.org/10.1145/ 3712256.3726314

    arXiv 2025

  13. [13]

    Grillotti, M

    L. Grillotti, M. Faldor, B. Gonz´alez Le´on, and A. Cully. Quality-diversity actor-critic: Learning high-performing and diverse behaviors via value and successor features critics. In International Conference on Machine Learning. PMLR, 2024

    work page 2024

  14. [14]

    P. Wu, A. Escontrela, D. Hafner, K. Goldberg, and P. Abbeel. DayDreamer: World Models for Physical Robot Learning, June 2022. URL http://arxiv.org/abs/2206.14176. arXiv:2206.14176 [cs]

    Pith/arXiv arXiv 2022

  15. [15]

    R. S. Sutton and A. G. Barto. Reinforcement learning: an introduction . Adaptive computation and machine learning series. The MIT Press, Cambridge, Massachusetts, second edition edition,

    work page

  16. [16]

    Barreto, W

    A. Barreto, W. Dabney, R. Munos, J. J. Hunt, T. Schaul, H. van Hasselt, and D. Silver. Suc- cessor features for transfer in reinforcement learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems , NIPS’17, pages 4058–4068, Red Hook, NY , USA, Dec. 2017. Curran Associates Inc. ISBN 978-1-5108-6096-4

    work page 2017

  17. [17]

    E. Altman. Constrained Markov decision processes. Routledge, 1999

    work page 1999

  18. [18]

    E. Parzen. On estimation of a probability density function and mode.The annals of mathematical statistics, 33(3):1065–1076, 1962

    work page 1962

  19. [19]

    Rosenblatt

    M. Rosenblatt. Remarks on Some Nonparametric Estimates of a Density Function. The Annals of Mathematical Statistics , 27(3):832 – 837, 1956. doi:10.1214/aoms/1177728190. URL https://doi.org/10.1214/aoms/1177728190

    arXiv 1956

  20. [20]

    D. W. Scott. Multivariate Density Estimation: Theory, Practice and Visualization . John Wiley & Sons, Inc., New York, 1992. 10

    work page 1992

  21. [21]

    D. P. Kingma and M. Welling. Auto-encoding variational bayes. In Y . Bengio and Y . LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , 2014. URL http://arxiv. org/abs/1312.6114

    Pith/arXiv arXiv 2014

  22. [22]

    Cheng, M

    J. Cheng, M. Vlastelica, P. Kolev, C. Li, and G. Martius. Learning diverse skills for local navigation under multi-constraint optimality. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 5083–5089. IEEE, 2024

    work page 2024

  23. [23]

    Eysenbach, A

    B. Eysenbach, A. Gupta, J. Ibarz, and S. Levine. Diversity is all you need: Learning skills without a reward function. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https: //openreview.net/forum?id=SJx63jRqFm

    work page 2019

  24. [24]

    Sharma, S

    A. Sharma, S. Gu, S. Levine, V . Kumar, and K. Hausman. Dynamics-aware unsupervised discovery of skills. In International Conference on Learning Representations , 2020. URL https://openreview.net/forum?id=HJgLZR4KvH

    work page 2020

  25. [25]

    S. Park, K. Lee, Y . Lee, and P. Abbeel. Controllability-aware unsupervised skill discovery. arXiv preprint arXiv:2302.05103, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  26. [26]

    S. Park, O. Rybkin, and S. Levine. Metra: Scalable unsupervised rl with metric-aware abstrac- tion. arXiv preprint arXiv:2310.08887, 2023

    Pith/arXiv arXiv 2023

  27. [27]

    Kumar, A

    S. Kumar, A. Kumar, S. Levine, and C. Finn. One solution is not all you need: few-shot extrapolation via structured MaxEnt RL. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS’20, pages 8198–8210, Red Hook, NY , USA, Dec. 2020. Curran Associates Inc. ISBN 978-1-71382-954-6

    work page 2020

  28. [28]

    Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality

    T. Zahavy, Y . Schroecker, F. Behbahani, K. Baumli, S. Flennerhag, S. Hou, and S. Singh. Discovering policies with domino: Diversity optimization maintaining near optimality. arXiv preprint arXiv:2205.13521, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  29. [29]

    Paolo, A

    G. Paolo, A. Laflaquiere, A. Coninx, and S. Doncieux. Unsupervised Learning and Exploration of Reachable Outcome Space. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 2379–2385, Paris, France, May 2020. IEEE. ISBN 978-1-72817-395-

    work page 2020

  30. [30]

    Paolo, M

    G. Paolo, M. Coninx, A. Laflaqui `ere, and S. Doncieux. Discovering and Exploiting Sparse Rewards in a Learned Behavior Space. Evolutionary Computation, 32(3):275–305, Sept. 2024. ISSN 1063-6560. doi:10.1162/evco a 00343. URL https://doi.org/10.1162/evco_ a_00343

    doi:10.1162/evco 2024

  31. [31]

    URL https://ieeexplore.ieee.org/ document/9196819/

    doi:10.1109/ICRA40945.2020.9196819. URL https://ieeexplore.ieee.org/ document/9196819/

    arXiv 2020

  32. [32]

    Laversanne-Finot, A

    A. Laversanne-Finot, A. P´er´e, and P.-Y . Oudeyer. Intrinsically motivated exploration of learned goal spaces. Frontiers in neurorobotics, 14:555271, 2021

    work page 2021

  33. [33]

    B. Lim, A. Reichenbach, and A. Cully. Learning to walk autonomously via reset-free quality- diversity. In Proceedings of the Genetic and Evolutionary Computation Conference , GECCO ’22, pages 86–94, New York, NY , USA, July 2022. Association for Computing Machinery. ISBN 978-1-4503-9237-2. doi:10.1145/3512290.3528715. URL https://doi.org/10. 1145/3512290.3528715

    arXiv 2022

  34. [34]

    Tarapore, J

    D. Tarapore, J. Clune, A. Cully, and J.-B. Mouret. How do different encodings influence the performance of the map-elites algorithm? In Proceedings of the Genetic and Evolutionary Computation Conference 2016, pages 173–180, 2016. 11

    work page 2016

  35. [35]

    Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion

    N. Bohlinger, J. Kinzel, D. Palenicek, L. Antczak, and J. Peters. Gait in eight: Efficient on-robot learning for omnidirectional quadruped locomotion. arXiv preprint arXiv:2503.08375, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  36. [37]

    S. Ha, P. Xu, Z. Tan, S. Levine, and J. Tan. Learning to walk in the real world with minimal human effort. In J. Kober, F. Ramos, and C. Tomlin, editors,Proceedings of the 2020 Conference on Robot Learning, volume 155 of Proceedings of Machine Learning Research, pages 1110–

    work page 2020

  37. [39]

    , NR) from the repertoire

    Component Selection: Uniformly select an index i ∼ Uniform(1, . . . , NR) from the repertoire

    work page

  38. [40]

    Gaussian Sampling: Sample ztarget ∼ N(zi, Σ) using the selected repertoire skill zi as the mean. The covariance matrix Σ is the empirical covariance of all skills in the repertoire scaled by Scott’s rule bandwidth h = N − 1 D+4 R , such that the final covariance is Σ = h2Σempirical. This sampling procedure naturally concentrates samples around regions of ...

    work page

  39. [1120]

    URL https://proceedings.mlr.press/v155/ ha21c.html

    PMLR, 16–18 Nov 2021. URL https://proceedings.mlr.press/v155/ ha21c.html. 12 −400 −300 −200 −100 0 100 200 300 400 Median Return 0.0 0.2 0.4 0.6 0.8 1.0P(Median Return > x) URSA URSA + ITE DOMiNiC DayDreamer Figure 7: Complementary cumulative distribution functions (CCDF) of the median performance across the simulated damage scenarios from Figure 4. HH1 1...

    work page 2021

  40. [2018]

    ISBN 978-0-262-03924-6

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.