Pith. sign in

REVIEW 4 major objections 4 minor 43 references

Divide, Discover, Deploy: Factorized Skill Learning with Symmetry and Style Priors

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Factorizing the robot's state space and assigning different unsupervised skill-discovery objectives to each factor yields a quadruped skill library that is safe, human-interpretable, deployable, and nearly as effective as an oracle on downs

desk verdict A solid modular USD framework for legged robots; the simulation-level results and honest limitations deserve a real referee, but the zero-shot hardware claim is over-sold. read the letter →

arxiv 2508.19953 v2 pith:TH2OYWLK submitted 2025-08-27 cs.RO

Rafael Cathomen , Mayank Mittal , Marin Vlastelica , Marco Hutter This is my paper
classification cs.RO
keywords unsupervisedskilldiscoveryfactorizedMDPquadrupedlocomotionsymmetryaugmentationMETRADIAYNsim-to-realtransferhierarchicalcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that unsupervised skill discovery can be turned from a simulation curiosity into a deployable robotics tool by dividing the robot's state into user-chosen factors and giving each factor the discovery objective that suits it. The central result is on a quadruped: pairing METRA-style rewards on unbounded factors like planar position with DIAYN-style rewards on bounded factors like heading yields a skill library that, on a rough-terrain waypoint-navigation task, scores close to an oracle policy trained with hand-crafted rewards and far above single-objective or mismatched libraries. The paper also introduces a style factor and regularization that cut illegal body contacts sharply, symmetry augmentation that makes skill-to-state mappings balanced and human-interpretable, and per-factor weights that let an operator prioritize behaviors. Together these components let a human command skills such as walking while pitching or rotating while crouched, and the paper reports that the skills, trained purely in simulation, transfer zero-shot to the real ANYmal-D robot. A reader should care because the approach points to a path where diverse, reusable robot behaviors come from intrinsic objectives rather than hand-designed reward functions.

What carries the argument

The load-bearing construction is the symmetric factored MDP: the state space S = S1 x ... x SN and the latent skill space Z = Z1 x ... x ZN are both factorized, so each skill component z_i is meant to drive only its state factor s_i. What carries the argument is the per-factor reward menu - DIAYN's log q_phi(z|s) - log p(z), a learned-discriminator mutual-information reward, versus METRA's (phi(s') - phi(s))^T z, a Wasserstein-dependence reward that aligns latent state transitions with the commanded skill direction - plus the policy objective J(theta) = sum_i lambda_i * I(S_i, Z_i) + lambda_(N+1) * J_style, a weighted sum of per-factor objectives and a style term, where the lambda are sample

What would settle it

Deploy the trained mixed-factor skill policy on the real ANYmal-D and command a fixed grid of skill vectors across repeated trials, measuring the correlation between commanded skill and achieved body motion plus the rate of illegal contacts and falls; if commanded direction and achieved motion are uncorrelated, or contact and fall counts exceed a pre-set threshold, the zero-shot deployability claim fails. A second check: re-run the waypoint-navigation comparison with substantially more than five seeds, since the mixed and oracle rewards (148.55 +/- 29.24 vs 164.37 +/- 21.42) overlap within one

Watch

Extended reading notes

Core claim

At the center of the paper is 'Divide, Discover, Deploy': a factored-MDP framework in which the user splits the robot's state into factors (position, heading rate, base height, roll-pitch) and pairs each factor with its own latent skill component and its own intrinsic reward. The guiding observation is that different USD objectives suit different factor types - METRA, which rewards moving a learned latent state in the commanded direction, gives broad coverage on unbounded factors like planar position, while DIAYN, which rewards states where a learned discriminator can identify the skill, gives separable behaviors on bounded factors like heading. Around this the paper builds three mechanisms:

Load-bearing premise

The zero-shot hardware-transfer claim rests on the domain-randomized simulation being a faithful enough proxy for the real ANYmal-D that skills trained purely in simulation execute safely on hardware; the paper's only hardware evidence is the qualitative video description in Section 5 and Figure 3, with no measured metrics or repeated trials.

Editorial extensions

If this is right

  • A mixed per-factor assignment (METRA on position, DIAYN on heading) broadens coverage and separability simultaneously; single-objective or mismatched libraries score markedly lower on the downstream navigation task (148.55 vs 27.87-81.62 mean reward).
  • Activating the style factor drops illegal shank contacts from roughly 4.04% to 0.03% per step and to 0.12% or less on the thigh, while improving skill discriminability on both METRA- and DIAYN-driven factors.
  • Symmetry augmentation does not speed convergence or raise metric scores - and slightly hurts METRA factors - but it produces balanced, interpretable skill-to-state mappings, such as rotating uniformly in both directions instead of one biased direction.
  • Because skills are resampled within episodes, the trained policy switches smoothly between commanded skills at deployment instead of locking onto the first behavior it reaches.
  • The learned skill library, used as a low-level controller under a high-level waypoint policy, performs near the oracle's level on the navigation benchmark, showing that unsupervised skills can substitute for hand-tuned low-level controllers in hierarchical reinforcement learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'one objective per factor' principle suggests a selection problem the paper leaves open: the best factorization, algorithm assignment, and skill dimensions could themselves be searched or learned, since the paper shows wrong choices degrade downstream performance sharply.
  • The factor-weighting failure on conflicting quadrant factors (weights barely change outcomes because per-factor networks still train on all collected data) points to a direct, testable fix the paper names: make the USD network losses respect the sampled weights.
  • Symmetry augmentation suppressed single-foot lifting, yet disabling it produced a lift on only one leg - so a graded or per-leg symmetry schedule, rather than a binary switch, might unlock pedipulation-like skills within the same framework.
  • If the near-oracle navigation result generalizes to other morphologies and tasks, unsupervised skill libraries could become standard building blocks for hierarchical control, reducing the reward-engineering bottleneck the introduction identifies; the paper's stated future work (loco-manipulation, other robots) is the natural test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a modular unsupervised skill discovery (USD) framework for quadrupedal robots. The state and skill spaces are factorized into user-defined components, and each factor can be assigned a different USD objective (METRA or DIAYN). Additional ingredients are a style factor and regularization penalties for safety, per-factor weighting to prioritize or combine objectives, and symmetry augmentation based on the robot's morphology. The framework is evaluated in Isaac Lab on an ANYmal-D model, with ablations on factor weighting, style factor, algorithm mixing, and symmetry augmentation, plus a downstream waypoint-navigation task. The authors also report a real-hardware demonstration described qualitatively and claim zero-shot transfer from simulation to hardware.

Significance. If the claims are substantiated, the paper would be a useful contribution to USD for legged robots: it shows a modular way to combine complementary intrinsic objectives per state factor, and it is one of the few attempts to push USD toward deployable hardware skills. Strengths include open-source code, multi-seed ablations, an honest limitations section, and a clear modular design that could be reused with other USD algorithms. However, the central deployment claim currently rests on qualitative evidence, and several evaluation metrics are circular with the training objectives. The downstream navigation result is promising but would benefit from tighter statistical reporting.

major comments (4)
  1. [§5 (Deployment of Learned Skills) / Abstract / Conclusion] The central deployment claim is supported only by qualitative evidence. Section 5 states that individual skill dimensions were commanded on the real robot, and Fig. 3 shows a few snapshots, but no quantitative hardware metrics are provided: no number of trials, success rates, tracking errors, safety statistics, or comparison between simulated and real state distributions. The abstract and conclusion nevertheless assert that zero-shot transfer is 'demonstrated.' Because deployability is a main contribution of the paper, this evidence is insufficient. Please add quantitative hardware experiments or explicitly downgrade the claim to a qualitative proof-of-concept.
  2. [§A.4, Fig. 4, Table 1] The per-factor evaluation is substantially circular. The 'metric score' for METRA factors is exactly the METRA reward (cosine similarity between latent transition and commanded skill); for DIAYN factors it is the discriminator posterior similarity; and for the style factor it is the scaled style reward. Thus Fig. 4 and the factor-metric columns of Table 1 largely re-measure the objectives being optimized. Furthermore, the 'illegal contacts' safety metric is a term in r_style with a weight of -30.0 (Table 9), so the improved contact numbers obtained with the style factor are partly by construction. Please add independent metrics (e.g., state-coverage statistics, task success, or held-out evaluations) or explicitly acknowledge and mitigate this circularity.
  3. [§5, Table 2] The cross-algorithm comparison is confounded by skill dimensionality and baseline modifications. The baseline labeled 'DUSDi' is not the original DUSDi: it is modified with a style factor and regularization. The algorithms also use different skill dimensions (DIAYN dim(z)=8, METRA dim(z)=3, mixed 2+2), and the single-objective baselines are non-factorized. Consequently, the conclusion that the mixed factor–algorithm assignment outperforms single-algorithm setups may reflect capacity, implementation, or regularization differences rather than the proposed assignment principle alone. Controlled skill-dimension sweeps and a true original-DUSDi baseline are needed to isolate the mechanism.
  4. [§5 (Factor Weights); Fig. 8] The paper motivates factor weights as a mechanism to resolve conflicts between skill factors, but the dedicated experiment on conflicting quadrant factors (Fig. 8) shows that weighting 'does not change the performance significantly.' This is an honest null result, but it undercuts the broad claim in the introduction and abstract that weighting 'facilitates handling of conflicting skills.' The text should either limit the claimed benefit of weighting to non-conflicting factor combinations or explain why the quadrant experiment does not contradict the more general claim.
minor comments (4)
  1. [§A.1] Typo: 'obejctive' in the text before Eq. (3). Also, the abbreviation 'CSD' is ambiguous: reference [8] is 'Constrained Skill Discovery,' while reference [20] is 'Controllability-aware Unsupervised Skill Discovery.' Please clarify which method is meant.
  2. [Table 3] Several termination-ratio entries have very large standard deviations (e.g., METRA Goal Reached 0.300 ± 0.234 and Base Collision 0.378 ± 0.532). The statement that the mixed setup 'significantly outperforms' single or mismatched USD setups would be more convincing with significance tests or confidence intervals rather than only means and standard deviations.
  3. [§5 (Symmetric Skill Discovery)] The interpretability benefit of symmetry augmentation is currently supported only by the qualitative scatter plot in Fig. 5. A quantitative measure of behavioral symmetry, such as correlation between mirrored state-visitation distributions or a symmetry-error metric, would strengthen the claim.
  4. [Abstract and Conclusion] The abstract and conclusion state zero-shot hardware transfer without caveats, while §7 candidly documents failures on more complex tasks. Consider adding a brief qualifier that the hardware demonstration is a qualitative proof-of-concept on locomotion skills, with quantitative validation left to future work.

Circularity Check

2 steps flagged · score 6.0 of 10

Several headline evaluations reduce to the training objective: the 'metric score' is the USD reward itself, factor-weighting results reweight by the same λ, and illegal-contact safety is directly penalized by the style reward.

  1. self definitional [Section 5 'Factor Weights'; Fig. 4 caption; App. A.4]
    "When using weights, each rollout contributes to the per-factor metrics proportionally to its assigned factor weights, normalized to avoid numerical bias. The weighted setup achieves substantially higher scores, showing that the agent learns to prioritize relevant factors."

    App. A.4 defines the 'metric score' for METRA factors as the cosine similarity between latent transition and skill—'This corresponds directly to the METRA reward signal'—and for DIAYN factors as the discriminator posterior, i.e., the exact intrinsic rewards being maximized in Eq. (1). Thus reporting that the framework improves these metric scores is equivalent to reporting that it optimizes its own training objective. The factor-weighting experiment aggravates the circularity: rollouts are re-weighted by the same λ used in training, so down-weighted factors contribute less to the reported score. The conclusion that factor weights improve skill quality is therefore partly an artifact of the evaluation definition, not an independent measure.

  2. self definitional [Section 5 'Safety and Extrinsic Rewards'; Table 1; Table 9 (Style Factor Rewards)]
    "Undesired Contacts \sum_{b∈{Thighs, Shanks, Base}} 1[contact(b)] -30.0 ... From Tab. 1, we observe that the style factor significantly reduces undesirable contacts and improves discriminability for both position and heading."

    The safety metric '% Illegal Contacts per Step' counts exactly the contacts that the style reward penalizes with weight -30.0 in Table 9. Adding a reward term that directly penalizes undesired contacts will, by construction, reduce those contacts; reporting this as evidence that the style factor 'promotes safer behaviors' validates the reward design rather than an independent safety property of the skill representation. The factor-metric improvements in the same table are also measured by the training rewards themselves, compounding the circularity.

full rationale

The paper's core pipeline—factorizing state, assigning METRA/DIAYN per factor, symmetry augmentation, style factor, and weights—is not itself derived from its conclusions. The downstream navigation comparison (Table 3) uses an external task reward, and the diversity metric (standard deviation of mean states, Table 2) is independent of the optimized intrinsic rewards, so those results are non-circular evidence. The self-citations to [25] and [34] are implementation references to a published method and simulator, not uniqueness theorems, so they are not load-bearing circularity. However, two headline evaluations are circular. First, the 'metric score' used in Fig. 4 and Table 1 is, by App. A.4's own definition, the METRA reward (cosine similarity) or the DIAYN discriminator signal; showing that the method improves this score is equivalent to showing it optimizes its own training objective. The factor-weighting experiment aggravates this: rollouts are re-weighted by the same λ used in training, so the improvement is partly an artifact of the evaluation. Second, the safety comparison in Table 1 counts 'illegal contacts' that are explicitly penalized with -30 in the style reward (Table 9); the reduction is the direct effect of adding that penalty, not independent validation. The zero-shot hardware claim (Abstract, Section 5, Conclusion) is supported only by a qualitative figure/video and lacks quantitative rigor, but that is a missing-evidence problem, not a circularity, so it does not enter the score. Section 7 honestly documents failures on loco-manipulation and obstacle avoidance, limiting scope but not repairing the circular evaluations. Overall: partial circularity in the headline safety/weighting evaluations; independent downstream and diversity results keep the score moderate.

Assumptions & free parameters 9 free parameters · 4 assumptions · 2 invented entities

The framework introduces several hand-chosen components: factor weight distributions, METRA switching thresholds, style and regularization reward weights, Dirichlet priors, and skill dimensions. The symmetry augmentation relies on domain assumptions about the robot's symmetries and about training on augmented data being sufficient. The style factor and factor weights are constructed entities without external falsifiable handles.

free parameters (9)
  • Factor weight sampling distribution = truncated Gaussian; mean/std not reported
    Lambda is sampled and normalized to unit L2 norm each episode; distribution parameters are not specified in the paper.
  • METRA objective switching range = (0.5, 0.7) cosine similarity
    Curriculum from alignment to norm-matching is triggered by these hand-chosen thresholds (App. Table 7).
  • Norm-matching scale sigma = 10.0
    Controls shape of the norm-matching reward (App. Table 7).
  • Dirichlet prior concentration range = 0.05 to 1.0
    Sparse-to-uniform annealing over discriminator accuracy; annealing threshold not specified (App. Table 8).
  • Skill dimensionality per factor = METRA: 2 (position), DIAYN: 4 (heading) in Tab. 2; d <= 3 for METRA in general
    Chosen by hand to match geometry; d > 3 empirically degrades METRA (App. A.2).
  • Style reward weights = joint torque -1e-3; joint accel -1e-5; action rate -0.2; action norm -0.4; undesired contacts -30; base height -10; flat
    All style component weights are hand-tuned (App. Table 9).
  • Regularization reward weights = joint torque -1e-3; joint accel -2.5e-7; action rate -0.05; torque limits -15; torque ratio limits -15; joint vel limits
    Global penalty weights are hand-tuned (App. Table 10).
  • DIAYN disentanglement penalty gamma = 0.1
    Entanglement penalty weight taken from DUSDi (App. Table 8).
  • Terrain curriculum thresholds = 10 m advance, 5 m descend
    State-coverage based difficulty thresholds chosen by hand (App. A.3).
assumptions (4)
  • domain assumption The ANYmal-D MDP has four exact symmetries and the skill mirroring functions preserve the skill prior and group composition.
    Invoked in Sec. 3-4 and App. A.2 to justify symmetry augmentation; if the mirroring functions are ill-defined, augmented data yields conflicting gradients.
  • ad hoc to paper Training all networks on symmetry-augmented data induces approximate reward and policy symmetry without explicit reward averaging.
    Sec. 4 states 'we found that it suffices' rather than proving it; this is an empirical heuristic.
  • domain assumption METRA is appropriate for unbounded state factors (position) and DIAYN for bounded factors (heading or orientation).
    Stated in Sec. 4-5 as an empirical observation guiding algorithm assignment; the mixed approach depends on this being true across environments.
  • ad hoc to paper The norm-matching-to-alignment curriculum improves skill training.
    App. A.1 introduces the curriculum with no theoretical justification; an alternative could fail.
invented entities (2)
  • Style factor
    purpose: Adds a separate skill dimension whose extrinsic reward encourages stable, safe postures (e.g., standing still), modulating safety versus diversity via factor weights.
    It is defined entirely by hand-chosen reward terms in Table 9; no external prediction is made.
  • Factor weight vector lambda
    purpose: Conditioning variable that prioritizes skill factors and style, intended to resolve conflicts between simultaneously learned skills.
    A new conditioning variable introduced in Sec. 4; the appendix shows it does not resolve conflicting quadrant factors, so its effectiveness is limited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Divide, Discover, Deploy: Factorized Skill Learning with Symmetry and Style Priors." pith.science (2026). https://pith.science/paper/TH2OYWLK

@misc{pith2026250819953,
  author       = {Pith},
  title        = {Pith review of: Divide, Discover, Deploy: Factorized Skill Learning with Symmetry and Style Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TH2OYWLK}},
  note         = {Machine review of arXiv:2508.19953}
}
read the original abstract

Unsupervised Skill Discovery (USD) allows agents to autonomously learn diverse behaviors without task-specific rewards. While recent USD methods have shown promise, their application to real-world robotics remains underexplored. In this paper, we propose a modular USD framework to address the challenges in the safety, interpretability, and deployability of the learned skills. Our approach employs user-defined factorization of the state space to learn disentangled skill representations. It assigns different skill discovery algorithms to each factor based on the desired intrinsic reward function. To encourage structured morphology-aware skills, we introduce symmetry-based inductive biases tailored to individual factors. We also incorporate a style factor and regularization penalties to promote safe and robust behaviors. We evaluate our framework in simulation using a quadrupedal robot and demonstrate zero-shot transfer of the learned skills to real hardware. Our results show that factorization and symmetry lead to the discovery of structured human-interpretable behaviors, while the style factor and penalties enhance safety and diversity. Additionally, we show that the learned skills can be used for downstream tasks and perform on par with oracle policies trained with hand-crafted rewards.

Figures

Figures reproduced from arXiv: 2508.19953 by the authors.

Figure 1
Figure 1. Approach overview. The agent’s state s is factorized by the user into N components, each paired with a latent skill zi and an intrinsic reward ri, selected from METRA or DIAYN objectives. An extrinsic reward rstyle promotes safe behaviors. The factor weights λ allow the agent to prioritize certain factors during training. The policy π is trained using on-policy RL with symmetry augmentation to discover structured, m… view at source ↗
Figure 2
Figure 2. Proposed algorithm for skill discovery. The agent πθ, conditioned on a sampled skill z and factor weights λ, collects transitions and receives a total reward combining per-factor intrinsic rewards and a style reward. The transitions are then augmented via symmetry-based mirroring, after which the intrinsic reward models, factorized value functions, and policy are updated using on-policy RL. More formally, the object… view at source ↗
Figure 3
Figure 3. Deployment of learned skills on the real robot. The learned structured skill space enables intuitive and composable control. Each behavior corresponds to a manually commanded skill z, set by adjusting individual skill factors zi. This results in diverse behaviors: pitching, walking, ducking, rotating and their combinations. Here we show walking while pitching, and ducking while rotating in the top and bottom rows re… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Effect of factor weighting on skill metrics. The metric score (details in App. A.4) reflects either dis￾criminator classification accuracy (state factors) or style reward (style factor). Incorporating per-factor weights λ enables the agent to prioritize relevant factor…
Figure 5
Figure 5. Figure 5: Impact of symmetry augmentation on skill-to-state mappings. Each point shows roll and pitch angles (in radians) reached by the policy, colored by the commanded skill. Without symmetry augmentation, the mapping is arbitrary and less structured. With symmetry augmentatio…
Figure 6
Figure 6. Figure 6: Environments for more complex skill discovery. (a) Adding a box pose factor encourages pushing, [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Environments used for learning skills. These environments are generated procedurally using the same mechanism as in Rudin et al. [38]. The policy receives the height-scan for perceiving the different terrains. A.4 Evaluation Metrics Metric Score. The metric score, a va…
Figure 8
Figure 8. Figure 8: Effect of factor weighting on conflicting fac [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Effect of symmetry augmentation on skill [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 38 canonical work pages

  1. [1]

    T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics, 2022

    work page 2022

  2. [2]

    Kaufmann, L

    E. Kaufmann, L. Bauersfeld, A. Loquercio, M. M ¨uller, V . Koltun, and D. Scaramuzza. Champion-level drone racing using deep reinforcement learning. Nature, 2023

    work page 2023

  3. [3]

    Akkaya, M

    I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, et al. Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113, 2019. 9

    arXiv 1910

  4. [4]

    Eysenbach, A

    B. Eysenbach, A. Gupta, J. Ibarz, and S. Levine. Diversity is all you need: Learning diverse skills without a reward function. In International Conference on Learning Representations (ICLR), 2018

    work page 2018

  5. [5]

    S. Park, O. Rybkin, and S. Levine. METRA: Scalable unsupervised rl with metric-aware abstraction. In International Conference on Learning Representations (ICLR) , 2024

    work page 2024

  6. [6]

    J. Hu, Z. Wang, P. Stone, and R. Mart´ın-Mart´ın. Disentangled unsupervised skill discovery for efficient hierarchical reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    work page 2024

  7. [7]

    Sharma, M

    A. Sharma, M. Ahn, S. Levine, V . Kumar, K. Hausman, and S. Gu. Emergent real-world robotic skills via unsupervised off-policy reinforcement learning. In Robotics: Science and Systems (RSS), 2020

    work page 2020

  8. [8]

    Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning

    V . Atanassov, W. Yu, A. L. Mitchell, M. N. Finean, and I. Havoutis. Constrained skill dis- covery: Quadruped locomotion with unsupervised reinforcement learning. arXiv preprint arXiv:2410.07877, 2024

    work page Pith review arXiv 2024

Show all 43 references
  1. [9]

    S. Kim, J. Kwon, T. Lee, Y . Park, and J. Perez. Safety-aware unsupervised skill discovery. In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA) , 2023

    2023

  2. [10]

    H. Kim, B. Lee, H. Lee, D. Hwang, D. Kim, and J. Choo. Do’s and don’ts: Learning desirable skills with instruction videos. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  3. [11]

    Sharma, S

    A. Sharma, S. Gu, S. Levine, V . Kumar, and K. Hausman. Dynamics-aware unsupervised discovery of skills. In International Conference on Learning Representations (ICLR) , 2020

    2020

  4. [12]

    D. J. Strouse, K. Baumli, D. Warde-Farley, V . Mnih, and S. Hansen. Learning more skills through optimistic exploration. In International Conference on Learning Representations (ICLR), 2022

    2022

  5. [13]

    Burda, H

    Y . Burda, H. Edwards, A. Storkey, and O. Klimov. Exploration by random network distillation. In International Conference on Learning Representations (ICLR) , 2018

    2018

  6. [14]

    R. Y . Chen, S. Sidor, P. Abbeel, and J. Schulman. Ucb exploration via q-ensembles. arXiv preprint arXiv:1706.01502, 2017

    2017 arXiv

  7. [15]

    L. Lee, B. Eysenbach, E. Parisotto, E. Xing, S. Levine, and R. Salakhutdinov. Efficient exploration via state marginal matching. arXiv preprint arXiv:1906.05274, 2019

    1906 arXiv

  8. [16]

    Osband, D

    I. Osband, D. Russo, and B. Van Roy. More efficient reinforcement learning via posterior sampling. Advances in Neural Information Processing Systems (NeurIPS) , 2013

    2013

  9. [17]

    Osband, C

    I. Osband, C. Blundell, A. Pritzel, and B. Van Roy. Deep exploration via bootstrapped dqn. Advances in Neural Information Processing Systems (NeurIPS) , 2016

    2016

  10. [18]

    Sukhija, S

    B. Sukhija, S. Coros, A. Krause, P. Abbeel, and C. Sferrazza. Maxinforl: Boosting ex- ploration in reinforcement learning through information gain maximization. arXiv preprint arXiv:2412.12098, 2024

    2024 arXiv

  11. [19]

    S. Park, J. Choi, J. Kim, H. Lee, and G. Kim. Lipschitz-constrained unsupervised skill discovery. In International Conference on Learning Representations (ICLR) , 2022

    2022

  12. [20]

    S. Park, K. Lee, Y . Lee, and P. Abbeel. Controllability-aware unsupervised skill discovery. In International Conference on Machine Learning (ICML) , 2023

    2023

  13. [21]

    S. Rho, L. Smith, T. Li, S. Levine, X. B. Peng, and S. Ha. Language guided skill discovery. In International Conference on Learning Representations (ICLR) , 2024. 10

    2024

  14. [22]

    Z. Wang, J. Hu, C. Chuck, S. Chen, R. Mart´ın-Mart´ın, A. Zhang, S. Niekum, and P. Stone. Skild: Unsupervised skill discovery guided by factor interactions. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  15. [23]

    D. O. Apraez, G. Turrisi, V . Kostic, M. Martin, A. Agudo, F. Moreno-Noguer, M. Pontil, C. Semini, and C. Mastalli. Morphological symmetries in robotics. International Journal of Robotics Research (IJRR), 2025

    2025

  16. [24]

    Z. Su, X. Huang, D. Ordo˜nez-Apraez, Y . Li, Z. Li, Q. Liao, G. Turrisi, M. Pontil, C. Semini, Y . Wu, and K. Sreenath. Leveraging symmetry in rl-based legged locomotion control. In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) , 2024

    2024

  17. [25]

    Mittal, N

    M. Mittal, N. Rudin, V . Klemm, A. Allshire, and M. Hutter. Symmetry considerations for learning task symmetric robot policies. In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), 2024

    2024

  18. [26]

    Cheng, M

    J. Cheng, M. Vlastelica, P. Kolev, C. Li, and G. Martius. Learning diverse skills for local navigation under multi-constraint optimality. In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), 2024

    2024

  19. [27]

    Zahavy, Y

    T. Zahavy, Y . Schroecker, F. Behbahani, K. Baumli, S. Flennerhag, S. Hou, and S. Singh. Discovering policies with domino: Diversity optimization maintaining near optimality. In International Conference on Learning Representations (ICLR) , 2023

    2023

  20. [28]

    Vlastelica, J

    M. Vlastelica, J. Cheng, G. Martius, and P. Kolev. Offline diversity maximization under imitation constraints. In Reinforcement Learning Conference, 2024

    2024

  21. [29]

    Kolev, M

    P. Kolev, M. Vlastelica, and G. Martius. Dual-force: Enhanced offline diversity maximization under imitation constraints. In Seventeenth European Workshop on Reinforcement Learning, 2025

    2025

  22. [30]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  23. [31]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning (ICML), 2018

    2018

  24. [32]

    Ravindran and A

    B. Ravindran and A. G. Barto. Symmetries and model minimization in markov decision processes. Technical report, University of Massachusetts, USA, 2001

    2001

  25. [33]

    D´enes and A

    J. D´enes and A. Keedwell. Latin Squares and Their Applications . Academic Press, 1974. URL https://books.google.ch/books?id=W2IPAQAAMAAJ

    1974

  26. [34]

    Mittal, C

    M. Mittal, C. Yu, Q. Yu, J. Liu, N. Rudin, D. Hoeller, J. L. Yuan, R. Singh, Y . Guo, H. Mazhar, A. Mandlekar, B. Babich, G. State, M. Hutter, and A. Garg. Orbit: A unified simulation framework for interactive robot learning environments. IEEE Robotics and Automation Letters (...

    2023

  27. [35]

    Rudin, D

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning (CoRL) , 2021

    2021

  28. [36]

    P. Arm, M. Mittal, H. Kolvenbach, and M. Hutter. Pedipulate: Enabling manipulation skills using a quadruped robot’s leg. In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), 2024

    2024

  29. [37]

    Imagawa, T

    T. Imagawa, T. Hiraoka, and Y . Tsuruoka. Unsupervised discovery of continuous skills on a sphere. arXiv preprint arXiv:2305.14377, 2023

    2023 arXiv

  30. [38]

    Rudin, D

    N. Rudin, D. Hoeller, M. Bjelonic, and M. Hutter. Advanced skills by learning locomotion and local navigation end-to-end. In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2022. 11 A Appendix A.1 Unsupervised Skill Discovery Algorithms DIAYN: Divers...

    2022

  31. [39]

    So far, symmetry biases have not been used as part of unsupervised skill discovery

    propose to simply augment the collected data instead of introducing an extra symmetry objective, or enforcing symmetry in the network architecture. So far, symmetry biases have not been used as part of unsupervised skill discovery. However, it might be useful to learn symmetri...

  32. [40]

    Sample a large number of skills, n >10, 000, from the prior p(z)

  33. [41]

    For each skill, execute a full rollout with the policy to collect a trajectory of states

  34. [42]

    Calculate the mean state for each of the n trajectories

  35. [43]

    This final standard deviation serves as our diversity metric, where higher values indicate broader state coverage

    Calculate the standard deviation over these n mean states. This final standard deviation serves as our diversity metric, where higher values indicate broader state coverage. A.5 Additional Results and Discussion In this section, we provide additional details and insights for t...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.