Pith. sign in

REVIEW 3 major objections 7 minor 50 references

Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GO-Skill claims that distilling offline trajectories into a discrete library of goal-oriented skills and selecting among them with a hierarchical policy improves average episode return on MetaWorld relative to action-level…

desk verdict A solid empirical skill-abstraction method with a real ambiguity in the state-difference representation and some evaluation gaps, but worth a serious referee. read the letter →

arxiv 2507.06628 v1 pith:YZXSX3XY submitted 2025-07-09 cs.LG cs.AI

Jinmin He , Kai Li , Yifan Zang , Haobo Fu , Qiang Fu , Junliang Xing , Jian Cheng This is my paper
classification cs.LGcs.AI
keywords offlinereinforcementlearningmulti-taskskillabstractiongoal-orientedrepresentationvectorquantizationhierarchicalpolicyroboticmanipulationMetaWorld
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GO-Skill is an offline multi-task reinforcement learning method that claims to improve average episode return on the MetaWorld robotic manipulation benchmark by first distilling a discrete library of reusable skills from task-mixed offline data, then learning a high-level policy that selects among those skills. The paper argues that representing a skill as the state difference $s_{t+H} - s_t$ over a fixed horizon $H$ gives a task-agnostic goal description, and that vector quantization turns these goal embeddings into a discrete skill codebook. A skill-enhancement phase resamples under-represented skills, and a Prompt-DT-style policy with focal loss orchestrates skill selection. On MT30 and MT50 under both Near-Optimal and Sub-Optimal datasets, GO-Skill reports higher average episode return than six baselines, with the advantage growing as the number of tasks increases. The central claim is that explicit goal-oriented skill abstraction is an effective knowledge-sharing mechanism for offline multi-task RL.

What carries the argument

The load-bearing object is the goal-oriented skill, defined as a fixed-horizon state transition $s_{t+H} - s_t$ encoded by a goal encoder $G$ into an embedding $z_{t,H} = G(s_{t+H} - s_t)$, then quantized against a codebook $\mathcal{Z} = \{\hat{z}^{(1)}, \ldots, \hat{z}^{(M)}\}$ via vector quantization. This codebook is the discrete skill library; a skill decoder $P$ reconstructs actions from the skill embedding plus the reached-goal history, and a skill-based decision transformer selects skill indices at $H$-step intervals. The goal representation excludes rewards and action sequences, which the paper argues makes skills transferable across tasks whose reward functions differ despite identical trajectories.

What would settle it

Train GO-Skill on a pair of MetaWorld tasks where the same state displacement corresponds to different required contacts (for example, a task that needs a grasp and a task that needs a push from the same start and end positions), and check whether the learned codebook assigns the same skill index to both sub-trajectories and whether evaluation return drops relative to a model that includes object identity in the goal. If performance does not degrade, the state-difference assumption is sufficient for that setting; if it does, the claim that state differences are task-agnostic skills is falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that goal-oriented skill abstraction—encoding an $H$-horizon behavior by the change in state it produces rather than by its action sequence—lets an offline agent extract reusable, task-agnostic skills from mixed-quality multi-task data, and that a hierarchical policy acting in the resulting discrete skill space solves more tasks better than action-level sequence models. The authors show that a Transformer-based skill decoder conditioned on a quantized goal embedding can reproduce the actions of an $H$-step sub-trajectory, and that a high-level skill-based decision transformer can learn to chain these skills. On MetaWorld MT30 and MT50, GO-Skill achieves the highest average episode return among the compared offline multi-task methods in both Near-Optimal and Sub-Optimal data settings, and its margin over baselines widens from 30 to 50 tasks. The evidence used to support this includes ablations showing that removing the reached-goal history, replacing discrete vector quantization with a continuous skill space, or replacing goal-based encoding with action-based encoding each lowers performance.

Load-bearing premise

The method assumes that the state difference over a fixed horizon is a sufficient, task-agnostic description of a reusable skill; if two behaviors with the same state difference require different actions or different task context, the skill codebook and the policy built on it can be misspecified.

Editorial extensions

If this is right

  • GO-Skill reports higher average episode return than six offline multi-task baselines on MetaWorld MT30 and MT50 under both Near-Optimal and Sub-Optimal data, and the advantage grows as the task count increases.
  • The method extracts useful skill fragments from sub-optimal trajectories, so lower-quality offline data still yields transferable skills for other tasks.
  • Ablations show that each component contributes: reached-goal history, discrete vector quantization, goal-based encoding, resampling during skill enhancement, and focal loss during policy learning all matter for the reported performance.
  • In the ML45 fine-tuning setup, GO-Skill reaches higher episode return after only 3,000 fine-tuning iterations on five new tasks than MTDT and PromptDT.
  • Because the skill-based policy makes decisions at $H$-step intervals instead of every time step, the decision horizon shortens, which the paper argues simplifies policy learning and improves sample efficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the state-difference goal representation could be tested against goal representations that include object identity or contact features; the paper itself notes that more robust goal-oriented representations remain an open direction.
  • A testable extension is whether the same learned skill codebook transfers across different benchmarks or embodiments without fine-tuning, since the paper only demonstrates re-use through fine-tuning within MetaWorld.
  • Because the paper identifies the predefined skill set size and skill horizon as limitations, an adaptive or growing codebook is a natural follow-up that could reduce sensitivity to these two hyper-parameters.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes GO-Skill, a hierarchical offline multi-task reinforcement learning method. A skill model encodes H-step sub-trajectories through the state difference s_{t+H} - s_t, maps them to a latent goal embedding, and vector-quantizes these embeddings into a discrete skill codebook. A Transformer-based skill decoder reconstructs actions conditioned on the skill embedding, the state history, and the reached-goal history, and a skill enhancement phase resamples skill classes to address class imbalance. A high-level skill-based Decision Transformer, trained with focal loss, selects skill indices at intervals of H steps. The method is evaluated on MetaWorld MT30/MT50 with near-optimal and sub-optimal offline datasets, compared against six baselines, and ablated on its components and hyperparameters; a fine-tuning experiment on ML45 is also reported.

Significance. If the performance claims hold, the paper makes a useful contribution by showing that state-difference-based skill abstraction with vector quantization can serve as an effective hierarchical inductive bias for offline multi-task RL, with strong aggregate results on MT30/MT50. The evaluation is broad and includes five seeds, component ablations, hyperparameter sweeps, skill visualizations, and per-task results for the proposed method. The method is empirical rather than a theoretical derivation, so the main claims are falsifiable benchmark comparisons rather than parameter-free predictions. The paper also provides a clear and actionable design: reward-free goal encoding, discrete codebook, class-imbalance mitigation, and a Prompt-DT-style high-level policy.

major comments (3)
  1. [Section 5.2, Figure 5] The headline claim that GO-Skill achieves superior performance compared to existing state-of-the-art approaches is not supported by any significance testing. The reported error bars overlap for some of the comparisons, particularly in the MT30 Sub-Optimal setting, and only five seeds are used; Table 6 shows very large per-task standard deviations (for example, assembly-v2 at 2366.7±1082.3 and pick-place-v2 at 358.6±409.1 in the Near-Optimal setting). The aggregate mean differences may therefore not be statistically reliable. Please add paired significance tests across tasks (for example, a Wilcoxon signed-rank test of GO-Skill against each baseline per task) or bootstrap confidence intervals over tasks and seeds, and provide per-task results for all baselines rather than only for GO-Skill.
  2. [Section 3.1, Equations (4) and (7)] The skill representation is a potential misspecification risk that is load-bearing for the claimed transfer mechanism. The goal encoder maps the H-horizon sub-trajectory to only s_{t+H} - s_t, and at the first step of a skill the decoder conditions on the skill code, the current state, and an empty reached-goal and action history. Consequently, two H-step segments with the same start state and the same endpoint difference but different intermediate states or actions receive the same skill code, and the MSE action-reconstruction objective in Algorithm 1 must fit a single action distribution to potentially incompatible behaviors. The paper's motivation that different action sequences can lead to the same transfer outcome is only one side of this conflation. The AE ablation in Table 1, row 4, provides a relevant check, but it is confounded because that row also removes reached-goal history. Please add a controlled comparison in which both the goal-based and action-based encoders are evaluated with reached-goal history, and report codebook purity or per-skill action-reconstruction error to demonstrate that the learned skill codes are behaviorally coherent.
  3. [Section 5.3, Figure 6] The hyperparameters H, M, |Z|, and t_e are selected via ablations on the same MT50 Near-Optimal setup that is subsequently used for the main results, meaning the reported margins over baselines include selection on the evaluation tasks. This is a particular concern because the differences over the best baselines in Figure 5 are not large relative to the seed noise. Please either fix the hyperparameters before evaluating on the final task sets, report results on a held-out validation split, or show that the relative ranking of GO-Skill against the baselines is stable across the swept hyperparameter range.
minor comments (7)
  1. [Tables 1 and 2] The checkmark and cross symbols in the ablation tables are not defined in the captions; please add an explicit legend.
  2. [Algorithm 2] The symbol T' is used in the focal loss computation before it is defined; clarify that it denotes the decision point at which the skill index is predicted.
  3. [Figure 5] Because the error bars overlap and the plotted values are difficult to read, please also provide a numeric table of mean and standard deviation for every method and setting.
  4. [Section 5.2, MH-GO-Skill] The multi-head variant of GO-Skill is described only briefly; please specify the number of heads, where they are inserted in the skill model, and how they are trained and combined.
  5. [Section 5.5, Figure 8] The fine-tuning result is presented as single learning curves without error bars or per-task breakdowns; please report mean and standard deviation over seeds and a table for the five new tasks.
  6. [Conclusion and Discussion] The Limitations paragraph identifies the reliance on a predefined skill horizon and skill set size, but it does not discuss the sufficiency or ambiguity of the state-difference representation in Section 3.1; acknowledging this would help readers calibrate the scope of the claims.
  7. [Reproducibility] The paper does not include a code availability statement; releasing the implementation and random seeds would substantially aid reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GO-Skill's effectiveness is established by external benchmarks, and its skill-abstraction design choices are not fitted inputs renamed as predictions.

full rationale

GO-Skill is an empirical method, not a derivation. The central claim—superior performance on MetaWorld MT30/MT50 under Near-Optimal and Sub-Optimal data—is evaluated against external baselines (MTDT, PromptDT, MTDIFF-P, HarmoDT, etc.), so the outcome is not an input to the method. The goal-oriented skill representation in Eq. 4 (zt,H = G(st+H − st)) is a design choice, not a fitted parameter whose value is later reported as a prediction. The skill codebook is learned by vector quantization, the decoder is trained with an action-reconstruction MSE loss, and the high-level policy selects discrete skill indices; each stage is a genuine learning step rather than a rename of the data. Self-citations to He et al. (2024a, 2024b) appear only in related work and in stating the common MTRL setting, and they are not load-bearing for the paper's claims. The ablation against an action-based skill encoder (Table 1) and the external benchmark comparisons provide independent evidence for the design choices. Therefore, no circular step can be exhibited from the paper's equations or citations.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hyperparameters chosen via ablation on the benchmark itself, and on the modeling assumption that state differences capture reusable skills. No new physical or conceptual entities are introduced beyond learned skill codes, which are internal representations.

free parameters (6)
  • Skill horizon H = 10
    Controls temporal abstraction length; chosen via ablation on MT50 Near-Optimal (Figure 6) and reported as the most impactful hyperparameter.
  • Codebook size M = 16
    Number of discrete skill codes; chosen via ablation on the same benchmark, so the model's capacity is tuned to the test distribution.
  • Goal embedding size |Z| = 64
    Dimension of goal/skill embeddings; selected by ablation (Figure 6).
  • Skill extraction iterations t_e = 30000 (30%)
    Iterations before freezing the goal encoder and codebook; selected to balance convergence and total budget.
  • Focal loss focusing parameter gamma = 2
    Set to 2 in Table 5; a standard value, not ablated in the paper.
  • VQ loss weight alpha = not specified
    The paper does not state the value of alpha in Eq. 3, leaving an unspecified tuning knob that affects the skill codebook.
assumptions (4)
  • domain assumption All tasks in the offline MTRL setting share the same state and action spaces.
    Section 2 establishes this MTRL setting; the goal encoder and skill decoder rely on a common state representation.
  • domain assumption The state difference over an H-horizon is a sufficient and task-agnostic representation of a reusable skill.
    Eq. (4) defines skills via state differences; this is the central modeling choice and is not verified independently.
  • domain assumption A fixed finite codebook (M=16) can represent all behavior needed for the task set.
    Section 3.1 and Table 5 fix M; the paper does not analyze codebook saturation or collisions.
  • standard math The stop-gradient VQ loss (Eq. 3) from Van Den Oord et al. (2017) trains a stable discrete codebook.
    Borrowed technique, not derived in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning." pith.science (2026). https://pith.science/paper/YZXSX3XY

@misc{pith2026250706628,
  author       = {Pith},
  title        = {Pith review of: Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YZXSX3XY}},
  note         = {Machine review of arXiv:2507.06628}
}
read the original abstract

Offline multi-task reinforcement learning aims to learn a unified policy capable of solving multiple tasks using only pre-collected task-mixed datasets, without requiring any online interaction with the environment. However, it faces significant challenges in effectively sharing knowledge across tasks. Inspired by the efficient knowledge abstraction observed in human learning, we propose Goal-Oriented Skill Abstraction (GO-Skill), a novel approach designed to extract and utilize reusable skills to enhance knowledge transfer and task performance. Our approach uncovers reusable skills through a goal-oriented skill extraction process and leverages vector quantization to construct a discrete skill library. To mitigate class imbalances between broadly applicable and task-specific skills, we introduce a skill enhancement phase to refine the extracted skills. Furthermore, we integrate these skills using hierarchical policy learning, enabling the construction of a high-level policy that dynamically orchestrates discrete skills to accomplish specific tasks. Extensive experiments on diverse robotic manipulation tasks within the MetaWorld benchmark demonstrate the effectiveness and versatility of GO-Skill.

Figures

Figures reproduced from arXiv: 2507.06628 by the authors.

Figure 1
Figure 1. Skill sets shared across different domains. (a) Skill set for human learning to operate four types of vehicles: bicycles and motorcycles share the skill of balancing on two wheels; unicycles and bicycles share the pedal-driven skill; motorcycles and auto￾mobiles share the ignition-to-move skill. (b) Skill set in robotic manipulation tasks: grabbing various handles, pressing buttons at different heights, and picking … view at source ↗
Figure 2
Figure 2. Illustration of the skill model in GO-Skill, which comprises three key components: (1) Goal Encoder maps the trajectory-level difference between states to a latent embedding space; (2) Skill Codebook distills the H-horizon goal embedding into a discrete skill set via vector quantization; (3) Skill Decoder reconstructs actions with skill prompt and history sequence using a Transformer architecture. Vector Quantizatio… view at source ↗
Figure 3
Figure 3. Illustration of the skill-based policy in GO-Skill. The policy, employing the Transformer architecture, uses the discrete skill space as the action space. Its inputs include (return-to-go, state, skill) prompts, and H-step interval historical trajectory. At each decision point, the policy predicts a skill index, which is subsequently mapped to its corresponding skill embedding via the skill codebook. Skill Decoder. … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Deploying GO-Skill as a control policy to interact with the environment. T represents the decision points of the skill-based policy at intervals of H time steps, while t indicates the time steps where the skill model predicts actions within the H-horizon. icy are fine-…
Figure 5
Figure 5. Figure 5: Average episode return across 5 random seeds on the MetaWorld benchmark with four different setups. The error bars represent the standard deviation across the 5 seeds. Each method is trained for 1e5 iterations, and each task is evaluated over 50 episodes. dynamics, yie…
Figure 6
Figure 6. Figure 6: Ablation study on the hyper-parameter of GO-Skill under the Near-Optimal setting in the MT50 setup. Default parameter values are set as skill horizon H = 10, codebook size M = 16, code embedding size |Z| = 64, and extraction iteration te = 3e4 (30% of total iteration).…
Figure 8
Figure 8. Figure 8: Fine-tuning on five new tasks under Near-Optimal and Sub-Optimal settings in the ML45 setup demonstrates that GO￾Skill is highly effective for transfer scenarios. this, the data associated with these unused skills can sup￾port other tasks in leveraging those skills eff…
Figure 9
Figure 9. Figure 9: Visualization of the five skills in GO-Skill. The three images represent the states within the H-horizon. D. Single-Task Performance We present the detailed performance evaluation of GO-Skill on the MT50 setup in Section 5.2, with results for each task provided in [PI…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 45 canonical work pages

  1. [1]

    Dynamic programming

    Bellman, R. Dynamic programming. Science, 153 0 (3731): 0 34--37, 1966

    work page 1966

  2. [2]

    End-to-end object detection with transformers

    Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers. In Proceedings of European Conference on Computer Vision, pp.\ 213--229, 2020

    work page 2020

  3. [3]

    Multitask learning

    Caruana, R. Multitask learning. Machine Learning, 28: 0 41--75, 1997

    work page 1997

  4. [4]

    Offline reinforcement learning via high-fidelity generative behavior modeling

    Chen, H., Lu, C., Ying, C., Su, H., and Zhu, J. Offline reinforcement learning via high-fidelity generative behavior modeling. In Proceedings of International Conference on Learning Representations, pp.\ 1--12, 2023

    work page 2023

  5. [5]

    Decision transformer: Reinforcement learning via sequence modeling

    Chen, L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., and Mordatch, I. Decision transformer: Reinforcement learning via sequence modeling. In Advances in Neural Information Processing Systems, pp.\ 15084--15097, 2021

    work page 2021

  6. [6]

    Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks

    Chen, Z., Badrinarayanan, V., Lee, C.-Y., and Rabinovich, A. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In Proceedings of International Conference on Machine Learning, pp.\ 794--803, 2018

    work page 2018

  7. [7]

    Sharing knowledge in multi-task deep reinforcement learning

    D'Eramo, C., Tateo, D., Bonarini, A., Restelli, M., and Peters, J. Sharing knowledge in multi-task deep reinforcement learning. In Proceedings of International Conference on Learning Representations, pp.\ 1--11, 2020

    work page 2020

  8. [8]

    BERT : Pre-training of deep bidirectional transformers for language understanding

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT : Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pp.\ 4171--4186, 2019

    work page 2019

Show all 50 references
  1. [9]

    Behavior retrieval: Few-shot imitation learning by querying unlabeled datasets

    Du, M., Nair, S., Sadigh, D., and Finn, C. Behavior retrieval: Few-shot imitation learning by querying unlabeled datasets. CoRR, abs/2304.08742, 2023

    2023 arXiv

  2. [10]

    Off-policy deep reinforcement learning without exploration

    Fujimoto, S., Meger, D., and Precup, D. Off-policy deep reinforcement learning without exploration. In Proceedings of International Conference on Machine Learning, pp.\ 2052--2062, 2019

    2019

  3. [11]

    Offline rl policies should be trained to be adaptive

    Ghosh, D., Ajay, A., Agrawal, P., and Levine, S. Offline rl policies should be trained to be adaptive. In Proceedings of International Conference on Machine Learning, pp.\ 7513--7530, 2022

    2022

  4. [12]

    D., Avoli, M., and Pineau, J

    Guez, A., Vincent, R. D., Avoli, M., and Pineau, J. Adaptive treatment of epilepsy via batch-mode reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.\ 1671--1678, 2008

    2008

  5. [13]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of International Conference on Machine Learning, pp.\ 1856--1865, 2018

    2018

  6. [14]

    Hierarchical few-shot imitation with skill transition models

    Hakhamaneshi, K., Zhao, R., Zhan, A., Abbeel, P., and Laskin, M. Hierarchical few-shot imitation with skill transition models. In Proceedings of International Conference on Learning Representations, pp.\ 1--12, 2021

    2021

  7. [15]

    Diffusion model is an effective planner and data synthesizer for multi-task reinforcement learning

    He, H., Bai, C., Xu, K., Yang, Z., Zhang, W., Wang, D., Zhao, B., and Li, X. Diffusion model is an effective planner and data synthesizer for multi-task reinforcement learning. In Advances in Neural Information Processing Systems, pp.\ 64896--64917, 2023

    2023

  8. [16]

    Efficient multi-task reinforcement learning with cross-task policy guidance

    He, J., Li, K., Zang, Y., Fu, H., Fu, Q., Xing, J., and Cheng, J. Efficient multi-task reinforcement learning with cross-task policy guidance. In Advances in Neural Information Processing Systems, pp.\ 117997--118024, 2024 a

    2024

  9. [17]

    Not all tasks are equally difficult: Multi-task deep reinforcement learning with dynamic depth routing

    He, J., Li, K., Zang, Y., Fu, H., Fu, Q., Xing, J., and Cheng, J. Not all tasks are equally difficult: Multi-task deep reinforcement learning with dynamic depth routing. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.\ 12376--12384, 2024 b

    2024

  10. [18]

    Harmodt: Harmony multi-task decision transformer for offline reinforcement learning

    Hu, S., Fan, Z., Shen, L., Zhang, Y., Wang, Y., and Tao, D. Harmodt: Harmony multi-task decision transformer for offline reinforcement learning. In Proceedings of International Conference on Machine Learning, pp.\ 19182--19197, 2024

    2024

  11. [19]

    B., and Levine, S

    Janner, M., Du, Y., Tenenbaum, J. B., and Levine, S. Planning with diffusion for flexible behavior synthesis. In Proceedings of International Conference on Machine Learning, pp.\ 9902--9915, 2022

    2022

  12. [20]

    Scalable deep reinforcement learning for vision-based robotic manipulation

    Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., and Levine, S. Scalable deep reinforcement learning for vision-based robotic manipulation. In Proceedings of Conference on Robot Learning, pp.\ 651...

    2018

  13. [21]

    Mt-opt: Continuous multi-task robotic reinforcement learning at scale

    Kalashnikov, D., Varley, J., Chebotar, Y., Swanson, B., Jonschkowski, R., Finn, C., Levine, S., and Hausman, K. Mt-opt: Continuous multi-task robotic reinforcement learning at scale. CoRR, abs/2104.08212, 2021

    2021 arXiv

  14. [22]

    W., Zhang, H., Subramanian, J., Fatemi, M., and Ghassemi, M

    Killian, T. W., Zhang, H., Subramanian, J., Fatemi, M., and Ghassemi, M. An empirical study of representation learning for reinforcement learning in healthcare. CoRR, abs/2011.11235, 2020

    2011 arXiv

  15. [23]

    Offline reinforcement learning with implicit q-learning

    Kostrikov, I., Nair, A., and Levine, S. Offline reinforcement learning with implicit q-learning. In Proceedings of International Conference on Learning Representations, pp.\ 1--11, 2021

    2021

  16. [24]

    Conservative q-learning for offline reinforcement learning

    Kumar, A., Zhou, A., Tucker, G., and Levine, S. Conservative q-learning for offline reinforcement learning. In Advances in Neural Information Processing Systems, pp.\ 1179--1191, 2020

    2020

  17. [25]

    S., Lee, L., Freeman, D., Guadarrama, S., Fischer, I., Xu, W., Jang, E., Michalewski, H., et al

    Lee, K.-H., Nachum, O., Yang, M. S., Lee, L., Freeman, D., Guadarrama, S., Fischer, I., Xu, W., Jang, E., Michalewski, H., et al. Multi-game decision transformers. In Advances in Neural Information Processing Systems, pp.\ 27921--27936, 2022

    2022

  18. [26]

    End-to-end training of deep visuomotor policies

    Levine, S., Finn, C., Darrell, T., and Abbeel, P. End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research, 17 0 (1): 0 1334--1373, 2016

    2016

  19. [27]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems

    Levine, S., Kumar, A., Tucker, G., and Fu, J. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. CoRR, abs/2005.01643, 2020

    2005 arXiv

  20. [28]

    P., Hunt, J

    Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. Continuous control with deep reinforcement learning. In Proceedings of International Conference on Learning Representations, pp.\ 1--10, 2016

    2016

  21. [29]

    Focal loss for dense object detection

    Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollar, P. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp.\ 2999--3007, 2017

    2017

  22. [30]

    Conflict-averse gradient descent for multi-task learning

    Liu, B., Liu, X., Jin, X., Stone, P., and Liu, Q. Conflict-averse gradient descent for multi-task learning. In Advances in Neural Information Processing Systems, pp.\ 18878--18890, 2021

    2021

  23. [31]

    J., Kumar, V., Zhang, A., Bastani, O., and Jayaraman, D

    Ma, Y. J., Kumar, V., Zhang, A., Bastani, O., and Jayaraman, D. Liv: language-image representations and rewards for robotic control. In Proceedings of International Conference on Machine Learning, pp.\ 23301--23320, 2023

    2023

  24. [32]

    A., Veness, J., Bellemare, M

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M. A., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. Human-level cont...

    2015

  25. [33]

    R3m: a universal visual representation for robot manipulation

    Nair, S., Rajeswaran, A., Kumar, V., Finn, C., and Gupta, A. R3m: a universal visual representation for robot manipulation. In Proceedings of Conference on Robot Learning, pp.\ 892--909, 2022

    2022

  26. [34]

    Learning and retrieval from prior data for skill-based imitation learning

    Nasiriany, S., Gao, T., Mandlekar, A., and Zhu, Y. Learning and retrieval from prior data for skill-based imitation learning. In Proceedings of Conference on Robot Learning, pp.\ 2181--2204, 2023

    2023

  27. [35]

    Accelerating reinforcement learning with learned skill priors

    Pertsch, K., Lee, Y., and Lim, J. Accelerating reinforcement learning with learned skill priors. In Proceedings of Conference on Robot Learning, pp.\ 188--204, 2021

    2021

  28. [36]

    Pertsch, K., Lee, Y., Wu, Y., and Lim, J. J. Demonstration-guided reinforcement learning with learned skills. In Proceedings of Conference on Robot Learning, pp.\ 729--739, 2022

    2022

  29. [37]

    Puterman, M. L. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, 2014

    2014

  30. [38]

    Hierarchical and interpretable skill acquisition in multi-task reinforcement learning

    Shu, T., Xiong, C., and Socher, R. Hierarchical and interpretable skill acquisition in multi-task reinforcement learning. In Proceedings of International Conference on Learning Representations, pp.\ 1--11, 2018

    2018

  31. [39]

    Multi-task reinforcement learning with context-based representations

    Sodhani, S., Zhang, A., and Pineau, J. Multi-task reinforcement learning with context-based representations. In Proceedings of International Conference on Machine Learning, pp.\ 9767--9779, 2021

    2021

  32. [40]

    MuJoCo : A physics engine for model-based control

    Todorov, E., Erez, T., and Tassa, Y. MuJoCo : A physics engine for model-based control. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.\ 5026--5033, 2012

    2012

  33. [41]

    Neural discrete representation learning

    Van Den Oord, A., Vinyals, O., et al. Neural discrete representation learning. In Advances in Neural Information Processing Systems, pp.\ 6306--6315, 2017

    2017

  34. [42]

    N., Kaiser, L

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems, pp.\ 5998--6008, 2017

    2017

  35. [43]

    J., and Zhou, M

    Wang, Z., Hunt, J. J., and Zhou, M. Diffusion policies as an expressive policy class for offline reinforcement learning. In Proceedings of International Conference on Learning Representations, pp.\ 1--12, 2023

    2023

  36. [44]

    Prompting decision transformer for few-shot policy generalization

    Xu, M., Shen, Y., Zhang, S., Lu, Y., Zhao, D., Tenenbaum, J., and Gan, C. Prompting decision transformer for few-shot policy generalization. In Proceedings of International Conference on Machine Learning, pp.\ 24631--24645, 2022

    2022

  37. [45]

    Q-learning decision transformer: Leveraging dynamic programming for conditional sequence modelling in offline rl

    Yamagata, T., Khalil, A., and Santos-Rodriguez, R. Q-learning decision transformer: Leveraging dynamic programming for conditional sequence modelling in offline rl. In Proceedings of International Conference on Machine Learning, pp.\ 38989--39007, 2023

    2023

  38. [46]

    Multi-task reinforcement learning with soft modularization

    Yang, R., Xu, H., Wu, Y., and Wang, X. Multi-task reinforcement learning with soft modularization. In Advances in Neural Information Processing Systems, pp.\ 4767--4777, 2020

    2020

  39. [47]

    Mastering complex control in MOBA games with deep reinforcement learning

    Ye, D., Liu, Z., Sun, M., Shi, B., Zhao, P., Wu, H., Yu, H., Yang, S., Wu, X., Guo, Q., Chen, Q., Yin, Y., Zhang, H., Shi, T., Wang, L., Fu, Q., Yang, W., and Huang, L. Mastering complex control in MOBA games with deep reinforcement learning. In Proceedings of the AAAI Confere...

    2020

  40. [48]

    Gradient surgery for multi-task learning

    Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., and Finn, C. Gradient surgery for multi-task learning. In Advances in Neural Information Processing Systems, pp.\ 5824--5836, 2020 a

    2020

  41. [49]

    Meta-World : A benchmark and evaluation for multi-task and meta reinforcement learning

    Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. Meta-World : A benchmark and evaluation for multi-task and meta reinforcement learning. In Proceedings of the Conference on Robot Learning, pp.\ 1094--1100, 2020 b

    2020

  42. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.