Scaling Algorithm Distillation for Continuous Control with Mamba

Mehdi Mounsif; Samuel Beaussant

REVIEW 4 major objections 6 minor 37 references

Scaling Algorithm Distillation for Continuous Control with Mamba

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read State-space models outperform transformers for in-context RL in continuous control, the paper argues.

desk verdict A useful empirical study showing Mamba-based AD beats Decision Transformer AD on continuous control, but the headline comparison is weakened by per-model context selection and missing error bars. read the letter →

arxiv 2506.13892 v1 pith:APKZ2TDD submitted 2025-06-16 cs.LG cs.AIcs.RO

Samuel Beaussant , Mehdi Mounsif This is my paper

classification cs.LGcs.AIcs.RO

keywords AlgorithmDistillationIn-ContextReinforcementLearningMambaStateSpaceModelsMeta-ReinforcementContinuousControlLongContextDecisionTransformer

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that Algorithm Distillation—a meta-reinforcement-learning method that learns to imitate another RL agent's learning process from offline trajectories—can be scaled to continuous control by swapping the transformer for Mamba, a state-space model that processes sequences in linear time. Across four continuous-control benchmarks, the authors claim that Mamba-based AD reaches better asymptotic returns than a similarly sized Decision Transformer on every environment, with the gap growing on the hardest tasks. They also show that longer in-context histories improve performance on the more complex tasks, and that the resulting offline-trained agent is competitive with an online meta-RL baseline and outperforms an offline one. If right, the result removes a key bottleneck of AD—the quadratic cost of attention—and makes in-context RL practical for longer-horizon, higher-dimensional control.

What carries the argument

The load-bearing object is the Mamba architecture, built on S6 selective state-space layers, which maintain a recurrent hidden state and scale linearly in sequence length, in contrast to attention's quadratic cost. The second element is the Algorithm Distillation objective itself: a behavior-cloning loss that trains the model to predict actions from a history of past transitions, tokenized as tuples $(s_t, a_t, r_t, s_{t+1})$, so the model must implicitly approximate the policy-improvement operator of the source RL algorithm. Mamba's ability to carry long cross-episodic context is what lets AD scale to the longer horizons of continuous control.

What would settle it

Retrain the Decision Transformer on the same datasets with an extensive hyper-parameter search (including context length, learning rate, and model width), and compare asymptotic returns on the same four environments using a fixed protocol; if the tuned transformer matches or exceeds Mamba on any environment, the claimed universal superiority fails. A cheaper check: fix one context length for both architectures and verify that Mamba's advantage persists without per-model selection.

Watch

Extended reading notes

Core claim

The central claim is that, given similar model sizes, a Mamba architecture achieves better asymptotic performance than a causal transformer for Algorithm Distillation in every continuous-control setting considered, and that scaling AD to very long contexts improves in-context RL performance, making it competitive with state-of-the-art online meta-RL (MQL) while training purely offline. The authors establish this by pretraining both models on the same offline learning histories generated by source RL algorithms (PPO, SAC, and DroQ), conditioning on cross-episodic histories, and evaluating adaptation on unseen tasks. They report that Mamba outperforms the Decision Transformer on all four tasks—Reacher-Goal, Pusher-Goal, Half-Cheetah-Vel, and Ant-Dir—and that the optimal context length grows with task complexity. They further compare AD against MACAW and MQL, finding it superior to MACAW and competitive with MQL.

Load-bearing premise

That the Decision Transformer baseline was tuned with comparable effort and that choosing each model's best context length on validation does not systematically favor Mamba.

Editorial extensions

If this is right

Mamba-based AD can be pre-trained fully offline and still match online meta-RL methods like MQL on several continuous-control tasks.
Longer in-context histories, up to full down-sampled learning trajectories, improve ICRL performance on complex tasks, so linear-time models unlock a scaling axis that transformers cannot afford.
The optimal context length is task-dependent; simpler tasks benefit from shorter contexts, so context length becomes a key hyper-parameter for AD.
AD with Mamba systematically outperforms a Decision Transformer baseline of comparable size in all four environments tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Since Mamba scales linearly in sequence length, AD could be extended to much longer horizons (e.g., episodes of thousands of steps) or to multi-task setups without the memory wall of attention; a natural test is to run AD on a continuous task with episode length beyond what transformers can fit.
The tokenization choice (one transition per token) cuts context length by 3x; an ablation varying tokenization could reveal how much of Mamba's advantage comes from the architecture versus the compressed tokenization.
Because the paper only compared against Decision Transformer, a broader comparison against other linear-time architectures (e.g., RWKV or hybrid models) would clarify whether the benefit is specific to selectivity in S6 or general to sub-quadratic models.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Key thing to know: this is a real, useful empirical result. The authors take Algorithm Distillation, which had only been shown in discrete gridworlds, and run it on four continuous-control meta-RL tasks with Mamba as the sequence model. They find Mamba-based AD consistently beats a Decision Transformer AD baseline and that longer contexts help on the harder tasks. They also introduce a new Pusher-Goal environment. That is a genuine contribution, and the context-length study in Figure 2 is the most interesting part.

Now the soft spots. The main comparison is not as clean as the text suggests. Section 5.2 says the Decision Transformer was trained "with the same context length," but Section 5.3 says Figure 1 plots "the best performing context length for each model." Those can't both be true for the main figure. If the DT's best context is shorter only because of memory or compute limits, then the comparison is architecture plus context length, and Mamba's linear scaling is doing the work. The paper does not report exact context lengths or per-context results for the DT, so you can't verify. Also, only 3 pretraining seeds and no error bars on the learning curves; Mamba may genuinely be more stable, but the variance is invisible.

The MACAW baseline is reduced to less training data than in its original paper, which the authors acknowledge, but that still makes the "outperforms prior offline SOTA" claim hard to take at face value. The MQL comparison is rough but honestly framed as "roughly situate."

None of this sinks the paper. The central finding is plausible and the experiments are internally coherent. But the missing numbers matter. This paper deserves serious refereeing; I'd send it out with a request for exact context lengths, per-context DT curves, error bars, and code/data release.

Who it's for: people working on in-context RL, meta-RL, or sequence-model architectures for RL. I'd bring it to a reading group and would cite it once the numbers are pinned down.

Referee Report

4 major / 6 minor

Summary. The paper proposes using the Mamba architecture (selective state-space models, S6 layers) as the sequence model for Algorithm Distillation (AD) in continuous-control meta-reinforcement learning. It compares Mamba-based AD against a similarly sized Decision Transformer on four continuous Mujoco/Gym environments (Reacher-Goal, Pusher-Goal, HalfCheetah-Vel, Ant-Dir), reporting that Mamba achieves better asymptotic performance in all settings. The paper also studies how context length affects in-context RL performance and compares AD-Mamba against two meta-RL baselines, MQL and MACAW, concluding that AD-Mamba is competitive with online MQL and outperforms offline MACAW.

Significance. The paper addresses a genuine limitation of the original AD work—the quadratic attention complexity of transformers restricts context length—and provides evidence that a linear-complexity sequence model can perform in-context RL in continuous control. If the comparison is sound, the result is a useful architectural contribution for offline meta-RL, and the context-length study offers practical guidance for scaling AD. The paper includes four environments, an honest discussion of baseline comparability, and a clearly stated limitations section. However, the central architecture comparison currently has unresolved methodological gaps (context-length selection, missing error bars, and asymmetric tuning reporting), so the strength of the claimed superiority is not yet fully established.

major comments (4)

[5.2/5.3, Figure 1] Section 5.2 states that Mamba is compared with 'a similarly sized Decision Transformer trained on the same data and with the same context length,' but Section 5.3 and the Figure 1 caption say that each learning curve plots 'the best performing context length for each model.' The exact context lengths per model and per environment are not reported in the main text or in Appendices A/B, creating a direct tension between the two statements. Since Mamba's advantage is precisely its linear scaling in sequence length, using each model's own best context length can conflate architecture with context-length budget, especially if the Transformer's best length was constrained by compute or memory rather than by task suitability. Please report per-context learning curves for both models, give exact context lengths, and clarify whether the Decision Transformer was evaluated at the same context lengths as Mamba or only at its own best length.
[5.3, Figure 1] Section 5.3 reports that all evaluations aggregate over 3 pre-training random seeds and 10 unseen test tasks, but Figure 1 shows no error bars, standard deviations, or confidence intervals. The text describes the Mamba advantage in Reacher-Goal and HalfCheetah-Vel as a 'small edge,' which is exactly the regime where a 3-seed average could be dominated by noise. Without a numerical mean±std table or a significance test, the central claim of 'better asymptotic performance in all settings considered' is not quantitatively supported. Please add error bars or a table with per-seed results and a measure of variance.
[5.3, Appendix B] No hyperparameter search or sensitivity analysis is reported for the Decision Transformer baseline, despite the statement in Section 5.3 that Mamba 'required less hyper-parameter tuning and being overall more stable during training.' Appendix B lists the chosen DT hyperparameters (heads, layers, feed-forward dimension, embedding dimension) but gives no indication of the search space or tuning effort. This asymmetry leaves open the possibility that the Decision Transformer is undertuned, which would undermine the architecture comparison. Please report the tuning procedure and budget for both models, and ideally include a sensitivity analysis over DT context length and model capacity.
[5.1, Appendix A] Section 5.1 describes the environments only as 'slightly modified versions' of standard continuous-control benchmarks, but the modifications are never specified in the main text or appendices. Table 4 gives the RL algorithm, number of training tasks, and timesteps per task, but does not describe the observation space, action space, reward function, goal sampling distribution, or any other modification relative to the original MuJoCo/Gym environments. This makes the experiments irreproducible and complicates the comparison with MQL and MACAW, whose numbers may correspond to different task distributions. Please provide the full environment specifications or open-source code.

minor comments (6)

[2.1] Section 2.1 contains the grammatical error 'share a some fundamental similarities'; please rephrase.
[Appendix A, Tables 2 and 3] Tables 2 and 3 list PPO-specific hyperparameters ('gae lambda', 'epsilon (clip)') for SAC and DroQ, which do not apply to these off-policy algorithms; this appears to be a copy-paste error and should be corrected.
[Table 4] Table 4 labels the fourth environment 'Ant-goal' while Section 5.1 and Figure 1 call it 'Ant-Dir'; please unify the naming.
[5.3, Figure 2] Figure 2 would be more informative if the legend or caption stated the exact context lengths (e.g., number of episodes or timesteps) for each learning curve, since the context-scaling claim depends on these values.
[Limitations] The Limitations section says 'shape the ICLR abilities'; 'ICLR' should be 'ICRL'.
[3, Equation (3)] Equation (3) uses notation such as 'M_N' and 'M_n' that is not clearly defined; please clarify the indexing (e.g., whether N is the number of tasks and n indexes tasks).

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims rest on an empirical benchmark comparison with an independent supervised objective.

full rationale

The paper's derivation chain is an empirical pipeline rather than a formal derivation: learning histories are generated by PPO/SAC/DroQ, a sequence model is trained with the action-prediction loss in Eq. (6), and the resulting policies are evaluated on unseen tasks. No parameter is fitted and then renamed as a prediction, and no result is forced by a normalization or by an equality that holds by construction. The architecture comparison between Mamba and Decision Transformer is a controlled experiment using the same data and matched parameter counts; the choice of each model's best context length on validation tasks is a standard model-selection step, not a definitional reduction. The skeptical concern that context-length selection may favor Mamba is a possible threat to experimental fairness, but it is not circularity because the reported performance is measured rather than derived from the selection rule. The paper contains no load-bearing self-citations: all cited foundations, including Mamba, Decision Transformer, MACAW, MQL, and prior AD work, are external prior work, and none is invoked to forbid alternative explanations or to declare a choice forced. Consequently, the central claims are self-contained relative to their empirical inputs, and the circularity score is 0.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

This is an empirical study with no theoretical derivation and no new postulated entities. The central claims depend on hyperparameter choices such as context length, downsampling rate, and source algorithm, plus assumptions inherited from Algorithm Distillation and Mamba.

free parameters (6)

Downsampling rate k = Reacher:4, Pusher:10, HalfCheetah:8, Ant:10
Chosen per environment and controls how many training episodes are retained, directly affecting the effective context length and ICRL stability (Section 5.2, Table 5).
Context length per model/environment = not reported; selected as best-performing
Figure 1 compares architectures using the best context length for each model on validation tasks; exact token counts are not listed, and this selection can favor the final comparison.
Source RL algorithm per environment = SAC (Reacher, HalfCheetah), PPO (Pusher), DroQ (Ant)
The distilled algorithm defines the learning histories; algorithm stability and sample efficiency shape the data distribution and in-context behavior (Section 5.2, Table 4).
Number of meta-training tasks = 70, 35, 15, 30 per env
Limits task diversity available for meta-learning; the authors deliberately reduced MACAW's data, which affects the baseline comparison (Table 4).
Training steps per task = Reacher 1e5, Pusher 5e5, HalfCheetah 4e5, Ant 5e5
Determines dataset size and trajectory length, interacting with the context scaling claims (Table 4).
Isotropic Gaussian noise added to tokens = not specified
Added to improve robustness, but the noise scale is not reported, making the exact training procedure underspecified (Section 5.2).

assumptions (6)

domain assumption Mamba/S6 layers are effective at long-range sequence modeling and scale linearly in sequence length.
Relied on as motivation and as explanation for observed superiority (Sections 2.2 and 5.3); based on cited prior work [10].
domain assumption Behavior cloning with an L2 loss on learning histories is sufficient to induce in-context RL, as proposed by Algorithm Distillation.
The entire training objective (Eq. 6) rests on this; the paper does not prove it for continuous control.
domain assumption The source RL algorithms (PPO/SAC/DroQ) produce learning trajectories that contain the improvement structure necessary for distillation.
The dataset is generated by these algorithms; if their learning dynamics are not imitable by the sequence model, ICRL would not emerge (Section 5.2).
ad hoc to paper The four environment modifications preserve the properties needed for meta-RL evaluation and allow comparison to MQL/MACAW results.
The paper says "slightly modified versions" of benchmark environments are used, but the precise modifications and reward/task distributions are not fully specified (Section 5.1).
domain assumption Downsampled, subsequence-based training and the chosen context lengths allow stable in-context learning.
The authors note too-high k causes instability (Section 3); they assume the chosen k and context length are adequate.
domain assumption Training tasks are sampled from a distribution that represents the test tasks; the unseen test tasks come from the same distribution.
Needed for the claim of generalization to new tasks; not statistically verified beyond 3 seeds and 10-30 test tasks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Scaling Algorithm Distillation for Continuous Control with Mamba." pith.science (2026). https://pith.science/paper/APKZ2TDD

@misc{pith2026250613892,
  author       = {Pith},
  title        = {Pith review of: Scaling Algorithm Distillation for Continuous Control with Mamba},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/APKZ2TDD}},
  note         = {Machine review of arXiv:2506.13892}
}

read the original abstract

Algorithm Distillation (AD) was recently proposed as a new approach to perform In-Context Reinforcement Learning (ICRL) by modeling across-episodic training histories autoregressively with a causal transformer model. However, due to practical limitations induced by the attention mechanism, experiments were bottlenecked by the transformer's quadratic complexity and limited to simple discrete environments with short time horizons. In this work, we propose leveraging the recently proposed Selective Structured State Space Sequence (S6) models, which achieved state-of-the-art (SOTA) performance on long-range sequence modeling while scaling linearly in sequence length. Through four complex and continuous Meta Reinforcement Learning environments, we demonstrate the overall superiority of Mamba, a model built with S6 layers, over a transformer model for AD. Additionally, we show that scaling AD to very long contexts can improve ICRL performance and make it competitive even with a SOTA online meta RL baseline.

Figures

Figures reproduced from arXiv: 2506.13892 by the authors.

**Figure 2.** Impact of context size on ICRL performance. Each learning curve is obtained by pre [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 23 canonical work pages

[1]

Vaswani, N

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017

2017
[2]

Oquab, T

M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. HAZIZA, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski. DINOv2: Learning robust visual features without s...

work page 2024
[3]

Kirillov, E

A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015–4026, 2023

2023
[4]

Vuong, S

Q. Vuong, S. Levine, H. R. Walke, K. Pertsch, et al. Open x-embodiment: Robotic learning datasets and RT-x models. In Towards Generalist Robots: Learning Paradigms for Scalable Skill Acquisition @ CoRL2023 , 2023

work page 2023
[5]

Laskin, L

M. Laskin, L. Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. S. Hansen, A. Filos, E. Brooks, maxime gazeau, H. Sahni, S. Singh, and V . Mnih. In-context reinforcement learning with algorithm distillation. In The Eleventh International Conference on Learning Representations, 2023

work page 2023
[6]

Y . Tay, M. Dehghani, S. Abnar, Y . Shen, D. Bahri, P. Pham, J. Rao, L. Yang, S. Ruder, and D. Metzler. Long range arena: A benchmark for efficient transformers. In International Conference on Learning Representations, 2020

work page 2020
[7]

N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024

2024
[8]

L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417, 2024

arXiv 2024

Show all 37 references

[9]

A. Gu, K. Goel, and C. Re. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations , 2022

2022
[10]

Gu and T

A. Gu and T. Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

2023 arXiv
[11]

Grazzi, J

R. Grazzi, J. Siems, S. Schrodi, T. Brox, and F. Hutter. Is mamba capable of in-context learning? arXiv preprint arXiv:2402.03170, 2024

2024 arXiv
[12]

T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems , 35: 16344–16359, 2022

2022
[13]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

1901
[14]

A. Gu, T. Dao, S. Ermon, A. Rudra, and C. Ré. Hippo: Recurrent memory with optimal polynomial projections. Advances in neural information processing systems , 33:1474–1487, 2020

2020
[15]

Y . Duan, J. Schulman, X. Chen, P. L. Bartlett, I. Sutskever, and P. Abbeel. Rl 2: Fast reinforce- ment learning via slow reinforcement learning. arXiv preprint arXiv:1611.02779, 2016. 10

2016 arXiv
[16]

J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Ku- maran, and M. Botvinick. Learning to reinforcement learn. arXiv preprint arXiv:1611.05763, 2016

2016 arXiv
[17]

Hochreiter and J

S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation , 9(8): 1735–1780, 1997

1997
[18]

C. Lu, Y . Schroecker, A. Gu, E. Parisotto, J. Foerster, S. Singh, and F. Behbahani. Structured state space models for in-context reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

2024
[19]

J. T. Smith, A. Warrington, and S. Linderman. Simplified state space layers for sequence modeling. In The Eleventh International Conference on Learning Representations , 2023

2023
[20]

Rakelly, A

K. Rakelly, A. Zhou, C. Finn, S. Levine, and D. Quillen. Efficient off-policy meta-reinforcement learning via probabilistic context variables. In International conference on machine learning , pages 5331–5340. PMLR, 2019

2019
[21]

D. P. Kingma and M. Welling. Auto-encoding variational bayes. In2nd International Conference Learning Representation (ICLR), 2014

2014
[22]

C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126–1135. PMLR, 2017

2017
[23]

Mitchell, R

E. Mitchell, R. Rafailov, X. B. Peng, S. Levine, and C. Finn. Offline meta-reinforcement learning with advantage weighting. In International Conference on Machine Learning , pages 7780–7791. PMLR, 2021

2021
[24]

X. B. Peng, A. Kumar, G. Zhang, and S. Levine. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning. arXiv preprint arXiv:1910.00177, 2019

1910 arXiv
[25]

Fakoor, P

R. Fakoor, P. Chaudhari, S. Soatto, and A. J. Smola. Meta-q-learning. In International Conference on Learning Representations, 2019

2019
[26]

K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y . Bengio. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processi...

2014
[27]

L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems , 34:15084–15097, 2021

2021
[28]

Janner, Q

M. Janner, Q. Li, and S. Levine. Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems , 34:1273–1286, 2021

2021
[29]

M. Xu, Y . Shen, S. Zhang, Y . Lu, D. Zhao, J. Tenenbaum, and C. Gan. Prompting decision transformer for few-shot policy generalization. In international conference on machine learning, pages 24631–24645. PMLR, 2022

2022
[30]

Todorov, T

E. Todorov, T. Erez, and Y . Tassa. Mujoco: A physics engine for model-based control. In2012 IEEE/RSJ international conference on intelligent robots and systems , pages 5026–5033. IEEE, 2012

2012
[31]

Brockman, V

G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016

2016 arXiv
[32]

Schulman, F

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. In arXiv preprint arXiv:1707.06347, 2017. 11

2017 arXiv
[33]

Haarnoja, A

T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. PMLR, 2018

2018
[34]

Hiraoka, T

T. Hiraoka, T. Imagawa, T. Hashimoto, T. Onishi, and Y . Tsuruoka. Dropout q-functions for dou- bly efficient reinforcement learning. In International Conference on Learning Representations , 2022

2022
[35]

Agarwal, A

R. Agarwal, A. Singh, L. M. Zhang, B. Bohnet, S. Chan, A. Anand, Z. Abbas, A. Nova, J. D. Co-Reyes, E. Chu, et al. Many-shot in-context learning. arXiv preprint arXiv:2404.11018 , 2024

2024 arXiv
[36]

M. F. Chen, N. Roberts, K. Bhatia, J. WANG, C. Zhang, F. Sala, and C. Re. Skill-it! a data- driven skills framework for understanding and training language models. In Thirty-seventh Conference on Neural Information Processing Systems , 2023

2023
[37]

C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. Robotics: Science and Systems 2023 (RSS) , 2023. 12 A Dataset Generation Table 1: PPO hyper-parameters Parameter Value num. epochs 10 ste...

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Vaswani, N

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017

2017

[2] [2]

Oquab, T

M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. HAZIZA, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski. DINOv2: Learning robust visual features without s...

work page 2024

[3] [3]

Kirillov, E

A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015–4026, 2023

2023

[4] [4]

Vuong, S

Q. Vuong, S. Levine, H. R. Walke, K. Pertsch, et al. Open x-embodiment: Robotic learning datasets and RT-x models. In Towards Generalist Robots: Learning Paradigms for Scalable Skill Acquisition @ CoRL2023 , 2023

work page 2023

[5] [5]

Laskin, L

M. Laskin, L. Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. S. Hansen, A. Filos, E. Brooks, maxime gazeau, H. Sahni, S. Singh, and V . Mnih. In-context reinforcement learning with algorithm distillation. In The Eleventh International Conference on Learning Representations, 2023

work page 2023

[6] [6]

Y . Tay, M. Dehghani, S. Abnar, Y . Shen, D. Bahri, P. Pham, J. Rao, L. Yang, S. Ruder, and D. Metzler. Long range arena: A benchmark for efficient transformers. In International Conference on Learning Representations, 2020

work page 2020

[7] [7]

N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024

2024

[8] [8]

L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417, 2024

arXiv 2024