Pith. sign in

REVIEW 4 major objections 2 minor 62 references

In-Context Reinforcement Learning via Communicative World Models

T0 review · 4 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CORAL claims that in-context RL can be solved by having a pre-trained world model communicate task knowledge to a separate control agent through messages shaped by their causal influence on the next action.

desk verdict Promising two-agent communication idea, but the arXiv full text is corrupted, so the causal-influence claim can't be checked. read the letter →

arxiv 2508.06659 v2 pith:Z4OXAACT submitted 2025-08-08 cs.LG cs.AI

Fernando Martinez-Lopez , Tao Li , Yingdong Lu , Juntao Chen This is my paper
classification cs.LGcs.AI
keywords in-contextreinforcementlearningemergentcommunicationworldmodelCausalInfluenceLosszero-shotadaptationsampleefficiencytransferablerepresentationtwo-agent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an RL agent can learn new tasks in context—without updating its parameters—if task knowledge is delivered to it as messages from a pre-trained world model. The proposed framework, CORAL, splits the problem in two: an Information Agent learns to model a distribution of tasks and compress that understanding into concise messages, while a separate Control Agent learns to act from those messages. The communication protocol is shaped by a Causal Influence Loss, which rewards messages that change the control agent's next action. The reported consequence is that a fresh control agent becomes sample-efficient and can adapt zero-shot across online and offline environments. A sympathetic reader would care because it turns in-context RL from a property of one giant policy into a reusable communication interface between a fixed world model and many small policies.

What carries the argument

CORAL's central mechanism is a two-agent communication setup: the Information Agent (IA) is pre-trained as a world model, compressing its latent understanding of a task distribution into concise messages; the Control Agent (CA) is a policy that conditions its actions on those messages. The Causal Influence Loss is the shaping signal: it measures the effect that the message has on the CA's next action, so the emergent protocol is optimized to change behavior rather than to reconstruct observations.

What would settle it

Train the Information Agent with the Causal Influence Loss removed or replaced by a pure world-model reconstruction objective, then measure whether a newly trained Control Agent still achieves the reported sample-efficiency and zero-shot gains. If the gains persist, the causal action-influence signal is not doing the work the paper assigns to it. A second check: after pre-training, feed two tasks with different optimal actions the same state history and verify that the IA emits different messages; identical messages for behaviorally distinct tasks would falsify the claim that the message carri

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that in-context RL can be reframed as emergent communication: a pre-trained Information Agent acts as a world model that distills its understanding into concise messages, and a separate Control Agent learns to act by reading those messages. The Causal Influence Loss—which scores a message by how much it changes the Control Agent's next action—is the mechanism claimed to make the protocol transferable. Across diverse online and offline environments, CORAL reports that the Control Agent gains sample efficiency and can perform zero-shot adaptation with the fixed Information Agent.

Load-bearing premise

The load-bearing premise is that a training signal based purely on a message's causal effect on the next action is enough to make the message carry the task knowledge a new control agent needs.

Editorial extensions

If this is right

  • If correct, a world model can be pre-trained once on a task distribution and then reused as a fixed message source for many downstream control agents.
  • The control agent can learn new tasks from the communicative context alone, without updating the Information Agent—the zero-shot adaptation claim.
  • Sample-efficiency gains should appear in both online RL, where the agent interacts with the environment, and offline RL, where it learns from a fixed dataset.
  • The action-influence objective implies that the protocol is grounded in behavior rather than observation reconstruction, so messages should stay concise and task-directed.
  • The separation of world modeling from control suggests that the expensive representation learning can be amortized across many different policies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to test whether the same pre-trained Information Agent can serve multiple control agents with different architectures or action spaces; if the communicative context is truly transferable, the Control Agent should not need any IA-specific initialization.
  • The Causal Influence Loss resembles an empowerment-style objective, and one could test whether messages optimized purely to influence the next action also carry enough information for long-horizon credit assignment or whether they drop features that matter only several steps later.
  • If action influence is the only training signal, then two tasks that demand identical immediate actions but diverge in future consequences might be conflated; probing with such task pairs would reveal the boundary of the protocol's expressiveness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 2 minor

Summary. The paper proposes CORAL (Communicative Representation for Adaptive RL), a two-agent emergent-communication framework for in-context reinforcement learning. An Information Agent is pre-trained as a world model and emits messages that are shaped by a Causal Influence Loss; a Control Agent then learns to act from those messages. The abstract claims significant gains in sample efficiency and successful zero-shot adaptation in both online and offline environments. However, the submitted full text is almost entirely unreadable replacement characters; no equations, algorithms, experimental details, baselines, or ablations are legible. The technical content and the evidence for the central claims therefore cannot be assessed.

Significance. If substantiated, the paper would present a conceptually interesting direction: using a pre-trained world model as a fixed communicative context for a control agent could be a general mechanism for in-context RL. The explicit separation of world modeling from control and the use of a causal influence objective to shape communication are plausible design choices. However, the submission contains no verifiable evidence. There are no equations, no experimental results, no baselines, and no code or proofs to inspect. The potential significance of the idea is therefore untestable in the current form.

major comments (4)
  1. [Full text, all pages (pp. 1–28)] The full text of the manuscript consists of unreadable replacement characters. The definitions of the Information Agent, the Control Agent, the Causal Influence Loss, the training objectives, the architectures, and the experimental setup are absent. This is the core technical content of the paper, and its absence prevents any substantive evaluation of the claimed contribution.
  2. [Abstract] The claim of "significant gains in sample efficiency" is unsupported by any quantitative comparison. The submitted text provides no task suite, no baseline algorithms, no performance curves, and no error bars. A central performance claim of this strength requires at least one legible experimental table or figure.
  3. [Abstract / zero-shot adaptation claim] The paper claims that the Control Agent can perform zero-shot adaptation with the help of the pre-trained Information Agent. The deployment protocol for this claim is not described: which parameters are frozen at test time, which tasks are held out, and whether the Control Agent receives any gradient updates. Without this information, the central claim is untestable.
  4. [Abstract / Causal Influence Loss] The Causal Influence Loss is described as measuring the effect of the message on the next action. Because the Control Agent is also trained to act from those messages, there is a co-adaptation risk: the message may be shaped to the action statistics of the training-time Control Agent rather than to task-general structure. Whether the authors train the two agents jointly, use a fixed policy when computing the causal effect, or include a transfer experiment that rules out this collapse cannot be checked because the relevant equation and experiments are not legible in the submitted text.
minor comments (2)
  1. [Abstract] The acronym ICRL is used without expansion in the abstract; "in-context RL" should be spelled out at first use, and IA/CA should be defined consistently.
  2. [Full text] The manuscript does not contain a limitations or discussion section. The authors should discuss when message-based context transfer could fail, especially when the Causal Influence Loss may not encode enough task information.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity established; abstract-level co-adaptation concern is not definitional circularity.

full rationale

The provided full text is unreadable (replacement characters on every page), so no equations, derivations, or experimental details can be inspected. The only substantive claims are in the abstract: an Information Agent is pre-trained to produce messages, a Causal Influence Loss measures the effect of a message on the next action, and a Control Agent is trained to act from those messages. This describes a coupled training loop, but coupling is not circularity: optimizing messages to influence actions and training a policy to use messages does not make the zero-shot transfer result true by construction. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no cited result is load-bearing by self-citation. Without an equation showing that the predicted quantity is identical to the training objective, or a fitted value that is then called a prediction, the hard-rule requirement to exhibit a specific reduction cannot be met. The skeptic's concern that action-influence may be too weak a training signal is a hypothesis about generalization, not a demonstration of circularity, and would belong in a correctness risk assessment rather than a circularity score. Therefore the honest finding is no significant circularity, score 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 2 invented entities

The abstract provides no hyperparameters or fitted constants. The two agent types are introduced as design components. The key unstated premises are that action-influence is the right training signal and that pre-trained world-model messages transfer to new tasks.

assumptions (2)
  • domain assumption A message that causally influences the control agent's next action is a sufficient training signal for a transferable communication protocol.
    This is the premise behind the Causal Influence Loss; the abstract states the loss 'measures the effect that the message has on the next action' but does not justify why this yields useful messages.
  • domain assumption A world model pre-trained on a diverse distribution of tasks produces messages that enable zero-shot adaptation to new tasks.
    The abstract asserts this transfer, but provides no theoretical justification.
invented entities (2)
  • Information Agent (IA)
    purpose: Pre-trained world model that distills understanding into concise messages via emergent communication.
    A new agent component introduced by the framework; no independent evidence in the abstract.
  • Control Agent (CA)
    purpose: Learns to solve tasks by interpreting communicative context from IA during deployment.
    A new agent component; no independent evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of In-Context Reinforcement Learning via Communicative World Models." pith.science (2026). https://pith.science/paper/Z4OXAACT

@misc{pith2026250806659,
  author       = {Pith},
  title        = {Pith review of: In-Context Reinforcement Learning via Communicative World Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z4OXAACT}},
  note         = {Machine review of arXiv:2508.06659}
}
read the original abstract

Reinforcement learning (RL) agents often struggle to generalize to new tasks and contexts without updating their parameters, mainly because their learned representations and policies are overfit to the specifics of their training environments. To boost agents' in-context RL (ICRL) ability, this work formulates ICRL as a two-agent emergent communication problem and introduces CORAL (Communicative Representation for Adaptive RL), a framework that learns a transferable communicative context by functionally separating latent representation learning from control. In CORAL, an Information Agent (IA) is pre-trained as a world model on a diverse distribution of tasks. Its objective is not direct return maximization, but world modeling and distilling its understanding into concise messages. The emergent communication protocol is shaped by a novel Causal Influence Loss, which measures the effect that the message has on the next action. During deployment, the previously trained IA serves as a fixed contextualizer for a new Control Agent (CA), which learns to solve tasks by interpreting the provided communicative context. Our experiments demonstrate that this approach enables the CA to achieve significant gains in sample efficiency and successfully perform zero-shot adaptation with the help of pre-trained IA in diverse online and offline environments, validating the efficacy of learning a transferable communicative representation.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 52 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bauer, J.; Baumli, K.; Behbahani, F.; Bhoopchand, A.; Bradley-Schmieg, N.; Chang, M.; Clay, N.; Collister, A.; Dasagi, V.; Gonzalez, L.; Gregor, K.; Hughes, E.; Kashem, S.; Loks-Thompson, M.; Openshaw, H.; Parker-Holder, J.; Pathak, S.; Perez-Nieves, N.; Rakicevic, N.; Rockt\" a schel, T.; Schroecker, Y.; Singh, S.; Sygnowski, J.; Tuyls, K.; York, S.; Zac...

    work page 2023

  4. [4]

    J.; Leary, C.; Maclaurin, D.; Necula, G.; Paszke, A.; Vander P las, J.; Wanderman- M ilne, S.; and Zhang, Q

    Bradbury, J.; Frostig, R.; Hawkins, P.; Johnson, M. J.; Leary, C.; Maclaurin, D.; Necula, G.; Paszke, A.; Vander P las, J.; Wanderman- M ilne, S.; and Zhang, Q. 2018. JAX : composable transformations of P ython+ N um P y programs. http://github.com/jax-ml/jax

    work page 2018

  5. [5]

    Chen, J.; Ganguly, B.; Xu, Y.; Mei, Y.; Lan, T.; and Aggarwal, V. 2024. Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directions. Transactions on Machine Learning Research. Survey Certification

    work page 2024

  6. [6]

    Chen, L.; Lu, K.; Rajeswaran, A.; Lee, K.; Grover, A.; Laskin, M.; Abbeel, P.; Srinivas, A.; and Mordatch, I. 2021. Decision Transformer: Reinforcement Learning via Sequence Modeling. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems, volume 34, 15084--15097

    work page 2021

  7. [7]

    S.; and Terry, J

    Chevalier-Boisvert, M.; Dai, B.; Towers, M.; Perez-Vicente, R.; Willems, L.; Lahlou, S.; Pal, S.; Castro, P. S.; and Terry, J. 2023. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. Advances in Neural Information Processing Systems, 36: 73383--73394

    work page 2023

  8. [8]

    Chung, J.; Gulcehre, C.; Cho, K.; and Bengio, Y. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555

    arXiv 2014

Show all 62 references
  1. [9]

    Cobbe, K.; Hesse, C.; Hilton, J.; and Schulman, J. 2020. Leveraging procedural generation to benchmark reinforcement learning. In International conference on machine learning, 2048--2056. PMLR

    2020

  2. [10]

    P.; and Sobel, J

    Crawford, V. P.; and Sobel, J. 1982. Strategic Information Transmission. Econometrica, 50(6): 1431

    1982

  3. [11]

    Dong, Q.; Li, L.; Dai, D.; Zheng, C.; Ma, J.; Li, R.; Xia, H.; Xu, J.; Wu, Z.; Chang, B.; Sun, X.; Li, L.; and Sui, Z. 2024. A Survey on In-context Learning. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natura...

    2024

  4. [12]

    L.; Sutskever, I.; and Abbeel, P

    Duan, Y.; Schulman, J.; Chen, X.; Bartlett, P. L.; Sutskever, I.; and Abbeel, P. 2016. RL \ 2\ : Fast Reinforcement Learning via Slow Reinforcement Learning. arXiv

    2016

  5. [13]

    Espeholt, L.; Soyer, H.; Munos, R.; Simonyan, K.; Mnih, V.; Ward, T.; Doron, Y.; Firoiu, V.; Harley, T.; Dunning, I.; et al. 2018. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures. In International conference on machine learning, 1407--...

    2018

  6. [14]

    Eysenbach, B.; Khazatsky, A.; Levine, S.; and Salakhutdinov, R. 2022. Joint Model-Policy Optimization of a Lower Bound for Model-Based RL . In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems

    2022

  7. [15]

    Fallah, A.; Georgiev, K.; Mokhtari, A.; and Ozdaglar, A. 2021. On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems...

    2021

  8. [16]

    Finn, C.; Abbeel, P.; and Levine, S. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In Precup, D.; and Teh, Y. W., eds., Proceedings of the 34th International Conference on Machine Learning, volume 70, 1126--1135

    2017

  9. [17]

    Furuta, H.; Matsuo, Y.; and Gu, S. S. 2021. Generalized Decision Transformer for Offline Hindsight Information Matching . In International Conference on Learning Representations

    2021

  10. [18]

    Grigsby, J.; Fan, L.; and Zhu, Y. 2024. AMAGO : Scalable In-Context Reinforcement Learning for Adaptive Agents. In The Twelfth International Conference on Learning Representations

    2024

  11. [19]

    Ha, D.; and Schmidhuber, J. 2018. Recurrent World Models Facilitate Policy Evolution. In Bengio, S.; Wallach, H.; Larochelle, H.; Grauman, K.; Cesa-Bianchi, N.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 31

    2018

  12. [20]

    Hafner, D.; Lillicrap, T.; Ba, J.; and Norouzi, M. 2020. Dream to Control: Learning Behaviors by Latent Imagination. In International Conference on Learning Representations

    2020

  13. [21]

    Hafner, D.; Lillicrap, T.; Fischer, I.; Villegas, R.; Ha, D.; Lee, H.; and Davidson, J. 2019. Learning Latent Dynamics for Planning from Pixels. In Chaudhuri, K.; and Salakhutdinov, R., eds., Proceedings of the 36th International Conference on Machine Learning, volume 97 of Pr...

    2019

  14. [22]

    Hafner, D.; Pasukonis, J.; Ba, J.; and Lillicrap, T. 2025. Mastering diverse control tasks through world models. Nature, 640(8059): 647--653

    2025

  15. [23]

    Hammar, K.; Li, T.; Stadler, R.; Zhu, Q.; and Hammar, K. 2025. Adaptive Security Response Strategies Through Conjectural Online Learning. IEEE Transactions on Information Forensics and Security , 20: 4055--4070

    2025

  16. [24]

    v.; Modayil, J.; and Silver, D

    Hessel, M.; Hasselt, H. v.; Modayil, J.; and Silver, D. 2019. On Inductive Biases in Deep Reinforcement Learning. arXiv

    2019

  17. [25]

    Huang, S.; Hu, J.; Yang, Z.; Yang, L.; Luo, T.; Chen, H.; Sun, L.; and Yang, B. 2024. Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  18. [26]

    Ioffe, S.; and Szegedy, C. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, 448--456. pmlr

    2015

  19. [27]

    P.; Littman, M

    Kaelbling, L. P.; Littman, M. L.; and Cassandra, A. R. 1998. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2): 99--134

    1998

  20. [28]

    J.; Zhang, C.; and Slivkins, A

    Krishnamurthy, A.; Harris, K.; Foster, D. J.; Zhang, C.; and Slivkins, A. 2024. Can large language models explore in-context? In ICML 2024 Workshop on In-Context Learning

    2024

  21. [29]

    S.; Filos, A.; Brooks, E.; maxime gazeau; Sahni, H.; Singh, S.; and Mnih, V

    Laskin, M.; Wang, L.; Oh, J.; Parisotto, E.; Spencer, S.; Steigerwald, R.; Strouse, D.; Hansen, S. S.; Filos, A.; Brooks, E.; maxime gazeau; Sahni, H.; Singh, S.; and Mnih, V. 2023. In-context Reinforcement Learning with Algorithm Distillation. In The Eleventh International Co...

    2023

  22. [30]

    Lee, J.; Xie, A.; Pacchiano, A.; Chandak, Y.; Finn, C.; Nachum, O.; and Brunskill, E. 2023. Supervised Pretraining Can Learn In-Context Reinforcement Learning. In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  23. [31]

    Li, T.; Guevara, J.; Xie, X.; and Zhu, Q. 2025 a . Self-Confirming Transformer for Belief-Conditioned Adaptation in Offline Multi-Agent Reinforcement Learning. In Proceedings of the Seventh Workshop on Adaptive and Learning Agents, the Twenty Fourth International Conference on...

    2025

  24. [32]

    Li, T.; Hammar, K.; Stadler, R.; and Zhu, Q. 2024 a . Conjectural Online Learning with First-order Beliefs in Asymmetric Information Stochastic Games. In 2024 IEEE 63rd Conference on Decision and Control ( CDC ) , IEEE CDC , 6780--6785

    2024

  25. [33]

    Li, T.; Lei, H.; Yin, M.; and Hu, Y. 2025 b . Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation. In Reinforcement Learning Conference 2025, Inductive Biases in Reinforcement Learning Workshop. [Online] Available at ht...

    2025 arXiv

  26. [34]

    Li, T.; Lei, H.; and Zhu, Q. 2023. Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), 7205--7211

    2023

  27. [35]

    Li, T.; Li, H.; Pan, Y.; Xu, T.; Zheng, Z.; and Zhu, Q. 2024 b . Meta stackelberg game: Robust federated learning against adaptive and mixed poisoning attacks. arXiv preprint arXiv:2410.17431. [Online] Available at https://arxiv.org/pdf/2410.17431

    2024 arXiv

  28. [36]

    Li, T.; Peng, G.; Zhu, Q.; and Baar, T. 2022. The Confluence of Networks, Games, and Learning a Game-Theoretic Framework for Multiagent Decision Making Over Networks. IEEE Control Systems , 42(4): 35--67

    2022

  29. [37]

    Li, T.; and Zhu, Q. 2023. On the Price of Transparency: A Comparison Between Overt Persuasion and Covert Signaling. In 2023 62nd IEEE Conference on Decision and Control (CDC), 4267--4272

    2023

  30. [38]

    Li, T.; and Zhu, Q. 2024. Symbiotic Game and Foundation Models for Cyber Deception Operations in Strategic Cyber Warfare. arXiv preprint arXiv:2403.10570. [Online] Available at https://arxiv.org/pdf/2403.10570

    2024 arXiv

  31. [39]

    Lowe, R.; Foerster, J.; Boureau, Y.-L.; Pineau, J.; and Dauphin, Y. 2019. On the Pitfalls of Measuring Emergent Communication. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 693--701

    2019

  32. [40]

    Lu, C.; Kuba, J.; Letcher, A.; Metz, L.; Schroeder de Witt, C.; and Foerster, J. 2022. Discovered policy optimisation. Advances in Neural Information Processing Systems, 35: 16455--16468

    2022

  33. [41]

    Micheli, V.; Alonso, E.; and Fleuret, F. 2023. Transformers are Sample-Efficient World Models. In The Eleventh International Conference on Learning Representations

    2023

  34. [42]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-Leve...

    2015

  35. [43]

    Moeini, A.; Wang, J.; Beck, J.; Blaser, E.; Whiteson, S.; Chandra, R.; and Zhang, S. 2025. A survey of in-context reinforcement learning. arXiv preprint arXiv:2502.07978

    2025 arXiv

  36. [44]

    Pan, Y.; Li, T.; and Zhu, Q. 2025. Model-Agnostic Meta-Policy Optimization via Zeroth-Order Estimation: A Linear Quadratic Regulator Perspective. arXiv preprint arXiv:2503.00385. [Online] Available at https://arxiv.org/pdf/2503.00385

    2025 arXiv

  37. [45]

    T.; Lu, C.; Castro, P

    Pignatelli, E.; Liesen, J.; Lange, R. T.; Lu, C.; Castro, P. S.; and Toni, L. 2024. NAVIX: Scaling MiniGrid Environments with JAX. arXiv preprint arXiv:2407.19396

    2024 arXiv

  38. [46]

    C.; Hambro, E.; Kirk, R.; Henaff, M.; and Raileanu, R

    Raparthy, S. C.; Hambro, E.; Kirk, R.; Henaff, M.; and Raileanu, R. 2024. Generalization to New Sequential Decision Making Tasks with In-Context Learning. In Proceedings of the 41st International Conference on Machine Learning, ICML , 42138--42158

    2024

  39. [47]

    Reed, S. 2022. A Generalist Agent. Transactions on Machine Learning Research

    2022

  40. [48]

    Robine, J.; H \"o ftmann, M.; Uelwer, T.; and Harmeling, S. 2023. Transformer-based World Models Are Happy With 100k Interactions. In The Eleventh International Conference on Learning Representations

    2023

  41. [49]

    Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T.; Lillicrap, T.; and Silver, D. 2020. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 588(7839): 604--609

    2020

  42. [50]

    Schrittwieser, J.; Hubert, T.; Mandhane, A.; Barekatain, M.; Antonoglou, I.; and Silver, D. 2021. Online and Offline Reinforcement Learning by Planning with a Learned Model. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural I...

    2021

  43. [51]

    Schulman, J.; Moritz, P.; Levine, S.; Jordan, M.; and Abbeel, P. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438

    2015 arXiv

  44. [52]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  45. [53]

    D.; Bachman, P.; and Courville, A

    Schwarzer, M.; Rajkumar, N.; Noukhovitch, M.; Anand, A.; Charlin, L.; Hjelm, R. D.; Bachman, P.; and Courville, A. C. 2021. Pretraining representations for data-efficient reinforcement learning. Advances in Neural Information Processing Systems, 34: 12686--12699

    2021

  46. [54]

    Stooke, A.; Lee, K.; Abbeel, P.; and Laskin, M. 2021. Decoupling Representation Learning from Reinforcement Learning. In Meila, M.; and Zhang, T., eds., Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research...

    2021

  47. [55]

    Sukhbaatar, S.; Szlam, A.; and Fergus, R. 2016. Learning multiagent communication with backpropagation. In Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc. ISBN 9781510838819

    2016

  48. [56]

    Toledo, E.; and Prorok, A. 2024. CoDreamer: Communication-Based Decentralised World Models. In Coordination and Cooperation for Multi-Agent Reinforcement Learning Methods Workshop

    2024

  49. [57]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  50. [58]

    H.; Daneshmand, H.; and Zhang, S

    Wang, J.; Blaser, E. H.; Daneshmand, H.; and Zhang, S. 2024. Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning. In ICML 2024 Workshop on In-Context Learning

    2024

  51. [59]

    X.; Kurth-Nelson, Z.; Tirumala, D.; Soyer, H.; Leibo, J

    Wang, J. X.; Kurth-Nelson, Z.; Tirumala, D.; Soyer, H.; Leibo, J. Z.; Munos, R.; Blundell, C.; Kumaran, D.; and Botvinick, M. 2016. Learning to reinforcement learn. arXiv

    2016

  52. [60]

    Xu, M.; Shen, Y.; Zhang, S.; Lu, Y.; Zhao, D.; Tenenbaum, J.; and Gan, C. 2022. Prompting Decision Transformer for Few-Shot Policy Generalization. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvari, C.; Niu, G.; and Sabato, S., eds., Proceedings of the 39th International Conf...

    2022

  53. [61]

    Yang, Y.-T.; Li, T.; and Zhu, Q. 2025. Transparent Tagging for Strategic Social Nudges on User-Generated Misinformation. IEEE Transactions on Network Science and Engineering, 1--14

    2025

  54. [62]

    Zhu, C.; Dastani, M.; and Wang, S. 2024. A survey of multi-agent deep reinforcement learning with communication. Autonomous Agents and Multi-Agent Systems, 38(1): 4

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.