Artificial Intelligence · arXiv:2611.10501 v1 · submitted 7 Nov 2026 · 47 references
Sampling Is Not Curiosity: Why LLM Agents Should Investigate
The paper says
“LLM agents are increasingly used for open-ended tasks, very common in research scenarios where they are required to explore, investigate, or act curiously.”
more of the abstract
At inference time, an LLM is a fixed map from context to next-token distribution, the pretrained weights determine the function, and only its conditioning changes as context accumulates. Mechanisms like temperature, best-of-N, tree search, or RAG vary how the distribution is sampled or what it is conditioned on, but do not update the agent's model of the environment. We argue that current LLM agents do not explore curiously; they explore stochastically. From how curiosity is understood in the RL literature, it requires a predictive structure over how the environment responds to the agent's actions, which improves online from current interaction, drives action selection through its predicted improvement, and persists beyond the context window. Additionally, in its open-ended form, the agent should intrinsically originate what to investigate. Thus, we propose a framework defining curiosity in LLM agents, reserving the term for systems that satisfy it. In doing so, we identify the research directions its implementation opens up.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 10 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.
A clear conceptual framework for curiosity in LLM agents, but the central universal claim is overbroad and should be qualified. This line is wrong
1 major objection · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 47 references say.
The paper says, at Abstract, §1, and §4
The review did not quote this passage.
Pith says: Objection 1 of 1 · major
The paper states without qualification that 'current LLM agents do not explore curiously' (Abstract, b30, b34). Yet in §4, WorldLLM [20] is presented as 'an example of the four task-driven commitments' (sentence b183), and the framework defines systems satisfying C1–C4 as curious agents in the task-driven form (§3, diamond symbol).
more of objection 1
Therefore, by the paper's own criteria, WorldLLM is a current curious LLM agent. The universal claim is internally inconsistent. The authors must either add an explicit qualifier (e.g., 'most current LLM agents' or 'the dominant paradigm of current LLM agents') or explain why WorldLLM does not count as a current agent (e.g., because it is a research prototype rather than a deployed system, or because its curiosity is only partial). Without such a qualification, the foundational premise of the paper is self-contradictory.
5 minor objections
The paper says, at §5, 'RL-finetuning' paragraph
Pith says: minor objection 1 of 5
The argument that RL fine-tuning does not qualify as curiosity because it operates across episodes rather than within an episode is sound, but the paper should acknowledge that streaming online RL and continual learning approaches—mentioned later—blur this line. The current text treats them as separate directions, but a brief note about the continuity would help avoid confusion.
The paper says, at §2.1, Eq. (1)
Pith says: minor objection 2 of 5
The notation S = ⟨p_θ, M, P, π⟩ is clear, but the components p_θ and M are both described as predictive structures. The paper would benefit from a more explicit distinction: p_θ is the static LLM prior, while M is the environment-specific predictive component that gets updated. A sentence clarifying that p_θ is not part of M would prevent potential misinterpretation.
show 3 more
The paper says, at §4, 'The wrong prediction target'
Pith says: minor objection 3 of 5
The critique of CDE [6] is valid but could be strengthened by noting that CDE's perplexity-based bonus, when computed over environmental observations (not model outputs), could actually satisfy C1. The current phrasing might be read as dismissing all uses of perplexity, which is not the authors' intent.
The paper says, at §6.1, Behavioral tests
Pith says: minor objection 4 of 5
The proposed behavioral tests are a nice addition, but test [C1] ('Before acting, the agent predicts the outcome') may be too strong for systems that predict latent states rather than explicit outcomes. Consider adding a note that the prediction can be over any representation of environment dynamics.
The paper says, at References
Pith says: minor objection 5 of 5
Reference [38] is a URL without a year in the text; it should include the access year as given. Also, some arXiv references lack the version? (e.g., [8] says 'arXiv:2606.19476'—please verify the correct identifier).
What would settle it
Verify from the WorldLLM paper [20] whether its action-selection policy π selects actions based on expected improvement of the predictive component (C3), not just on current prediction error. If it does, then the paper's central claim is contradicted by its own evidence and must be qualified. If it does not, then the paper's characterization of WorldLLM as satisfying C1–C4 is incorrect and the contradiction is resolved.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 10 Nov 2026
- Reader holds up
- Skeptic disagrees
- Editor wrote the verdict above
- Referee major revision: 1 major objection, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
This paper does something useful: it pulls the term 'curiosity' out of the LLM agent slop and anchors it in the RL tradition. The five commitments (C1–C5) are well-motivated, the diagnosis of current work is fair (the distinction between sampling and learning is the right target), and the authors are honest about the gaps (ICL, memory, within-episode measurement). It’s a tidy conceptual contribution that could give the field a more precise vocabulary. That alone makes it worth reading.
The main soft spot is the universal claim. The paper states repeatedly that ‘current LLM agents do not explore curiously’ (sentence b30). But it then presents WorldLLM as a current system that satisfies C1–C4 by design. That is a contradiction on the paper’s own terms. The claim needs a qualifier: ‘most current LLM agents without explicit curiosity mechanisms’ or ‘typical LLM agents that only vary sampling and context.’ The authors acknowledge that WorldLLM is an exception in the final section, but the abstract and introduction do not reflect that at all. This is a straightforward revision problem, not a fatal flaw, but it undermines the opening rhetoric.
The paper also leans heavily on the assumption that in-context learning cannot satisfy C3 and C4. That is plausible but not proven, and the authors themselves leave it open (Section 5). That is a legitimate uncertainty, not a flaw—they state it. But readers who think ICL can implement the full curiosity loop may find the central diagnosis incomplete.
The rest of the framework is solid, the literature connection is authentic, and the open problems are real. The paper knows what it is (a position paper, not an empirical result). It could benefit from a slightly less absolute framing, but the core argument holds up under qualification.
For whom? Anyone working on LLM agents, especially those claiming curiosity-driven exploration. It will annoy people who use the term loosely, and that’s good. It deserves a serious referee: send to peer review. I would cite it when writing about exploration in agents. I’d also bring it to reading group for the framework alone. The authors are serious thinkers—this is conceptually clean.
For revision: please add ‘most’ or ‘typical’ before ‘current LLM agents’ in the claim. Also, separate the description of WorldLLM more clearly from the diagnostic category of stochastic explorers. That fixes the tension without weakening the paper’s value.
The strongest objection
Sentence b30 states: 'In this paper, we argue that current LLM agents do not explore curiously, they explore stochastically.' This is an unqualified claim about all current LLM agents. However, in §4, the paper presents WorldLLM [20] as 'an example of the four task-driven commitments' (C1–C4), with action selection 'driven entirely by intrinsic signals from these hypotheses' (sentence b183). The paper's own framework defines a curious agent (task-driven form) as one satisfying C1–C4 (sentence b133). Therefore, by the paper's own criteria, WorldLLM is a current curious LLM agent. The central claim is thus either false or requires an explicit qualifier (e.g., 'most current LLM agents without explicit curiosity mechanisms do not explore curiously'). The paper does not provide such a qualifier, and the contradiction weakens the empirical premise of the diagnosis.
The reader's weakest assumption concerned in-context learning's ability to satisfy C3 and C4, which the paper acknowledges as an open question. That assumption is not load-bearing because the paper's framework is forward-looking. Instead, the most immediate load-bearing problem is the internal inconsistency: the paper's blanket claim that current LLM agents lack curiosity is contradicted by its own example of a system that meets the task-driven criteria. This does not undermine the framework's value, but it does require the authors to qualify their central claim to avoid overstatement. The verdict remains unchanged because the primary contribution—the five-commitment framework—is unaffected by this rhetorical flaw, and the paper can be accepted with a minor textual adjustment.
The referee's report
If taken as a conceptual clarification, the paper performs a useful service: it forces the community to distinguish between stochastic variation of outputs (sampling) and genuine model-level adaptation driven by predictive improvement. The framework (C1–C5) is well-grounded in the existing curiosity literature and provides a precise vocabulary for discussing exploration in LLM agents. The diagnosis of current systems is incisive in many places. However, the paper's central universal claim is contradicted by its own positive example (WorldLLM), which weakens the rhetorical force; fixing this inconsistency would raise the significance substantially.
Pith's reading of the paper (a machine's summary, not the authors')
This position paper argues that current LLM agents do not exhibit genuine curiosity; they only explore stochastically by varying how they sample from a fixed next-token distribution. The authors derive five architectural commitments for curiosity from the reinforcement-learning tradition: a predictive component over environment responses, online improvement from current interaction, action selection driven by expected predictive improvement, persistence of improvements beyond the context window, and, in open-ended settings, self-generated prediction targets. Using these commitments, they diagnose existing methods—such as temperature sampling, tree search, retrieval-augmented generation, and memory systems—as satisfying only interaction-level exploration, not model-level adaptation. The paper reserves the term 'curiosity' for systems that meet these commitments and identifies open research problems for building such systems.
What carries the argument. The central machinery is the framework's five commitments (C1 to C5) and the accompanying system decomposition $S = \langle p_\theta, M, P, \pi \rangle$, where $p_\theta$ is the fixed LLM, $M$ is a predictive component over environment responses, $P$ is persistent state, and $\pi$ is the action-selection policy. The commitments require that $M$ improves from observations (C2), that $\pi$ is shaped by expected improvement in $M$ (C3), and that improvements persist in $P$ beyond the context window (C4). C5 extends curiosity to open-ended settings by requiring the agent to generate its own prediction targets.
Core claim. The core discovery is a conceptual framework that precisely defines curiosity for LLM agents by adapting long-standing ideas from artificial curiosity and intrinsic motivation. The framework's five commitments distinguish stochastic exploration—where the agent varies actions and contexts but leaves the underlying predictive function unchanged—from curious exploration—where the agent maintains a model of how the environment responds to its actions, updates that model online from the current interaction, chooses actions to reduce predictive uncertainty or increase learning progress, and retains the improved model beyond the immediate context. In open-ended scenarios, the agent must additionally originate its own prediction targets without external specification. The paper shows that none of the prominent current methods fully satisfy these commitments, and that terms like 'curiosity' and 'exploration' are often used for mechanisms that only diversify sampling without model-level adaptation.
The premise it rests on. The paper assumes that in-context learning cannot implement the full curiosity loop (specifically commitments C3 and C4) because the model weights remain fixed and context is bounded, which precludes action selection driven by expected predictive improvement and persistence beyond the context window.
If the paper is right (Pith's inferences, not the authors')
Existing methods that claim curiosity must be reevaluated; many amount to stochastic sampling, not model-level adaptation.
New benchmarks should measure within-episode prediction improvement and the influence of intrinsic signals on action selection, not just final outcomes.
Architecturally, curiosity requires integrating a learnable predictive component with the LLM, which may involve parametric updates, hypothesis states, or structured memory.
The framework clarifies the terminology, reserving 'curiosity' for systems that meet the commitments, enabling precise discussion of progress.
Combining intrinsic curiosity signals with extrinsic task rewards remains an open design problem, especially in open-ended action spaces.
The framework implies that in-context learning alone cannot satisfy commitments C3 (act to learn) and C4 (retain improvements), because it lacks a mechanism for action selection driven by expected predictive improvement and because improvements vanish with the context window.
Weight updates during deployment, perhaps through streaming online RL or continual learning, may be necessary to fully satisfy C4, raising engineering challenges for large-scale LLMs.
The behavioral tests proposed for C1-C4 could serve as a practical evaluation suite for curious agents, complementing outcome-based benchmarks.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
LLM agents are increasingly used for open-ended tasks, very common in research scenarios where they are required to explore, investigate, or act curiously. At inference time, an LLM is a fixed map from context to next-token distribution, the pretrained weights determine the function, and only its conditioning changes as context accumulates. Mechanisms like temperature, best-of-N, tree search, or RAG vary how the distribution is sampled or what it is conditioned on, but do not update the agent's model of the environment. We argue that current LLM agents do not explore curiously; they explore stochastically. From how curiosity is understood in the RL literature, it requires a predictive structure over how the environment responds to the agent's actions, which improves online from current interaction, drives action selection through its predicted improvement, and persists beyond the context window. Additionally, in its open-ended form, the agent should intrinsically originate what to investigate. Thus, we propose a framework defining curiosity in LLM agents, reserving the term for systems that satisfy it. In doing so, we identify the research directions its implementation opens up.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-10
The authors' main claim, as the classifier picked it out 0.95
“We argue that current LLM agents do not explore curiously; they explore stochastically.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.16 · human participants 0.04 · controlled comparison 0.03 · randomized assignment 0.01 · states quantitative result 0.02 · compares with prior results 0.09 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| LLM agents are increasingly used for open-ended tasks, very common in research scenarios where they are required to explore, investigate, or act curiously. | background 1.00 | 0.05 | 0.03 | asserted 0.99 |
| At inference time, an LLM is a fixed map from context to next-token distribution, the pretrained weights determine the function, and only its conditioning changes as context accumulates. | background 1.00 | 0.08 | 0.03 | asserted 1.00 |
| Mechanisms like temperature, best-of-N, tree search, or RAG vary how the distribution is sampled or what it is conditioned on, but do not update the agent's model of the environment. | background 0.92 | 0.11 | 0.06 | asserted 1.00 |
| We argue that current LLM agents do not explore curiously; they explore stochastically. | result 0.34 | 0.39 | 0.76 | asserted 1.00 |
| From how curiosity is understood in the RL literature, it requires a predictive structure over how the environment responds to the agent's actions, which improves online from current interaction, drives action selection through its predicted improvement, and persists beyond the context window. | background 0.86 | 0.08 | 0.12 | asserted 1.00 |
| Additionally, in its open-ended form, the agent should intrinsically originate what to investigate. | contribution 0.30 | 0.12 | 0.08 | asserted 0.98 |
| Thus, we propose a framework defining curiosity in LLM agents, reserving the term for systems that satisfy it. | contribution 1.00 | 0.88 | 0.34 | asserted 0.97 |
| In doing so, we identify the research directions its implementation opens up. | future work 0.91 | 0.74 | 0.06 | asserted 0.75 |
What kind of paper the classifier reports this is: position or commentary 1.00 · evidence from argument 1.00
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-10: 1 request, 6,232 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (1)
References (47)
- [20] Guillaume Levy, Cédric Colas, Pierre-Yves Oudeyer, Thomas Carta, and Clément Romac. Worldllm: Improving llms’ world modeling using curiosity-driven theory-making.arXiv preprint arXiv:2506.06725, 2025 on PitharXiv
- [1] Dhruv Agarwal, Bodhisattwa Prasad Majumder, Reece Adamson, Megha Chakravorty, Satvika Reddy Gavireddy, Aditya Parashar, Harshit Surana, Bhavana Dalvi Mishra, Andrew McCallum, Ashish Sabharwal, et al. Autodiscovery: Open-ended scientific discovery via bayesian surprise.Advances... on Pith
- [2] Alexis Audran-Reiss, Jordi Armengol-EstapÊ, Karen Hambardzumyan, Amar Budhiraja, Martin Josifoski, Edan Toledo, Rishi Hazra, Despoina Magka, Michael Shvartsman, Parth Pathak, et al. What does it take to be a good ai research agent? studying the role of ideation diversity.arXi... on PitharXiv
- [3] Daniel E Berlyne. Conflict, arousal, and curiosity. 1960 on Pith
- [4] Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. InProceedings of the AAAI confer... on Pith
- [5] Yuri Burda, Harri Edwards, Deepak Pathak, Amos Storkey, Trevor Darrell, and Alexei A Efros. Large-scale study of curiosity-driven learning.arXiv preprint arXiv:1808.04355, 2018 on PithPith's reviewarXiv
- [6] Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, et al. Cde: Curiosity-driven exploration for efficient reinforcement learning in large language models.arXiv preprint arXiv:2509.09675, 2025 on PithPith's reviewarXiv
- [7] Abhimanyu Dubey and Alex ‘Sandy’ Pentland. Kernel methods for cooperative multi-agent con- textual bandits. In Hal Daumé III and Aarti Singh, editors,Proceedings of the 37th International Conference on Machine Learning, volume 119 ofProceedings of Machine Learning Research, pa... on Pith
- [8] Eric Elmoznino, Sangnie Bhardwaj, Johannes von Oswald, Rajai Nasser, João Sacramento, Rif A Saurous, Guillaume Lajoie, et al. Can in-context learning support intrinsic curiosity? arXiv preprint arXiv:2606.19476, 2026 on PithPith's reviewarXiv
- [9] Leon Engländer, Sophia Althammer, Ahmet Üstün, Matthias Gallé, and Tom Sherborne. Agents explore but agents ignore: Llms lack environmental curiosity.arXiv preprint arXiv:2604.17609, 2026 on PithPith's reviewarXiv
- [10] Sébastien Forestier, Rémy Portelas, Yoan Mollard, and Pierre-Yves Oudeyer. Intrinsically motivated goal exploration processes with automatic curriculum learning.Journal of Machine Learning Research, 23(152):1–41, 2022 on Pith
- [11] David Ha and Jürgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2(3):440, 2018 on PithPith's reviewarXiv
- [12] Vincent Herrmann, Louis Kirsch, and Jürgen Schmidhuber. Learning one abstract bit at a time through self-invented experiments encoded as neural networks. InInternational Workshop on Active Inference, pages 254–274. Springer, 2023 on Pith
- [13] Qian Huang, Jian V ora, Percy Liang, and Jure Leskovec. Mlagentbench: Evaluating language agents on machine learning experimentation.arXiv preprint arXiv:2310.03302, 2023 on PithPith's reviewarXiv
- [14] Laurent Itti and Pierre Baldi. Bayesian surprise attracts human attention.Advances in neural information processing systems, 18, 2005 on Pith
- [15] Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Stanley Jungkyu Choi, and Minjoon Seo. Towards continual knowledge learning of language models.arXiv preprint arXiv:2110.03215, 2021 on PithPith's reviewarXiv
- [16] Peter Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Oyvind Tafjord, and Peter Clark. Discoveryworld: A virtual environment for developing and evaluating automated scientific discovery agents.Advances in Neural Inform... on Pith
- [17] Yulun Jiang, Liangze Jiang, Damien Teney, Michael Moor, and Maria Brbic. Meta-rl induces exploration in language agents.arXiv preprint arXiv:2512.16848, 2025 on PitharXiv
- [18] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks.Proceedings of the national academy of scienc... on Pith
- [19] Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steiger- wald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, et al. In-context reinforcement learning with algorithm distillation.arXiv preprint arXiv:2210.14215, 2022 on PithPith's reviewarXiv
show 27 more
- [21] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems,... on Pith
- [22] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scien- tist: Towards fully automated open-ended scientific discovery.arXiv preprint arXiv:2408.06292, 2024 on PithPith's reviewarXiv
- [23] Alexander Novikov, Ngân V˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery.arXiv preprint arXiv:2506.1... on PithPith's reviewarXiv
- [24] Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.arXiv preprint arXiv:2209.11895, 2022 on PithPith's reviewarXiv
- [25] Pierre-Yves Oudeyer, Frdric Kaplan, and Verena V Hafner. Intrinsic motivation systems for autonomous mental development.IEEE transactions on evolutionary computation, 11(2): 265–286, 2007 on Pith
- [26] Ao Qu, Han Zheng, Zijian Zhou, Yihao Yan, Yihong Tang, Shao Yong Ong, Fenglu Hong, Kaichen Zhou, Chonghe Jiang, Minwei Kong, et al. Coral: Towards autonomous multi-agent evolution for open-ended discovery.arXiv preprint arXiv:2604.01658, 2026 on PithPith's reviewarXiv
- [27] Mark B Ring. Child: A first step towards continual learning.Machine Learning, 28(1):77–104, 1997 on Pith
- [28] Mark Bishop Ring.Continual learning in reinforcement environments. The University of Texas at Austin, 1994 on Pith
- [29] Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 625(7995... on Pith
- [30] Cansu Sancaktar, Christian Gumbsch, Andrii Zadaianchuk, Pavel Kolev, and Georg Martius. SENSEI: Semantic exploration guided by foundation models to learn versatile world models. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Ki... on Pith
- [31] Juergen Schmidhuber. Exploring the predictable. InAdvances in evolutionary computing: theory and applications, pages 579–612. Springer, 2003. 12 on Pith
- [32] Jürgen Schmidhuber. Curious model-building control systems. InProc. international joint conference on neural networks, pages 1458–1463, 1991 on Pith
- [33] Jürgen Schmidhuber. A possibility for implementing curiosity and boredom in model-building neural controllers. InProc. of the international conference on simulation of adaptive behavior: From animals to animats, pages 222–227, 1991 on Pith
- [34] Jürgen Schmidhuber. On learning how to learn learning strategies. Technical Report FKI- 198-94, Fakultät für Informatik, Technische Universität München, November 1994. URL https://people.idsia.ch/~juergen/FKI-198-94ocr.pdf on Pith
- [35] Jürgen Schmidhuber. Formal theory of creativity, fun, and intrinsic motivation (1990–2010). IEEE transactions on autonomous mental development, 2(3):230–247, 2010 on Pith
- [36] Jürgen Schmidhuber. Powerplay: Training an increasingly general problem solver by continually searching for the simplest still unsolvable problem.Frontiers in psychology, 4:313, 2013 on Pith
- [37] Jürgen Schmidhuber, Jieyu Zhao, and Marco Wiering. Shifting inductive bias with success-story algorithm, adaptive levin search, and incremental self-improvement.Machine Learning, 28(1): 105–130, 1997 on Pith
- [38] Jürgen Schmidhuber. Artificial curiosity & creativity since 1990-91. https://people.idsia. ch/~juergen/artificial-curiosity-since-1990.html , 2021, 2022, 2023, 2025. Ac- cessed: 2026-02-17 on Pith
- [39] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.Advances in neural information processing systems, 36:8634–8652, 2023 on Pith
- [40] Jan Storck, Sepp Hochreiter, Jürgen Schmidhuber, et al. Reinforcement driven information acquisition in non-deterministic environments. InProceedings of the international conference on artificial neural networks, Paris, volume 2, pages 159–164, 1995 on Pith
- [41] Yi Sun, Faustino Gomez, and Jürgen Schmidhuber. Planning to be surprised: Optimal bayesian exploration in dynamic environments. InInternational conference on artificial general intelli- gence, pages 41–51. Springer, 2011 on Pith
- [42] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023 on PithPith's reviewarXiv
- [43] Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference.arXiv preprint arXiv:2111.02080, 2021 on PithPith's reviewarXiv
- [44] Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search.arXiv preprint arXiv:2504.08066, 2025 on PithPith's reviewarXiv
- [45] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809–11822, 2023 on Pith
- [46] Jenny Zhang, Joel Lehman, Kenneth Stanley, and Jeff Clune. Omni: Open-endedness via models of human notions of interestingness.arXiv preprint arXiv:2306.01711, 2023 on PithPith's reviewarXiv
- [47] Kongcheng Zhang, Qi Yao, Shunyu Liu, Yingjie Wang, Baisheng Lai, Jieping Ye, Mingli Song, and Dacheng Tao. Consistent paths lead to truth: Self-rewarding reinforcement learning for llm reasoning.arXiv preprint arXiv:2506.08745, 2025. 13 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.

No comments yet.
Sign in to comment