How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison

Jiayin Wang; Min Zhang; Weizhi Ma; Zhiquang Guo

REVIEW 3 major objections 4 minor 39 references

How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs can improve from experience, but they learn more slowly and less steadily than humans.

desk verdict A useful evaluation axis for test-time learning, but the main semantic-game result is confounded by using the same LLM as judge; still worth a serious referee. read the letter →

arxiv 2506.14448 v2 pith:YGACVKZH submitted 2025-06-17 cs.CL

Jiayin Wang , Zhiquang Guo , Weizhi Ma , Min Zhang This is my paper

classification cs.CL

keywords test-timelearningLLMevaluationsemanticgamesTwentyQuestionsWhoisUndercoverhumancomparisonexperience-basedpolicycumulative

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that how quickly a model improves from its own experience at test time is a distinct and under-measured axis of intelligence, separate from how well it performs on static benchmarks. To measure it, the authors build an evaluation around semantic games—Twenty Questions and Who is Undercover, plus the AIME math exam—in which the model plays rounds, reflects on its interactions and rewards, and distills that reflection into a short policy that guides the next round. Under this protocol, current LLMs do show measurable gains from experience-derived policies, but the gains are uneven: they shrink or reverse as experience accumulates, and they are outweighed by policies written by human players. The authors read this as evidence that LLMs have real but immature test-time learning ability, and that the gap between self-derived and human-authored strategies marks concrete headroom for improvement.

What carries the argument

The load-bearing mechanism is the policy-based distillation of experience: after each block of games, the model is given its past dialogue interactions, rewards, and its own reflections, and is prompted to write a short test-time policy (roughly 250 tokens) that it will follow in the next block. This compresses the full history into an executable strategy, making repeated experience cheap enough for a cumulative setting, where policies are updated as rounds grow. The evaluation framework contrasts four experience representations—no policy, a rule-only policy, an experience-derived policy, and a human-authored policy—and runs them in two regimes: a fixed five-round experience setting and an incremental cumulative setting over fifty rounds. The semantic games themselves are chosen as testbeds because they are open-ended, resistant to saturation, and require discovering latent strategies rather than recalling memorized answers.

What would settle it

Run the same Twenty Questions protocol with an independent answerer (a different model, or a human who knows the target word) instead of the questioner's own LLM, keeping the policy-distillation pipeline fixed; if the experience-derived policy no longer improves scores, the measured learning is self-consistency, not general improvement. A second check is the cumulative curve: a model whose experience-enabled reward stays at or below its no-experience baseline across fifty rounds would refute the claim that gains accumulate.

Watch

Extended reading notes

Core claim

The paper's central claim is that large language models can improve their performance on experience-based, reasoning-intensive tasks through test-time experience, but not as well as humans. Concretely, when a model is given a five-round history of its own questions, answers, and rewards and is asked to write a policy from that history, its subsequent performance improves on both Twenty Questions and Who is Undercover—this is Finding 1. Yet the improvement is not robust: in the cumulative setting, only Claude sustains a consistent advantage from accumulating experience, while GPT-4o's gains appear late and DeepSeek-V3 degrades after five rounds (Finding 3). Eight human players, given the same Twenty Questions setup, improve faster and more steadily than the best model, approaching near-perfect binary-questioning performance (Finding 4). The paper also reports that thinking models such as o1 and DeepSeek-R1 show no test-time learning gains from self-derived policies, which it links to R1's known few-shot degradation (Finding 5).

Load-bearing premise

The whole comparison assumes that using the same LLM to simulate the game environment produces a neutral, trustworthy source of feedback; if the model is instead just getting better at predicting its own judge's behavior, the measured learning may not transfer to real interactions.

Editorial extensions

If this is right

Static benchmark rankings will systematically overstate how capable a model is as a learner; two models with equal static scores can differ sharply in whether and how fast they improve from experience.
Test-time learning should be treated as a separate evaluation axis, with both a fixed-experience score and a cumulative-learning curve, because the two can disagree.
Because human-authored policies consistently beat self-derived ones, current models have headroom: better strategy induction, not just more compute at inference, is what stands between them and human-level learning.
The absence of test-time gains in thinking models suggests that long chain-of-thought reasoning optimized for single problems does not automatically transfer to accumulating and applying experience across episodes.
Cumulative learning trajectories discriminate among models, so reporting only average gains can hide the instability that matters for deployment in interactive settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to replace the self-simulated answerer with another model or a human oracle; if learning gains vanish, the measured improvement may be calibration to the judge's own biases rather than general learning.
The framework could be moved from games to real-world interactive tasks such as customer support or scientific tool use, where feedback is noisy; the cumulative setting's instability would then become a practical reliability metric.
The policy-distillation step resembles an in-context version of what fine-tuning does in-weight; open-weights models could be tested with explicit test-time training to see whether parameter updates close the gap to human learning curves.
The eight-participant human sample is a floor, not a ceiling; a larger, more diverse sample would let the human learning curve be decomposed into strategy acquisition versus task familiarity effects.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper gives the community something real: a measurement axis for test-time learning, with a clean pipeline and a genuine human comparison. That is worth having, and the authors deserve credit for framing the question clearly and for not overselling the cumulative curves, where instability is visible.

What is actually new is applying the adaptive memory pipeline (Suzgun et al.) to open-ended semantic games, and adding a human learning baseline. The finding that non-thinking models improve modestly with experience-based policies, while thinking models do not, is interesting even if the latter is consistent with known few-shot degradation in R1. The AIME contrast is a nice touch, even with negative results.

The paper is also honest. The Limitations section admits the narrow environment set, the small human sample, and the absence of test-time training. The neutral role-name fix for the undercover game is a thoughtful detail.

The soft spots are real and load-bearing. In Twenty Questions, the judge is the same LLM under test, run at temperature 1. That is stated plainly in Section 4.1, so it is not hidden. But it means the measured improvement could be the model learning its own judge's biases and category ordering, not the objective structure of the word set. The human baseline in Section 4.4 uses the same simulated judge, so the human-model gap also reflects how well each human's questions align with that judge's semantics. This confound weakens the central claim that LLMs show test-time learning on this task.

The AIME results contradict the 'across models and tasks' framing in Finding 1: two models get worse, one is flat. The claim should be qualified to the semantic games. Also, there is no uncertainty quantification anywhere; with M=32, three cumulative samples, and eight humans, point estimates overstate precision. The rule-based policy baseline hurts all models in Twenty Questions, which is itself a finding, but it suggests the policy pipeline is sensitive to prompt style and needs more discussion.

None of this kills the paper's value as a proposal for a new evaluation axis. The confound is addressable with an independent judge or a fixed different model, and adding confidence intervals would tighten the claims. As is, the central comparison needs a caveat.

This paper deserves peer review. The evaluation idea is important enough, and the empirical work is honest enough, that a good referee can push the authors to fix the controls and soften the claims. I would not cite the current version's numbers as evidence for test-time learning ability, but I would cite the framing and the game-based testbeds. Bring it to reading group if your group cares about LLM evaluation or agentic learning; there is a good discussion to be had about what counts as learning when the environment is the model itself.

Referee Report

3 major / 4 minor

Summary. The paper advocates test-time learning as a distinct evaluation axis for LLMs and proposes semantic games—Twenty Questions and Who is Undercover—together with AIME 2025 as testbeds. It introduces a lightweight framework that compares model performance without policy, with a rule-based policy, with a policy distilled from five rounds of the model's own experience, and with a human-authored policy, and it adds an incremental setting that tracks cumulative reward over 50 rounds. On this basis the authors report that experience-derived policies yield measurable improvements in the two semantic games, that cumulative gains are model-dependent (Claude being the clearest beneficiary), and that eight human participants learn faster and more stably than the best model. The paper concludes that LLMs have measurable but limited test-time learning ability relative to humans.

Significance. If the central finding held, the paper would add a useful evaluation axis beyond static benchmarks and provide a repeatable protocol for comparing human and machine learning curves. The design has several strengths: it uses a policy-based representation of experience rather than unbounded context, it includes an external human baseline, and it ships code and data. The contribution is conditional, however, because the main quantitative claims rest on a self-simulated environment and on small improvements without reported variance. The framing contrast with humans is valuable but needs stronger experimental controls before the general claim can be accepted.

major comments (3)

[Sections 3.1, 4.1, 4.2, 4.4] Section 4.1 states that in Twenty Questions "the environment is simulated using the same model under evaluation," and Section 3.1 says the environment responds with Yes/No/Invalid. Since the judge is the same LLM being measured, a policy distilled from five prior rounds can improve by learning the stochastic judge's category ordering and lexical preferences (including its notion of "Invalid") rather than by discovering the objective structure of the 157-word set. The human comparison in Section 4.4 uses this same simulated judge, so the human-model learning-speed gap is also contaminated by how well each participant's questions align with the LLM judge's semantics. Please either replace the self-simulation with a fixed independent judge (e.g., a different, frozen model or a rule-based oracle over the word set) or provide a control showing that the learned policies also improve against ground-truth category labels. Without this, Findings 1 and 4 are not established for the Twenty Questions testbed.
[Section 4.2, Table 2 (AIME 2025 rows)] Finding 1 claims "measurable improvements across models and tasks," but the AIME 2025 rows in Table 2 show negative or zero improvement for all three models (GPT-4o -30.57%, Claude 3.5 Sonnet 0.00%, DeepSeek-V3 -10.62%). This is a direct contradiction of the cross-task wording. The paper should either restrict Finding 1 to the two semantic games, or provide a substantive explanation of why AIME is a valid test-time-learning testbed and why the negative result is consistent with the framework.
[Tables 2–3, Figures 3–4] No uncertainty quantification is reported for the central comparisons. In Twenty Questions the reported experience-policy improvements are 3.97–6.33 percentage points; with M=32 test cases, sampling noise can be material. In Who is Undercover, 32-game win rates have binomial standard errors up to roughly 9 percentage points, so the 10–25 point gains need confidence intervals or paired tests to be described as "measurable." The human study comprises eight participants with no statistical test of the human-model gap, and Figure 4 shows two groups without error bars. Please report per-run values, bootstrap intervals, or significance tests for the improvements and for the human comparison.

minor comments (4)

[Section 3.2.2, Eq. (1)] The definition of r_exp(t) for t=1 appears to mix baseline and experience rewards in the numerator; please clarify the intended initialization and why it differs from t>1.
[Section 4.4, Figure 4] The figure caption refers to an "upper figure" and a "lower figure," but the panels are not labeled in the image; add (a)/(b) labels and describe the split criterion (performance variance) in the caption.
[Section 3.2.1] The paper reports a pilot study comparing full-history and policy-based representations and says the latter underperformed, but no pilot results are shown; please include the numbers or state that they are provided in supplementary material.
[Appendix B.1] The human policy in Appendix B.1 contains a typo ("electoricity" instead of "electricity"); please copyedit the appendix, which also has inconsistent formatting in the policy lists.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation chain: no fitted parameters, no self-citations, and an external human baseline anchor the claims. The sole self-referential element is that the Twenty Questions judge is the same LLM under evaluation, a disclosed confound that partially conflates self-consistency with task learning but does not force the measured gains.

full rationale

Walking the derivation chain from the definition of test-time learning to the reported findings, the chain is empirical and largely self-contained. There are no fitted parameters renamed as predictions, no self-citations (the reference list contains no papers by the present authors), no imported uniqueness theorems, and no ansatz smuggled in via citation. The evaluation is an operational before/after comparison of policies derived from experience, with cumulative curves computed from Equation 1 whose pooled starting-point normalization does not force divergence; indeed the curves move in opposite directions across models (Claude gains, DeepSeek declines), and the thinking-model results and AIME condition show no gains or degradation, which would not happen if the improvement were equivalent to its input by construction. The human study provides an external anchor: humans play the same simulated game and their cumulative rewards, approaching the perfect-binary-search ceiling, are genuinely measured. The only load-bearing self-referential element is the single-agent environment simulation using the same model as the judge, disclosed in Sections 3.1 and 4.1. That design makes part of the measured 'learning' a matter of the model aligning with its own judge's semantics, which is a validity confound and external-consistency limitation rather than a definitional circularity: the experiment could have shown, and often did show, no improvement. The Limitations section acknowledges the narrow scope of environments but does not list the same-model judge as a threat to attribution; per the review rule, that omission is weighed here in the verdict. Overall: no significant circularity in the derivation or self-citation sense; the score of 2 reflects the mild, disclosed self-referential design that partially contaminates what is measured as 'test-time learning' in the Twenty Questions and, by extension, the human-model gap.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no invented entities. The free parameters are experimental design choices selected by hand, and they could affect the magnitude of the measured learning effect. The listed axioms are domain assumptions required for the evaluation to measure what it claims.

free parameters (4)

Prior experience rounds N = 5
Chosen in pilot as 'yield stable results' (Section 4.1); no theory or variance analysis. The measured learning gain may depend on this amount.
Test cases M = 32
Chosen to yield stable results (Section 4.1); no power analysis. With only 32 games, confidence intervals would be wide.
Cumulative rounds t = 50
Chosen for the cumulative setting (Section 4.1); arbitrary and only applied to Twenty Questions in the reported cumulative analysis.
Sampling temperature = 1.0
Set to 1 to support dynamic testbeds (Section 4.1); affects variance and the stability of learning measurements.

assumptions (5)

domain assumption A distilled policy paragraph captures the model's test-time learning at least as well as the full interaction history.
Section 3.2.1: the authors found that the full-history approach underperformed and was more expensive, so they adopted policy-based representation. If distillation loses information, the measurement may understate learning.
domain assumption The same-LLM simulated environment gives valid yes/no feedback.
Section 4.1: the environment is simulated using the same LLM as the questioning model. The model may exploit its own judge's biases, threatening validity.
domain assumption 32 games per condition are enough for stable estimates.
No error bars or significance tests are reported; with N=32, the observed small improvements may be within sampling noise.
domain assumption Eight human participants are representative of human test-time learning.
Section 4.4 and Limitations: eight undergraduate and PhD students are a small and homogeneous sample, and participants are split into two groups post hoc based on performance variance.
domain assumption Neutral role names prevent value-alignment distortions in the undercover task.
Appendix C: neutral names such as 'difference' and 'normal' are used because models sometimes deny being 'undercover'. The validity of the multi-agent results depends on this fix.

how reviews work

0 comments

Cite this review

Pith. "Pith review of How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison." pith.science (2026). https://pith.science/paper/YGACVKZH

@misc{pith2026250614448,
  author       = {Pith},
  title        = {Pith review of: How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YGACVKZH}},
  note         = {Machine review of arXiv:2506.14448}
}

read the original abstract

As evaluation designs of large language models may shape our trajectory toward artificial general intelligence, comprehensive and forward-looking assessment is essential. Existing benchmarks primarily assess static knowledge, while intelligence also entails the ability to rapidly learn from experience. To this end, we advocate for the evaluation of Test-time Learning, the capacity to improve performance in experience-based, reasoning-intensive tasks during test time. In this work, we propose semantic games as effective testbeds for evaluating test-time learning, due to their resistance to saturation and inherent demand for strategic reasoning. We introduce an objective evaluation framework that compares model performance under both limited and cumulative experience settings, and contains four forms of experience representation. To provide a comparative baseline, we recruit eight human participants to complete the same task. Results show that LLMs exhibit measurable test-time learning capabilities; however, their improvements are less stable under cumulative experience and progress more slowly than those observed in humans. These findings underscore the potential of LLMs as general-purpose learning machines, while also revealing a substantial intellectual gap between models and humans, irrespective of how well LLMs perform on static benchmarks.

Figures

Figures reproduced from arXiv: 2506.14448 by the authors.

**Figure 1.** Test-time Learning Evaluation Pipeline. predominantly focus on measuring the expertise of language models in performing specific tasks. However, intelligence is not solely defined by the possession of expert knowledge (Krathwohl, 2002; Minsky, 1988). For example, individuals without profound knowledge can still demonstrate intelligence through their speed to acquire new skills through experience (Silver and Sutton,… view at source ↗

**Figure 2.** Test-time Learning Evaluation Settings with Fixed Amount and Incremental Experience. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Cumulative Test-Time Learning Performance on Twenty Question. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Human Performance on Twenty Question. Finding 5: Test-time learning is not observed in thinking models, consistent with the findings reported in R1’s original paper that CoT in fewshot cases may degrade model performance. 4.6 Further Analyses In the Twenty Questions e…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 16 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Marwa Abdulhai, Isadora White, Charlie Snell, Charles Sun, Joey Hong, Yuexiang Zhai, Kelvin Xu, and Sergey Levine. 2023. Lmrl gym: Benchmarks for multi-turn reinforcement learning with language models. arXiv preprint arXiv:2311.18232

arXiv 2023
[4]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023
[5]

Anthropic. 2024. https://www.anthropic.com/news/claude-3-5-sonnet Claude 3.5 sonnet

work page 2024
[6]

Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1--45

2024
[7]

Gonzalez, Ion Stoica, and Eric P

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

2023
[8]

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

arXiv 2021

Show all 39 references

[9]

Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel. 2016. Rl2: Fast reinforcement learning via slow reinforcement learning. arXiv preprint arXiv:1611.02779

2016 arXiv
[10]

Yossi Gandelsman, Yu Sun, Xinlei Chen, and Alexei Efros. 2022. Test-time training with masked autoencoders. Advances in Neural Information Processing Systems, 35:29374--29385

2022
[11]

Aur \'e lien Garivier and Eric Moulines. 2011. On upper-confidence bound policies for switching bandit problems. In International conference on algorithmic learning theory, pages 174--188. Springer

2011
[12]

Amago: Scalable in-context reinforcement learning for adaptive agents

Jake Grigsby, Linxi Fan, and Yuke Zhu. Amago: Scalable in-context reinforcement learning for adaptive agents. In The Twelfth International Conference on Learning Representations
[13]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

2025 arXiv
[14]

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

2024 arXiv
[15]

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

2024 arXiv
[16]

Shuyang Jiang, Yuhao Wang, and Yu Wang. 2023. Selfevolve: A code evolution framework via large language models. arXiv preprint arXiv:2306.02907

2023 arXiv
[17]

David R Krathwohl. 2002. A revision of bloom's taxonomy: An overview. Theory into practice, 41(4):212--218

2002
[18]

Robert Lange, Yingtao Tian, and Yujin Tang. 2024. Large language models as evolution strategies. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 579--582

2024
[19]

In-context reinforcement learning with algorithm distillation

Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Stenberg Hansen, Angelos Filos, Ethan Brooks, and 1 others. In-context reinforcement learning with algorithm distillation. In The Eleventh International Conference ...
[20]

Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. 2023. Supervised pretraining can learn in-context reinforcement learning. Advances in Neural Information Processing Systems, 36:43057--43083

2023
[21]

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

2024 arXiv
[22]

Yuejiang Liu, Parth Kothari, Bastien Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. 2021. Ttt++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34:21808--21820

2021
[23]

Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, and Feryal Behbahani. 2023 a . Structured state space models for in-context reinforcement learning. Advances in Neural Information Processing Systems, 36:47016--47031

2023
[24]

Jianqiao Lu, Wanjun Zhong, Wenyong Huang, Yufei Wang, Fei Mi, Baojun Wang, Weichao Wang, Lifeng Shang, and Qun Liu. 2023 b . Self: Language-driven self-evolution for large language model

2023
[25]

Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568

2023 arXiv
[26]

MAA. 2025. https://maa.org/ American invitational mathematics examination (aime)

2025
[27]

Marvin Minsky. 1988. Society of mind. Simon and Schuster

1988
[28]

Steven Morad, Ryan Kortvelesy, Matteo Bettini, Stephan Liwicki, and Amanda Prorok. 2023. Popgym: Benchmarking partially observable reinforcement learning. arXiv preprint arXiv:2303.01859

2023 arXiv
[29]

Cheng Qian, Shihao Liang, Yujia Qin, Yining Ye, Xin Cong, Yankai Lin, Yesai Wu, Zhiyuan Liu, and Maosong Sun. 2024. Investigate-consolidate-exploit: A general strategy for inter-task agent self-evolution. arXiv preprint arXiv:2401.13996

2024 arXiv
[30]

David Silver and Richard S Sutton. 2025. Welcome to the era of experience. Google AI

2025
[31]

Samarth Sinha, Peter Gehler, Francesco Locatello, and Bernt Schiele. 2023. Test: Test-time self-training under distribution shift. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2759--2769

2023
[32]

Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. 2020. Test-time training with self-supervision for generalization under distribution shifts. In International conference on machine learning, pages 9229--9248. PMLR

2020
[33]

Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A Efros, and Moritz Hardt. 2019. Test-time training for out-of-distribution generalization

2019
[34]

Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. 2025. Dynamic cheatsheet: Test-time learning with adaptive memory. arXiv preprint arXiv:2504.07952

2025 arXiv
[35]

Zhengwei Tao, Ting-En Lin, Xiancai Chen, Hangyu Li, Yuchuan Wu, Yongbin Li, Zhi Jin, Fei Huang, Dacheng Tao, and Jingren Zhou. 2024. A survey on self-evolution of large language models. arXiv preprint arXiv:2404.14387

2024 arXiv
[36]

Lin Xu, Zhiyuan Hu, Daquan Zhou, Hongyu Ren, Zhen Dong, Kurt Keutzer, See Kiong Ng, and Jiashi Feng. 2023. Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration. arXiv preprint arXiv:2311.08562

2023 arXiv
[37]

Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744--20757

2022
[38]

Yajie Yu and Yue Feng. 2025. Policyevol-agent: Evolving policy via environment perception and self-awareness with theory of mind. arXiv preprint arXiv:2504.15313

2025 arXiv
[39]

Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, and Aviral Kumar. 2024. Archer: Training language model agents via hierarchical multi-turn rl. arXiv preprint arXiv:2402.19446

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Marwa Abdulhai, Isadora White, Charlie Snell, Charles Sun, Joey Hong, Yuexiang Zhai, Kelvin Xu, and Sergey Levine. 2023. Lmrl gym: Benchmarks for multi-turn reinforcement learning with language models. arXiv preprint arXiv:2311.18232

arXiv 2023

[4] [4]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023

[5] [5]

Anthropic. 2024. https://www.anthropic.com/news/claude-3-5-sonnet Claude 3.5 sonnet

work page 2024

[6] [6]

Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1--45

2024

[7] [7]

Gonzalez, Ion Stoica, and Eric P

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

2023

[8] [8]

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

arXiv 2021