Machine Learning · arXiv:2611.19700 v1 · submitted 15 Nov 2026 · 45 references
Where Should LLM Agents Adapt at Test Time? In Context, in Weights, or Both
The paper says
“LLM agents operating repeatedly in the same environment, such as a code repository, a database, or an application suite, encounter tasks that share latent structure.”
more of the abstract
Experience can reveal this structure, allowing agents to reuse environment knowledge and learn from environmental feedback. Agents can adapt at test-time using this experience through two channels: retaining it in context and updating their weights. These channels are usually studied separately, leaving unclear what each contributes and when they complement each other. We compare in-context learning (ICL), reinforcement learning with verifiable rewards using GRPO, and self-distillation using SDPO, alone and in combination, using a shared online protocol and modular framework. Agents encounter matched task streams, are evaluated on each task before adapting to it, and learn only from their own attempts and environment feedback. We evaluate 14 adaptation mechanisms and a stateless baseline with Qwen3.5-9B on 109 tasks from four environments drawn from SWE-bench Verified, AppWorld, ScienceWorld, and BIRD-Critic-SQLite, using five task orders per environment. ICL provides the largest standalone gain: its best policy raises mean success across environments from 28.8% to 49.4%. Adding GRPO to this policy achieves the highest mean success, 51.6%. While adding GRPO improves the overall mean under every ICL policy, SDPO has mixed effects. Trace analyses show fewer lookups and shorter attempts under ICL, consistent with reusing knowledge of the shared structure. However, ICL also frequently skips inspection and verification for the current task. Adding GRPO to ICL helps restore these behaviors while preserving its reduced environment lookups. These observations suggest that reinforcement learning with verifiable rewards can refine how agents use retained experience, motivating the joint study of context management and weight updates.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 17 Nov 2026 · read the full text (137,630 characters) · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up. A careful empirical comparison of in-context learning, GRPO, and SDPO for test-time adaptation of LLM agents, finding that ICL gives the largest standalone gain and adding GRPO yields the best overall performance. This line is wrong
10 minor
How often objections from reviews like this one are right: not yet measured
The paper, first character to last
The reviewer was given the whole text, 137,630 characters. Not checked: figures, code, what the 45 references say.
10 minor objections
The paper says, at Abstract
Pith says: minor objection 1 of 10
The abstract states 'We evaluate 14 adaptation mechanisms and a stateless baseline with Qwen3.5-9B on 109 tasks from four environments.' The number 109 is the sum of tasks across worlds (22+21+22+44=109), but the paper also mentions additional experiments on scikit-learn (32 tasks) and Toxicology (64 tasks) in Appendix D. The abstract should clarify that the main evaluation uses 109 tasks.
The paper says, at Section 3.1 (Stream of related tasks)
Pith says: minor objection 2 of 10
The world names (Xarray, File System, Measurement, Financial) are descriptive but the mapping to benchmarks is not immediately clear from the text. Table 2 in Appendix A.2 clarifies this, but a brief reminder in the main text would help readability.
show 8 more
The paper says, at Section 3.2 (Adaptation mechanisms)
Pith says: minor objection 3 of 10
The description of GRPO says 'without a KL penalty or standard-deviation normalization.' This is a notable departure from standard PPO/GRPO. The paper should briefly justify this choice or cite prior work that uses this variant.
The paper says, at Section 4.2 (Results), Table 1
Pith says: minor objection 4 of 10
The table uses superscripts for standard errors (e.g., '13.4^±1.0'). The formatting is unusual and may be difficult to parse. Consider using parentheses or separate columns.
The paper says, at Section 5.1 (Outcomes and Interaction Length), Figure 3
Pith says: minor objection 5 of 10
The figure caption says 'Results weight the five stream seeds equally, with 16 attempts per task and stream.' This is fine, but the paper should clarify whether the error bars (if any) in the figure represent standard errors across seeds or across tasks within a seed. The figure as described does not show error bars, but the text refers to 'mean proportions' without variability.
The paper says, at Section 7 (Discussion and Limitations)
Pith says: minor objection 6 of 10
The paper states 'We study one model on subsets of four benchmarks, with five task orders, raw-trajectory context policies, and specific update recipes.' This is an honest limitation, but the paper should also note that the model (Qwen3.5-9B) is not the most recent or largest, and results may not generalize to other model families or sizes.
The paper says, at Appendix A.2 (Benchmark environments)
Pith says: minor objection 7 of 10
Table 2 reports turn limits: SWE-bench 40, AppWorld 40, ScienceWorld 30, BIRD 5. The BIRD limit of 5 turns is very short compared to the others. This may affect the behavioral metrics (e.g., querying) and should be discussed in the main text.
The paper says, at Appendix A.3 (Adaptation settings)
Pith says: minor objection 8 of 10
The paper states 'Our SDPO implementation supports only one optimizer step per rollout batch, so we allocate an additional round to provide more fresh on-policy experience and adaptation opportunities.' This is an implementation limitation, not a principled choice. The paper should note whether this could be addressed in future work.
The paper says, at Appendix B.1 (Success rates by seed)
Pith says: minor objection 9 of 10
The longtable in Appendix B.1 is very detailed but the formatting makes it hard to read. Consider using a more compact layout or highlighting the best results per seed more clearly.
The paper says, at Appendix D (Additional Environments)
Pith says: minor objection 10 of 10
The additional experiments on scikit-learn and Toxicology use only seed 4 and one task order. The paper should acknowledge that these results are preliminary and do not have the same statistical strength as the main results.
What would settle it
A replication on a different model (e.g., Llama 3 70B) or with different ICL policies (e.g., summarization-based retrieval instead of raw replay) that shows ICL no longer provides the largest standalone gain, or that GRPO does not improve over ICL alone, would falsify the paper's central claim.
from Pith's summary of the paper
How the review went
5 passes by deepseek-v4-flash · 17 Nov 2026
- Reader holds up
- Skeptic concern withdrawn: Pith's reviewer misread the paper
- Referee 10 minor
- Check against the paper kept 0 of 5 objections
- Editor wrote the verdict above
Not run on this paper: the simulated authors' defence.
Pith's passes raised 5 objections. A check against the paper's text kept 0 and withdrew 5 (5 Pith's reviewer misread the paper).
The review in full
The review in brief, machine-written
This paper asks a practical question: when an LLM agent works repeatedly in the same environment (a code repo, a database, an app suite), should it adapt by keeping experience in its context, by updating its weights, or both? The authors run a controlled comparison of 14 mechanisms on 109 tasks from four benchmarks, using one model (Qwen3.5-9B) and a shared online protocol where each task is scored before the agent adapts to it.
The headline result is clear: in-context learning (ICL) provides the largest standalone gain, raising mean success from 28.8% to 49.4%. Adding GRPO (a policy-gradient method) to ICL yields the highest overall success at 51.6%. The behavioral analyses are the most interesting part: ICL reduces environment lookups and shortens attempts, consistent with reusing knowledge of shared structure, but it also skips task-specific verification. Adding GRPO helps restore verification while preserving the efficiency gains. SDPO (self-distillation) has mixed effects.
The paper is thorough and honest about its limitations. It studies one model, specific hyperparameters, and raw-trajectory ICL policies. The SDPO family gets three rounds per task versus two for GRPO and ICL, a confound the authors acknowledge. The behavioral observations are correlational, not causal. These are real but bounded weaknesses; the paper does not overclaim.
Who should read this? Anyone working on continual learning for LLM agents, or on combining in-context and weight-based adaptation. The framework is extensible and the results provide a useful baseline for future work. The paper deserves a serious referee; the main questions for review are whether the conclusions hold under different ICL policies (e.g., with summarization or retrieval) and whether the GRPO advantage persists with more rounds or different models.
The stress test's concern
Withdrawn: Pith's reviewer misread the paper.
The paper says
“Raw-trajectory replay lets us study context adaptation without introducing additional summarization, retrieval, or judgment procedures, while keeping the source of experience consistent across mechanisms.”
The referee's report
Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 4 of its 4 major objections did not survive that check.
Pith's reading of the paper (a machine's summary, not the authors')
The paper investigates how LLM agents should adapt at test time when operating repeatedly in the same environment. It compares two adaptation channels: retaining experience in context (in-context learning, ICL) and updating model weights via reinforcement learning (GRPO) or self-distillation (SDPO). Using a shared online protocol across four benchmarks, the authors find that ICL alone raises mean success from 28.8% to 49.4%, while adding GRPO to ICL achieves the highest mean success of 51.6%. Behavioral analyses show that ICL reduces redundant exploration but can skip task-specific verification; adding GRPO helps restore verification while preserving efficiency gains.
What carries the argument. The central comparison is between two adaptation channels: in-context learning (ICL), which retains prior attempts as conversation history in the context window, and weight updates via GRPO (reinforcement learning with verifiable rewards) or SDPO (self-distillation). The framework uses a shared online protocol where agents encounter matched task streams, are evaluated on each task before adapting to it, and learn only from their own attempts and environment feedback. Four ICL policies (crossing FIFO/Prefix retention with All/Successful admission) and two weight-update rules are tested alone and in combination.
Core claim. The paper's central claim is that in-context learning (ICL) provides the largest standalone gain for test-time adaptation of LLM agents, and adding reinforcement learning with verifiable rewards (GRPO) to ICL yields the highest overall success rate. Across 109 tasks from four benchmarks, ICL raises mean success from 28.8% to 49.4%, and ICL+GRPO achieves 51.6%. The authors argue that the two channels are complementary: ICL reuses environment knowledge to reduce exploration, while GRPO refines behavior by restoring task-specific inspection and verification that ICL tends to skip.
The premise it rests on. The results depend on the specific implementations of ICL policies (raw-trajectory replay with FIFO/Prefix retention and All/Successful admission) and weight updates (GRPO and SDPO with LoRA adapters and fixed hyperparameters), and may not generalize to other ICL policies, other models, longer streams, or different weight-update recipes.
If the paper is right (Pith's inferences, not the authors')
If ICL+GRPO consistently outperforms either channel alone, then practical deployments of LLM agents should combine context management with reinforcement learning rather than relying on one channel.
The finding that ICL reduces environment lookups but skips verification suggests that pure ICL may lead to over-reliance on past experience at the expense of checking current task requirements.
The mixed effects of SDPO indicate that self-distillation does not reliably complement ICL, possibly because its self-teacher inherits ICL's limitations.
The result that GRPO restores verification behaviors while preserving ICL's efficiency gains suggests that reinforcement learning can correct undesirable shortcuts learned from context.
The paper's protocol could be extended to study longer streams, where context window pressure might shift the balance between channels further toward weight updates.
The observation that ICL reduces exploration but also verification suggests a potential failure mode: agents may become overconfident in familiar patterns and miss novel task requirements.
The framework could be used to test whether other weight-update methods (e.g., online fine-tuning with different objectives) also complement ICL, or whether the complementarity is specific to GRPO's reward-based learning.
The transfer experiments hint that retained memory can help or hurt across environments, raising questions about when context should be reset or curated for new domains.
Check this review
What was read
The passes were given all 137,630 characters of the paper's LaTeX source.
Fingerprint of that text (SHA-256): 1992089f8afbfba68b4900271fb4fa2553068c4642f9f61d57ea60ff7c705c1b
Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:
PYTHONPATH=. python scripts/review_input_check.py 2611.19700
The passes, in the order they ran
- Reader deepseek-v4-flash · medium effort · $0.012 · instructions of the Reader, fingerprint f9ead7300382…
- Skeptic deepseek-v4-flash · medium effort · $0.0044 · instructions of the Skeptic, fingerprint 8f88c0f61488…
- Referee deepseek-v4-flash · medium effort · $0.0047 · instructions of the Referee, fingerprint a852d99aea0d…
- Check against the paper deepseek-v4-flash · medium effort · $0.0048 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
- Editor deepseek-v4-flash · medium effort · $0.0043 · instructions of the Editor, fingerprint e7211f6a3d27…
- Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0046 · instructions of the Summary for the page, fingerprint e4f56bf568e7…
The passes cost $0.035 in model calls together.
Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.0ef1a750a3e3
Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.
curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum
prints the fingerprint of the reader's instructions. The same works for every pass.
The receipt
The review's fingerprint was taken on 17 Nov 2026. The review you are reading still matches it.
Fingerprint of the review (SHA-256): bf0918f7da4b85785a9425cc02c0c43059cde6165608c8a776f660b5aaadfecc
curl -s 'https://pith.science/api/v1/verdicts/2611.19700?form=canonical' | sha256sum
prints the fingerprint of the review as it stands now. The rule, in four steps.
A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, from the paper's LaTeX source.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
How Pith checks papers · The full audit of this paper · Review history (2 versions)
The paper
Abstract
LLM agents operating repeatedly in the same environment, such as a code repository, a database, or an application suite, encounter tasks that share latent structure. Experience can reveal this structure, allowing agents to reuse environment knowledge and learn from environmental feedback. Agents can adapt at test-time using this experience through two channels: retaining it in context and updating their weights. These channels are usually studied separately, leaving unclear what each contributes and when they complement each other. We compare in-context learning (ICL), reinforcement learning with verifiable rewards using GRPO, and self-distillation using SDPO, alone and in combination, using a shared online protocol and modular framework. Agents encounter matched task streams, are evaluated on each task before adapting to it, and learn only from their own attempts and environment feedback. We evaluate 14 adaptation mechanisms and a stateless baseline with Qwen3.5-9B on 109 tasks from four environments drawn from SWE-bench Verified, AppWorld, ScienceWorld, and BIRD-Critic-SQLite, using five task orders per environment. ICL provides the largest standalone gain: its best policy raises mean success across environments from 28.8% to 49.4%. Adding GRPO to this policy achieves the highest mean success, 51.6%. While adding GRPO improves the overall mean under every ICL policy, SDPO has mixed effects. Trace analyses show fewer lookups and shorter attempts under ICL, consistent with reusing knowledge of the shared structure. However, ICL also frequently skips inspection and verification for the current task. Adding GRPO to ICL helps restore these behaviors while preserving its reduced environment lookups. These observations suggest that reinforcement learning with verifiable rewards can refine how agents use retained experience, motivating the joint study of context management and weight updates.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-17
The authors' main claim, as the classifier picked it out 0.91
“These observations suggest that reinforcement learning with verifiable rewards can refine how agents use retained experience, motivating the joint study of context management and weight updates.”
How the main claim is stated: hedged possibility 0.99
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.38 · human participants 0.03 · controlled comparison 0.97 · randomized assignment 0.04 · states quantitative result 0.99 · compares with prior results 0.92 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| LLM agents operating repeatedly in the same environment, such as a code repository, a database, or an application suite, encounter tasks that share latent structure. | background 1.00 | 0.08 | 0.02 | asserted 1.00 |
| Experience can reveal this structure, allowing agents to reuse environment knowledge and learn from environmental feedback. | background 1.00 | 0.07 | 0.02 | hedged possibility 0.68 |
| Agents can adapt at test-time using this experience through two channels: retaining it in context and updating their weights. | background 0.95 | 0.06 | 0.04 | hedged possibility 0.50 |
| These channels are usually studied separately, leaving unclear what each contributes and when they complement each other. | gap 1.00 | 0.06 | 0.02 | asserted 0.98 |
| We compare in-context learning (ICL), reinforcement learning with verifiable rewards using GRPO, and self-distillation using SDPO, alone and in combination, using a shared online protocol and modular framework. | method 0.68 | 0.93 | 0.04 | asserted 0.96 |
| Agents encounter matched task streams, are evaluated on each task before adapting to it, and learn only from their own attempts and environment feedback. | method 0.99 | 0.83 | 0.02 | asserted 0.99 |
| We evaluate 14 adaptation mechanisms and a stateless baseline with Qwen3.5-9B on 109 tasks from four environments drawn from SWE-bench Verified, AppWorld, ScienceWorld, and BIRD-Critic-SQLite, using five task orders per environment. | method 0.99 | 0.96 | 0.02 | asserted 0.97 |
| ICL provides the largest standalone gain: its best policy raises mean success across environments from 28.8% to 49.4%. | result 1.00 | 0.95 | 0.18 | asserted 1.00 |
| Adding GRPO to this policy achieves the highest mean success, 51.6%. | result 1.00 | 0.95 | 0.22 | asserted 1.00 |
| While adding GRPO improves the overall mean under every ICL policy, SDPO has mixed effects. | result 1.00 | 0.93 | 0.12 | asserted 1.00 |
| Trace analyses show fewer lookups and shorter attempts under ICL, consistent with reusing knowledge of the shared structure. | result 0.81 | 0.93 | 0.05 | hedged possibility 0.60 |
| However, ICL also frequently skips inspection and verification for the current task. | result 0.94 | 0.89 | 0.03 | asserted 0.99 |
| Adding GRPO to ICL helps restore these behaviors while preserving its reduced environment lookups. | result 0.79 | 0.91 | 0.20 | asserted 0.98 |
| These observations suggest that reinforcement learning with verifiable rewards can refine how agents use retained experience, motivating the joint study of context management and weight updates. | interpretation 1.00 | 0.71 | 0.69 | hedged possibility 0.99 |
What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.98
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-17: 1 request, 10,143 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (23)
Figures are the authors', reproduced from the paper.
References (45)
- [1] Additional environments and evaluations of final agents are described separately in Appendices E and F. A.1 Framework interfaces and adaptation mechanisms A benchmark registers four contracts: aworlddefines the tasks and shared interaction rules; astreamorders those tasks; ata... on Pith
- [2] URLhttp://arxiv.org/abs/2411.07279. arXiv:2411.07279 [cs]. Marc-Antoine Allard, Arnaud Teinturier, Victor Xing, and Gautier Viaud. Experiential Reflective Learning for Self- Improving LLM Agents, March on PithPith's reviewarXiv
- [3] Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE- bench: CanLanguageModelsResolveReal-WorldGitHubIssues?, November2024. URLhttp://arxiv.org/abs/2310.06770. arXiv:2310.06770 [cs]. Haechan Kim, Yoonho Lee, Gisang Lee,... on PithPith's reviewarXiv
- [4] URLhttp://arxiv.org/abs/2606.05661. arXiv:2606.05661 [cs.AI]. A. P. Dawid. Present position and potential developments: Some personal views: Statistical theory: The prequential approach.Journal of the Royal Statistical Society. Series A (General), 147(2):278–290, on PithPith's reviewarXiv
- [6] URLhttp://arxiv.org/abs/2603.01761. arXiv:2603.01761 [cs.LG]. Kion Fallah, Silen Naihin, Barak Widawsky, and Qingqing Mao. CLaaS: Continual learning as a service for sample efficient online learning, June on PithPith's reviewarXiv
- [7] URLhttp://arxiv.org/abs/2606.05559. arXiv:2606.05559 [cs.LG]. João Gama, Raquel Sebastião, and Pedro Pereira Rodrigues. On evaluating stream learning algorithms.Machine Learning, 90(3):317–346, on PithPith's reviewarXiv
- [9] URL http://arxiv.org/abs/2607.07847. arXiv:2607.07847 [cs.LG]. Zexue He, Yu Wang, Churan Zhi, Yuanzhe Hu, Tzu-Ping Chen, Lang Yin, Ze Chen, Tong Arthur Wu, Siru Ouyang, Zihan Wang, Jiaxin Pei, Julian McAuley, Yejin Choi, and Alex Pentland. MemoryArena: Benchmarking Agent Memor... on PithPith's reviewarXiv
- [10] URLhttp://arxiv.org/abs/2602.16313. arXiv:2602.16313 [cs]. Prannay Hebbar, Yogendra Manawat, Samuel Verboomen, Alesia Ivanova, Selvam Palanimalai, Kunal Bhatia, and Vignesh Baskaran. SIA: Self Improving AI with Harness & Weight Updates, May on PithPith's reviewarXiv
- [11] URLhttp://arxiv.org/abs/2605. 27276. arXiv:2605.27276 [cs.AI]. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, on PithPith's reviewarXiv
- [12] URLhttp://arxiv.org/abs/2505.20633. arXiv:2505.20633 [cs]. Qisheng Hu, Quanyu Long, and Wenya Wang. When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agents, April on PithPith's reviewarXiv
- [13] URLhttp://arxiv.org/abs/2604.27003. arXiv:2604.27003 [cs.LG]. Jonas Hübotter, Leander Diaz-Bone, Ido Hakimi, Andreas Krause, and Moritz Hardt. Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning, October on PithPith's reviewarXiv
- [14] URLhttp://arxiv.org/abs/2510.04786. arXiv:2510.04786 [cs]. 16 Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and Andreas Krause. Reinforcement Learning via Self- Di... on PitharXiv
- [15] URLhttp://arxiv.org/abs/2512.16301. arXiv:2512.16301 [cs] version: on PitharXiv
- [17] URLhttp://arxiv.org/abs/2609.00196. arXiv:2609.00196 [cs]. Ben Krause, Emmanuel Kahembwe, Iain Murray, and Steve Renals. Dynamic evaluation of neural sequence models. InProceedings of the 35th International Conference on Machine Learning, volume 80 ofProceedings of Machine Lea... on PithPith's reviewarXiv
- [18] URLhttp://arxiv.org/abs/2505.00661. arXiv:2505.00661 [cs.CL]. Jinyang Li, Xiaolong Li, Ge Qu, Per Jacobsson, Bowen Qin, Binyuan Hui, Shuzheng Si, Nan Huo, Xiaohan Xu, Yue Zhang, Ziwei Tang, Yuanshuai Li, Florensia Widjaja, Xintong Zhu, Feige Zhou, Yongfeng Huang, Yannis Papako... on PitharXiv
- [19] URLhttps://arxiv.org/abs/2506.18951. arXiv:2506.18951 [cs.DB]; NeurIPS on PitharXiv
- [20] URLhttps://arxiv.org/abs/2602.23008v2. Haodong Lu and Dong Gong. ASCENT: Online Test-Time Training of Long-Horizon Agents via Self-Distillation of Verified Experience, October on PitharXiv
- [21] URLhttp://arxiv.org/abs/2610.05303. arXiv:2610.05303 [cs]. XufangLuo, YugeZhang, ZhiyuanHe, ZilongWang, SiyunZhao, DongshengLi, LunaK.Qiu, andYuqingYang. Agent Lightning: Train ANY AI agents with reinforcement learning, August 2025a. URLhttps://arxiv.org/abs/2508.03680. arXiv:... on PithPith's reviewarXiv
- [23] URL http://arxiv.org/abs/2606.04815. arXiv:2606.04815 [cs]. Siru Ouyang, Jun Yan, I.-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and To... on PithPith's reviewarXiv
- [24] URLhttp: //arxiv.org/abs/2509.25140. arXiv:2509.25140 [cs]. 17 Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative Agents: Interactive Simulacra of Human Behavior. InProceedings of the 36th Annual ACM Sympos... on PithPith's reviewarXiv
show 25 more
- [27] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. HybridFlow: A flexible and efficient RLHF framework.arXiv preprint arXiv:2409.19256, on PithPith's reviewarXiv
- [28] doi: 10.48550/arXiv.2409.19256. URLhttps://arxiv.org/abs/2409.19256. Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, Zifeng Wang, Sayna Ebrahimi, and Hao Wang. Continual Learning of Large Language Models: A Comprehensive Survey, June on PithPith's reviewarXivdoi:10.48550/arxiv.2409.19256
- [29] URL http://arxiv.org/abs/2404.16789. arXiv:2404.16789 [cs] version: on PithPith's reviewarXiv
- [30] URLhttps://arxiv.org/abs/2303.11366. David Silver and Richard S. Sutton. Welcome to the era of experience, on PithPith's reviewarXiv
- [32] URLhttp://arxiv.org/abs/2607.11012. arXiv:2607.11012 [cs.CL]. Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test-time training with self- supervision for generalization under distribution shifts. InProceedings of the 37th International Conf... on PithPith's reviewarXiv
- [33] URLhttps://arxiv.org/abs/2512.23675. Rishabh Tiwari, Kusha Sareen, Lakshya A. Agrawal, Joseph E. Gonzalez, Matei Zaharia, Kurt Keutzer, Inderjit S. Dhillon, Rishabh Agarwal, and Devvrit Khatri. Learning, Fast and Slow: Towards LLMs That Adapt Continually, May on PitharXiv
- [34] URLhttp://arxiv.org/abs/2605.12484. arXiv:2605.12484 [cs.LG]. Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, andNiranjanBalasubramanian. AppWorld: AControllableWorldofAppsandPeopleforBenchmarking Interactive... on PithPith's reviewarXiv
- [35] URLhttp://arxiv.org/abs/2407.18901. arXiv:2407.18901 [cs]. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandku- mar. Voyager: An open-ended embodied agent with large language models, on PithPith's reviewarXiv
- [36] URLhttps://arxiv.org/abs/2305.16291. Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. ScienceWorld: Is your agent smarter than a 5th grader? InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, on PithPith's reviewarXiv
- [37] URL http://arxiv.org/abs/2409.07429. arXiv:2409.07429 [cs]. Tianxin Wei, Noveen Sachdeva, Benjamin Coleman, Zhankui He, Yuanchen Bei, Xuying Ning, Mengting Ai, Yunzhe Li, Jingrui He, Ed H. Chi, Chi Wang, Shuo Chen, Fernando Pereira, Wang-Cheng Kang, and Derek Zhiyuan 18 Cheng.... on PithPith's reviewarXiv
- [38] URL http://arxiv.org/abs/2511.20857. arXiv:2511.20857 [cs.CL]. Cheng-Kuang Wu, Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, and Hung-yi Lee. StreamBench: Towards Benchmarking Continuous Improvement of Language Agents, October on PithPith's reviewarXiv
- [39] URLhttp://arxiv.org/abs/2406.08747. arXiv:2406.08747 [cs]. Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, and Botian Shi. EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle. In... on PithPith's reviewarXiv
- [40] Tianzhu Ye, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, and Furu Wei. Online Experiential Learning for Language Models, June 2026a. URLhttp://arxiv.org/abs/2603.16856. arXiv:2603.16856 [cs.CL]. Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, and Furu Wei. On-Policy Context Disti... on PithPith's reviewarXiv
- [41] URLhttps: //arxiv.org/abs/2601.16175v2. Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic Context Engineer- ing: Evolving Contexts for... on PithPith's reviewarXiv
- [42] URLhttps://arxiv.org/abs/2510.04618v3. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners, on PithPith's reviewarXiv
- [43] URLhttps://arxiv.org/abs/2308.10144. Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, and Jun Wang. Memento: Fine-tuning LLM Agents without Fine-tuning LLMs, August on PithPith's reviewarXiv
- [44] URL http://arxiv.org/abs/2508.16153. arXiv:2508.16153 [cs]. Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, Biqing Qi, Youbang Sun, Zhiyuan Ma, Lifan Yuan, Ning Ding, and Bowen Zhou. TTRL: Test-time reinforce... on PithPith's reviewarXiv
- [1984] doi: 10.2307/2981683. Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov, Lucas Caccia, Antonio Carta, Laurent Charlin, Barbara Hammer, Tyler L. Hayes, Timm Hess, Christopher Kanan, Dhireesha Kudithipudi, Xialei Liu, Vincenzo Lomonaco, Jorge Mendez-Mendez, Darshan Patil, A... on Pithdoi:10.2307/2981683
- [2013] doi: 10.1007/s10994-012-5320-9. Moritz Hardt and Yu Sun. Test-time training on nearest neighbors for large language models. InInternational Conference on Learning Representations, on Pithdoi:10.1007/s10994-012-5320-9
- [2017] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, on PithPith's reviewarXiv
- [2022] URLhttps://arxiv.org/abs/ 2209.15189. arXiv:2209.15189 [cs.CL]. Dilara Soylu, Christopher Potts, and Omar Khattab. Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, on PithPith's reviewarXiv
- [2023] Association for Computing Machinery. ISBN 979-8-4007-0132-0. doi: 10.1145/3586183.3606763. URLhttps://dl.acm.org/doi/10.1145/ 3586183.3606763. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint ... on PithPith's reviewarXiv
- [2024] URLhttps://proceedings.neurips.cc/paper_files/paper/2024/hash/ 877b40688e330a0e2a3fc24084208dfa-Abstract-Datasets_and_Benchmarks_Track.html. Bo Mao, Jie Zhou, Yutao Yang, Xin Li, Xian Wei, Qin Chen, Xingjiao Wu, and Liang He. Learning While Acting: A Skill-Enhanced Test-Time C... on Pith
- [2025] URLhttp://arxiv.org/abs/2510.07841. arXiv:2510.07841 [cs]. Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoic... on PitharXiv
- [2026] URLhttp://arxiv.org/abs/2603.24639. arXiv:2603.24639 [cs] version: on PitharXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment