Pith.

Machine Learning · arXiv:2611.01747 v1 · submitted 1 Nov 2026 · 104 references

Evolving Memory, Learning to Meta-Control: Test-Time Adaptation for Experience-augmented LLM Agents

Zeyuan Zhao, Yuanyuan Shi, Peng Yu, Lingyue Fu, Shao Zhang, Ying Wen

The paper says

“Self-evolving agents accumulate experience from task streams, but existing frameworks typically rely on fixed pipelines to retrieve and utilize this experience, neglecting when and how to deploy it.”

more of the abstract

We introduce Online Meta-Control of Experience (OMCE), a framework that updates experience memory and learns a meta-control policy from task feedback during test-time adaptation. OMCE employs a lightweight contextual-bandit meta-controller that uses compact retrieval signals to select among four cognitive actions: DIRECT, EPISODE, GUIDE, and DISTILL. This routing mechanism operationalizes three coupled decisions: whether to invoke experience, which memory format to utilize, and when to consolidate new guidance. OMCE evolves online, leveraging task feedback to update the memory bank and refine the routing policy via contextual bandit updates. Experiments on four online adaptation benchmarks across four backbones show that OMCE improves first-attempt success by up to 20.5 percentage points over the strongest compared baseline, while reducing per-task token use by a factor of 5.7 to 62.5 relative to ACE with GPT-5.2. Controlled ablations further support the benefits of learned routing and continued routing policy adaptation.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 4 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. Solid paper: clean formulation of a real problem (learned routing over experience), backed by extensive experiments and a theoretical bound. The meta-state assumption is a soft spot but not a fatal one. This line is wrong

2 major objections · 3 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 104 references say.

The paper says, at §3.2, Eq. (1) and (9); meta-state definition

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The meta-state is a hand-crafted 9-dimensional vector that condenses retrieval scores, quality scores, and outcome diversity. It never reads the actual text of retrieved memories. The paper does not ablate this representation (e.g., by comparing to a version that includes text embeddings or a learned representation).

more of objection 1

While the empirical results are strong and suggest that the summary statistics suffice for the tasks tested, the claim that the controller learns 'when and how to use experience' implicitly assumes that all task-relevant variation for routing is captured by these nine numbers. A direct ablation or a discussion of why text content is unnecessary (or why the summary is sufficient) would strengthen the paper. I consider this a missing experiment, but not a fatal flaw given the overall empirical support.

The paper says, at §4.1 and Appendix G.4

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.91):

“The 100 selected BigCodeBench tasks remain part of the final evaluation stream, so the BigCodeBench result contains a limited overlap between hyperpa- rameter selection and evaluation.”

Pith says: Objection 2 of 2 · major

The hyperparameter configuration (Table 5) was selected on a 100-task subset of BigCodeBench, and those same 100 tasks remain in the final evaluation stream. The authors acknowledge this overlap (Appendix G.4: 'the BigCodeBench result contains a limited overlap between hyperparameter selection and evaluation').

more of objection 2

For a single configuration shared across all benchmarks and backbones, this is acceptable, but the paper should quantify the potential for optimistic bias on BigCodeBench, e.g., by reporting the performance on the held-out portion separately. A brief sentence clarifying how much of the 1,140-task stream is the 'selection' subset would suffice.

3 minor objections

The paper says, at §4.3, Table 3

Pith says: minor objection 1 of 3

The controlled analysis (shared memory snapshot) uses only GPT-5.2. The authors note this limitation in the main text (line: 'Our controlled comparisons of memory use and controller updates focus on GPT-5.2, while the routing comparison spans all four backbones'). Adding the same analysis on at least one more backbone (e.g., DeepSeek-V3.2) would strengthen the generality claim. As is, this is a minor completeness concern.

The paper says, at Appendix E

Pith says: minor objection 2 of 3

The theoretical regret bound (Theorem E.2) assumes the stationary regime (γ=1) and analyzes an 'optimistic instantiation' (LinUCB-style) rather than the Thompson Sampling used in experiments. While the connection is plausible, the paper should state explicitly that the bound does not directly cover the experimental setting (non-stationary with TS) and that it serves as an intuition for why the linear bandit can learn. The current Remark acknowledges this in part but could be clearer.

show 1 more

The paper says, at Figure 3 and Figure 5

Pith says: minor objection 3 of 3

The x-axis label in the performance–efficiency plots is 'One-shot Success Rate (OSR)' but the markers include both OSR and CSR for some methods. Please ensure the legend and axis labels match the reported metrics. For example, Reflexion is plotted only on CSR, which is fine, but the figure caption should clarify what is shown.

What would settle it

Augment the meta-state with the mean embedding of the top-M retrieved entries (e.g., from the same text-embedding-3-large used for retrieval), adding two extra dimensions (episodic mean and guidance mean). Compare OMCE-TS with this augmented state against the original 9-d version on all four benchmarks with GPT-5.2 (one run each, as in the paper). If the augmented version does not improve OSR by more than 1 percentage point on any benchmark, the 9-d state is sufficient. If it does improve, the paper's representation is a limiting factor and the central claim should be qualified.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 4 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 41 s · 29,348 tokens read, 3,687 written
  3. Editor wrote the verdict above 11 s · 30,261 tokens read, 976 written
  4. Referee minor revision: 2 major objections, 3 minor 21 s · 29,760 tokens read, 2,013 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This is a solid, well-executed paper. The core idea—learning when and how to use accumulated experience via a lightweight contextual bandit—is genuinely new and practically motivated. The authors formalize three coupled decisions (invocation, form, consolidation) that prior work either hardwires or ignores. The experiments are thorough: four benchmarks, four backbones, multiple baselines, and controlled ablations that isolate the routing policy from memory content. The token-efficiency gains are striking. The theoretical regret bound, while for a stationary simplification, adds rigor.

The main soft spot is the meta-state representation. The 9-dimensional vector condenses retrieval scores, quality scores, and outcome diversity, but never reads the actual memory text. The authors do not test whether a richer representation (e.g., including embeddings of retrieved content) would further improve performance. This is the stress-test concern, and it is legitimate: the meta-state could conflate semantically different memories with similar scores. However, the empirical results are strong enough that this assumption, while untested, has not obviously bottlenecked performance. The paper would be stronger with an ablation that adds text embeddings, but the absence is not a dealbreaker—the current representation already works well across diverse tasks.

Other minor issues: hyperparameter selection overlaps with BigCodeBench (acknowledged), no error bars (a real limitation but common in LLM API experiments), and the theoretical analysis assumes a stationary regime (authors note this). These are proportionately small concerns.

This paper is for researchers working on self-evolving agents, memory-augmented LLMs, and meta-cognitive control. It deserves a serious referee. The central claim holds up, the evidence is reproducible (code presumably released), and the framing is likely to influence subsequent work.

The strongest objection

The central claim that OMCE learns when and how to use experience depends critically on the meta-state representation (Section 3.2, Equation 1 and 9). The meta-state is a hand-crafted 9-dimensional vector that condenses retrieval scores, outcome-based quality scores, and outcome-type diversity. Crucially, it never reads the actual text of retrieved episodic records or guidance entries. The linear bandit therefore bases its decision on summary statistics that may conflate semantically different memories that happen to have similar scores. For example, two episodic records with the same retrieval score and outcome type could contain very different lessons; the meta-state treats them identically. The paper does not provide an ablation that validates this compression (e.g., by comparing to a version that includes retrieved-text embeddings or a learned representation). While the empirical results are strong, the assumption that all task-relevant variation is captured by these nine numbers remains unexamined. If the meta-state is insufficient, the bandit's learning is bottlenecked, and the observed gains might be further improvable with a richer representation, or might degrade on tasks where score-alone signals are misleading.

The reader correctly identified the sufficiency of the meta-state as the weakest assumption. My concern sharpens this: the meta-state discards semantic content entirely, which is a design choice that could be a bottleneck for learning the optimal routing policy. However, the paper provides extensive empirical evidence that the learned routing already outperforms fixed policies and a hand-crafted heuristic across four benchmarks and four backbones. The absence of a direct validation of the representation is a gap, but the experimental results partly address the concern by showing that the representation works well in practice. Therefore, the reader's ACCEPT verdict is still justified; the concern suggests a natural future direction rather than a fundamental flaw. I concur with the reader that the paper is sound and the central claim is supported. The concrete test would strengthen the paper by ruling out a plausible alternative explanation, but its absence does not invalidate the current results.

The referee's report

If the claimed results hold, this work makes a meaningful contribution by reframing experience-augmented reasoning as a learnable meta-control problem. The empirical evidence is extensive: four diverse benchmarks, four backbones, multiple baselines (including fixed policies, heuristics, and shared-memory controls), and controlled ablations separate the contribution of routing from memory content. The theoretical regret bound (Appendix E, for a stationary simplified version) adds rigor. The performance–token trade-off is particularly compelling, showing order-of-magnitude efficiency gains relative to ACE while maintaining or improving accuracy. The paper identifies a genuine bottleneck in current self-evolving agents and proposes a lightweight, practical solution.

Pith's reading of the paper (a machine's summary, not the authors')

Self-evolving LLM agents accumulate experience from task streams, but existing frameworks retrieve and inject that experience unconditionally, wasting tokens and sometimes harming performance. This paper introduces Online Meta-Control of Experience (OMCE), a framework that treats the decision of whether and how to use experience as a learnable meta-control problem. OMCE uses a lightweight contextual bandit that reads a nine-dimensional summary of retrieval quality and outcome diversity to choose among four actions: direct execution (no memory), episodic reuse, guidance reuse, or distillation of new guidance. The bandit policy is updated online from task reward signals, allowing it to adapt as memory evolves. Experiments across four benchmarks and four backbone LLMs show that OMCE improves first-attempt success by up to 20.5 percentage points over the strongest baseline and reduces per-task token consumption by a factor of 5.7 to 62.5 relative to the playbook-based ACE method.

What carries the argument. The key machinery is a contextual-bandit meta-controller with linear reward models. The meta-state is a nine-dimensional vector composed of: mean retrieval scores from episodic and guidance memory (relevance), mean quality scores and their relevance-weighted counterparts (quality), and the balance and normalized entropy of outcome types among retrieved episodic records (outcome diversity). The controller uses either Thompson Sampling or LinUCB to select among four cognitive actions. The bandit statistics are updated online with a forgetting factor after each task, using the observed reward. The entire meta-controller operates on small matrices and requires no additional LLM calls beyond the standard task execution and memory evolution.

Core claim. The paper argues that the bottleneck in self-evolving agents is not accumulating richer experience but learning when and how to deploy it. The central claim is that a contextual-bandit meta-controller operating on compact meta-state features—relevance, quality, and outcome diversity of retrieved memory—can learn to select among DIRECT, EPISODE, GUIDE, and DISTILL actions, and that this learned routing improves first-attempt success across diverse benchmarks and backbone models while drastically reducing token cost compared to always-inject baselines. The authors demonstrate that no fixed experience-use strategy is consistently best, and that the learned controller outperforms both fixed policies and a hand-crafted heuristic rule. Ablations confirm that the gains stem from the routing policy, not merely the memory content, and that continued policy adaptation as memory evolves is necessary for best performance.

The premise it rests on. The paper assumes that a nine-dimensional vector of retrieval scores, quality scores, and outcome diversity captures all information needed to decide whether and how to use retrieved experience, without the controller ever reading the actual content of the memories.

If the paper is right (Pith's inferences, not the authors')

Self-evolving agents can be designed with a lightweight meta-controller that learns to gate experience, improving efficiency and robustness across different task types.

The meta-control approach suggests that the value of experience is not uniform; agents should dynamically choose between raw cases, distilled principles, and direct execution rather than always injecting memory.

The substantial token savings (5.7–62.5×) imply that learned routing can make test-time adaptation practical for cost-sensitive deployments where API call budgets are limited.

The finding that indiscriminate retrieval can hurt performance (worse than no memory) underscores the need for selective invocation, and that routing policy must be learned, not hard-coded.

The success of a linear model on nine hand-crafted features suggests that effective meta-control does not require full access to memory content, opening the door to even simpler controllers.

The nine-dimensional meta-state may encode sufficient structure for the four benchmarks tested, but could miss information (e.g., semantic novelty or task difficulty) that matters on other domains; testing on a broader distribution is needed.

The paper assumes linear payoff functions; richer nonlinear models or learned embeddings might improve performance on streams with complex dependencies, though at the cost of interpretability and compute.

The forgetting factor (γ=0.98) provides a simple mechanism for non-stationarity; a formal analysis of catastrophic forgetting in the meta-controller under abrupt distribution shifts could reveal failure modes.

The four action set is hand-designed; future work could extend it to actions like 'ignore irrelevant memory' or 'retrieve additional evidence before deciding', potentially yielding further gains.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Self-evolving agents accumulate experience from task streams, but existing frameworks typically rely on fixed pipelines to retrieve and utilize this experience, neglecting when and how to deploy it. We introduce Online Meta-Control of Experience (OMCE), a framework that updates experience memory and learns a meta-control policy from task feedback during test-time adaptation. OMCE employs a lightweight contextual-bandit meta-controller that uses compact retrieval signals to select among four cognitive actions: DIRECT, EPISODE, GUIDE, and DISTILL. This routing mechanism operationalizes three coupled decisions: whether to invoke experience, which memory format to utilize, and when to consolidate new guidance. OMCE evolves online, leveraging task feedback to update the memory bank and refine the routing policy via contextual bandit updates. Experiments on four online adaptation benchmarks across four backbones show that OMCE improves first-attempt success by up to 20.5 percentage points over the strongest compared baseline, while reducing per-task token use by a factor of 5.7 to 62.5 relative to ACE with GPT-5.2. Controlled ablations further support the benefits of learned routing and continued routing policy adaptation.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-03

The authors' main claim, as the classifier picked it out 1.00

“Experiments on four online adaptation benchmarks across four backbones show that OMCE improves first-attempt success by up to 20.5 percentage points over the strongest compared baseline, while reducing per-task token use by a factor of 5.7 to 62.5 relative to ACE with GPT-5.2.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.08 · human participants 0.04 · controlled comparison 0.96 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Self-evolving agents accumulate experience from task streams, but existing frameworks typically rely on fixed pipelines to retrieve and utilize this experience, neglecting when and how to deploy it. gap 0.78 0.06 0.03 asserted 1.00
We introduce Online Meta-Control of Experience (OMCE), a framework that updates experience memory and learns a meta-control policy from task feedback during test-time adaptation. contribution 0.99 0.93 0.14 asserted 1.00
OMCE employs a lightweight contextual-bandit meta-controller that uses compact retrieval signals to select among four cognitive actions: DIRECT, EPISODE, GUIDE, and DISTILL. method 0.99 0.87 0.04 asserted 1.00
This routing mechanism operationalizes three coupled decisions: whether to invoke experience, which memory format to utilize, and when to consolidate new guidance. method 0.99 0.77 0.04 asserted 1.00
OMCE evolves online, leveraging task feedback to update the memory bank and refine the routing policy via contextual bandit updates. method 0.96 0.84 0.08 asserted 1.00
Experiments on four online adaptation benchmarks across four backbones show that OMCE improves first-attempt success by up to 20.5 percentage points over the strongest compared baseline, while reducing per-task token use by a factor of 5.7 to 62.5 relative to ACE with GPT-5.2. result 1.00 0.96 0.76 asserted 1.00
Controlled ablations further support the benefits of learned routing and continued routing policy adaptation. result 0.85 0.93 0.09 asserted 0.93

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-03: 1 request, 5,681 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (5)
Figure 1 of the paper
Figure 1. From Fixed Pipelines to Learned Meta-Control. Up: Existing methods typically follow a fixed routine for using and consolidating experience, even when the retrieved experience does not help the current task. Down: OMCE learns from task feedback when to use experience, whether to draw on past cases or general guidance, and when to distill new guidance. troller regulates whether to invoke experience, which form to use, and when to distill new guid￾ance (
Figure 2 of the paper
Figure 2. Architecture of OMCE. The lightweight Meta-Controller selects a cognitive action specifying whether and which experience to inject into the task prompt. The agent solves the task with the augmented context containing the query and any injected experience. Memory evolution records outcomes in episodic memory, distills guidance before execution when DISTILL is selected, and refines injected guidance afterward using task feedback. Here, Augment injects the selected experience into the task prompt containing the query, following the task prompt template without an additional LLM call.…
Figure 3 of the paper
Figure 3. Performance–efficiency trade-off on four benchmarks using GPT-5.2. The x-axis represents the one-shot success rate, and the y-axis reports the token cost in log scale (average total prompt and output tokens consumed per task). form costs accuracy where it fits poorly: Fixed￾aEPISODE trails OMCE-TS by 11.4 points on Multi￾Challenge, and Fixed-aGUIDE&DISTILL trails it by 10.2 points on BBEH. Which form helps therefore depends on the task; the same reversal appears at the level of individual backbones, where the BBEH ordering of the two fixed policies flips between GPT-5.2 and…
Figure 4 of the paper
Figure 4. Self-correction detection in MultiChallenge. The meta-controller routes the query to GUIDE (aGUIDE) rather than to episodic cases that enforce the user’s outdated constraint. to 3.1 points; on BigCodeBench it instead rises, from 59.2 to 65.4. Continued controller updates improve OSR on all four benchmarks, while their effect on CSR varies across benchmarks. 4.4 Efficiency Analysis: Token–Performance Trade-off We evaluate efficiency using tokens per task, the average total LLM tokens consumed per instance over the full online adaptation pipeline; Section G.3 gives the exact accounting
Figure 5 of the paper
Figure 5. Performance–efficiency trade-off on four benchmarks using GPT-5.2, Deepseek-V3.2, Deepseek-V4-pro,

Figures are the authors', reproduced from the paper.

References (104)
  1. [1] Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards on Pith
  2. [2] Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume = on Pith
  3. [3] Deep learning , author=. 2016 , publisher= on Pith
  4. [4] Reflexion: Language agents with verbal reinforcement learning , author=. Advances in Neural Information Processing Systems , volume= on Pith
  5. [7] Expel: Llm agents are experiential learners , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume= on Pith
  6. [9] React: Synergizing reasoning and acting in language models , author=. The eleventh international conference on learning representations , year= on Pith
  7. [14] Dual-process theories of reasoning: Contemporary issues and developmental applications , author=. Developmental review , volume=. 2011 , publisher= on Pith
  8. [15] Dual-process theories of higher cognition: Advancing the debate , author=. Perspectives on psychological science , volume=. 2013 , publisher= on Pith
  9. [18] Language models are capable of metacognitive monitoring and control of their internal activations , author=. ArXiv , pages= on Pith
  10. [20] Agent workflow memory , author=. arXiv preprint arXiv:2409.07429 , year= on PithPith's reviewarXiv
  11. [22] Retrieval-augmented generation for knowledge-intensive nlp tasks , author=. Advances in neural information processing systems , volume= on Pith
  12. [23] Distinguishing the reflective, algorithmic, and autonomous minds: Is it time for a tri-process theory , author=. In two minds: Dual processes and beyond , pages= on Pith
  13. [24] Robertson, Stephen E and Walker, Steve and Jones, Susan and Hancock-Beaulieu, Micheline M and Gatford, Mike and others , journal= on Pith
  14. [25] Improving language models by retrieving from trillions of tokens , author=. International conference on machine learning , pages=. 2022 , organization= on Pith
  15. [26] MemEvolve: Meta-Evolution of Agent Memory Systems , author=. arXiv preprint arXiv:2512.18746 , year= on PithPith's reviewarXiv
  16. [30] Humanity's last exam , author=. arXiv preprint arXiv:2501.14249 , year= on PithPith's reviewarXiv
  17. [32] LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners , author=. arXiv preprint arXiv:2505.11942 , year= on PithPith's reviewarXiv
  18. [37] Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages= on Pith
  19. [38] Big-bench extra hard , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  20. [40] ^2 -Bench: Evaluating Conversational Agents in a Dual-Control Environment , author=. 2025 , eprint= on Pith
show 80 more
  1. [41] Improved algorithms for linear stochastic bandits , author=. Advances in neural information processing systems , volume= on Pith
  2. [42] Contextual bandits with linear payoff functions , author=. Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages=. 2011 , organization= on Pith
  3. [43] Thompson sampling for contextual bandits with linear payoffs , author=. International conference on machine learning , pages=. 2013 , organization= on Pith
  4. [44] Weighted linear bandits for non-stationary environments , author=. Advances in Neural Information Processing Systems , volume= on Pith
  5. [48] EvoFSM: Controllable Self-Evolution for Deep Research with Finite State Machines , author=. arXiv preprint arXiv:2601.09465 , year= on PitharXiv
  6. [51] Doing more with less: meta-reasoning and meta-learning in humans and machines , author=. Current Opinion in Behavioral Sciences , volume=. 2019 , publisher= on Pith
  7. [52] Metacognition and cognitive monitoring: A new area of cognitive--developmental inquiry. , author=. American psychologist , volume=. 1979 , publisher= on Pith
  8. [53] Challenging big-bench tasks and whether chain-of-thought can solve them , author=. Findings of the Association for Computational Linguistics: ACL 2023 , pages= on Pith
  9. [54] Beyond the imitation game: Quantifying and extrapolating the capabilities of language models , author=. Transactions on machine learning research , year= on Pith
  10. [55] Gpt-4o system card , author=. arXiv preprint arXiv:2410.21276 , year= on PithPith's reviewarXiv
  11. [56] Openai gpt-5 system card , author=. arXiv preprint arXiv:2601.03267 , year= on PithPith's reviewarXiv
  12. [59] DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence , author= on Pith
  13. [60] OpenAI , title =. 2024 , url = on Pith
  14. [61] OpenAI , title =. 2025 , url = on Pith
  15. [62] Google DeepMind , title =. 2025 , url = on Pith
  16. [63] Sentence-bert: Sentence embeddings using siamese bert-networks , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages= on Pith
  17. [64] Dense passage retrieval for open-domain question answering , author=. Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , pages= on Pith
  18. [65] Metamemory: A theoretical framework and new findings , author=. Psychology of learning and motivation , volume=. 1990 , publisher= on Pith
  19. [66] Language models are few-shot learners , author=. Advances in neural information processing systems , volume= on Pith
  20. [67] A survey on large language model based autonomous agents , author=. Frontiers of Computer Science , volume=. 2024 , publisher= on Pith
  21. [68] Generative agents: Interactive simulacra of human behavior , author=. Proceedings of the 36th annual acm symposium on user interface software and technology , pages= on Pith
  22. [69] Moca: Measuring human-language model alignment on causal and moral judgment tasks , author=. Advances in Neural Information Processing Systems , volume= on Pith
  23. [76] Skill-pro: Learning reusable skills from experience via non-parametric ppo for llm agents , author=. arXiv preprint arXiv:2602.01869 , year= on PithPith's reviewarXiv
  24. [77] SoK: Agentic Skills--Beyond Tool Use in LLM Agents , author=. arXiv preprint arXiv:2602.20867 , year= on PithPith's reviewarXiv
  25. [78] Evolver: Self-evolving llm agents through an experience-driven lifecycle , author=. arXiv preprint arXiv:2510.16079 , year= on PithPith's reviewarXiv
  26. [80] Meta-reasoner: Dynamic guidance for optimized inference-time reasoning in large language models , author=. Findings of the Association for Computational Linguistics: ACL 2026 , pages= on Pith
  27. [81] Metascale: Test-time scaling with evolving meta-thoughts , author=. Findings of the Association for Computational Linguistics: ACL 2026 , pages= on Pith
  28. [82] New embedding models and API updates , year = on Pith
  29. [83] Yasin Abbasi-Yadkori, D \'a vid P \'a l, and Csaba Szepesv \'a ri. 2011. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24 on Pith
  30. [84] Shipra Agrawal and Navin Goyal. 2013. Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning, pages 127--135. PMLR on Pith
  31. [85] Marc-Antoine Allard, Arnaud Teinturier, Victor Xing, and Gautier Viaud. 2026. Experiential reflective learning for self-improving llm agents. arXiv preprint arXiv:2603.24639 on PitharXiv
  32. [86] Mingguang Chen, Licheng Wang, and Bo Qu. 2026. Recursive self-improvement in ai: From bounded self-refinement to autonomous research loops. arXiv preprint arXiv:2607.07663 on PithPith's reviewarXiv
  33. [87] Silin Chen, Shaoxin Lin, Xiaodong Gu, Yuling Shi, Heng Lian, Longfei Yun, Dong Chen, Weiguo Sun, Lin Cao, and Qianxiang Wang. 2025. Swe-exp: Experience-driven software issue resolution. arXiv preprint arXiv:2507.23361 on PitharXiv
  34. [88] Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. 2011. Contextual bandits with linear payoff functions. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 208--214. JMLR Workshop and Conference Proceedings on Pith
  35. [89] C Nicol \`o De Sabbata, Theodore R Sumers, Badr AlKhamissi, Antoine Bosselut, and Thomas L Griffiths. 2024. Rational metareasoning for large language models. arXiv preprint arXiv:2410.05563 on PithPith's reviewarXiv
  36. [90] DeepSeek-AI. 2026. Deepseek-v4: Towards highly efficient million-token context intelligence on Pith
  37. [91] Kaustubh Deshpande, Ved Sirdeshmukh, Johannes Baptist Mols, Lifeng Jin, Ed-Yeremai Hernandez-Cardona, Dean Lee, Jeremy Kritz, Willow E Primack, Summer Yue, and Chen Xing. 2025. Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llm... on Pith
  38. [92] Emmanuel Dupoux, Yann LeCun, and Jitendra Malik. 2026. Why ai systems don't learn and what to do about it: Lessons on autonomous learning from cognitive science. arXiv preprint arXiv:2603.15381 on PitharXiv
  39. [93] Jonathan St BT Evans. 2011. Dual-process theories of reasoning: Contemporary issues and developmental applications. Developmental review, 31(2-3):86--102 on Pith
  40. [94] Jonathan St BT Evans and Keith E Stanovich. 2013. Dual-process theories of higher cognition: Advancing the debate. Perspectives on psychological science, 8(3):223--241 on Pith
  41. [95] Jinyuan Fang, Yanwen Peng, Xi Zhang, Yingxu Wang, Xinhao Yi, Guibin Zhang, Yi Xu, Bin Wu, Siwei Liu, Zihao Li, et al. 2025 a . A comprehensive survey of self-evolving ai agents: A new paradigm bridging foundation models and lifelong agentic systems. arXiv preprint arXiv:2508.07407 on PithPith's reviewarXiv
  42. [96] Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. 2025 b . Memp: Exploring agent procedural memory. arXiv preprint arXiv:2508.06433 on PithPith's reviewarXiv
  43. [97] Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, et al. 2025. A survey of self-evolving agents: On path to artificial super intelligence. arXiv preprint arXiv:2507.21046 on PithPith's reviewarXiv
  44. [98] Li Ji-An, Marcelo G Mattar, Hua-Dong Xiong, Marcus K Benna, and Robert C Wilson. 2025. Language models are capable of metacognitive monitoring and control of their internal activations. ArXiv, pages arXiv--2505 on Pith
  45. [99] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770 on PithPith's reviewarXiv
  46. [100] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)... on Pith
  47. [101] Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Yuanzhu Peter Chen, et al. 2025. Big-bench extra hard. In Proceedings of the 63rd Annual Meeting of the Association for ... on Pith
  48. [102] Emre K c man, Robert Ness, Amit Sharma, and Chenhao Tan. 2023. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050 on PithPith's reviewarXiv
  49. [103] Wenjun Li, Dexun Li, Kuicai Dong, Cong Zhang, Hao Zhang, Weiwen Liu, Yasheng Wang, Ruiming Tang, and Yong Liu. 2025. Adaptive tool use in large language models with meta-cognition trigger. arXiv preprint arXiv:2502.12961 on PithPith's reviewarXiv
  50. [104] Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. 2025. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556 on PithPith's reviewarXiv
  51. [105] Jijia Liu, Chao Yu, Jiaxuan Gao, Yuqing Xie, Qingmin Liao, Yi Wu, and Yu Wang. 2023. Llm-powered hierarchical language agent for real-time human-ai coordination. arXiv preprint arXiv:2312.15224 on PithPith's reviewarXiv
  52. [106] Qin Liu, Wenxuan Zhou, Nan Xu, James Y Huang, Fei Wang, Sheng Zhang, Hoifung Poon, and Muhao Chen. 2026. Metascale: Test-time scaling with evolving meta-thoughts. In Findings of the Association for Computational Linguistics: ACL 2026, pages 11828--11842 on Pith
  53. [107] Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, and Peter Clark. 2023. Clin: A continually learning language agent for rapid task adaptation and generalization. arXiv preprint arXiv:2310.10134 on PithPith's reviewarXiv
  54. [108] Allen Nie, Yuhui Zhang, Atharva Shailesh Amdekar, Chris Piech, Tatsunori B Hashimoto, and Tobias Gerstenberg. 2024. Moca: Measuring human-language model alignment on causal and moral judgment tasks. Advances in Neural Information Processing Systems, 36 on Pith
  55. [109] OpenAI. 2024. https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/ Gpt-4o mini: advancing cost-efficient intelligence on Pith
  56. [110] OpenAI . 2024. https://openai.com/index/new-embedding-models-and-api-updates/ New embedding models and api updates . January 25, 2024 on Pith
  57. [111] OpenAI. 2025. https://openai.com/index/gpt-5-system-card-update-gpt-5-2/ Update to gpt-5 system card: Gpt-5.2 on Pith
  58. [112] Siru Ouyang, Jun Yan, I Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T Le, Samira Daruki, Xiangru Tang, et al. 2025. Reasoningbank: Scaling agent self-evolving with reasoning memory. arXiv preprint arXiv:2509.25140 on PithPith's reviewarXiv
  59. [113] Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJC... on Pith
  60. [114] Yoan Russac, Claire Vernade, and Olivier Capp \'e . 2019. Weighted linear bandits for non-stationary environments. Advances in Neural Information Processing Systems, 32 on Pith
  61. [115] John Scoville, Shengzhuang Chen, Yejin Bang, Stefan Winzeck, and Jonathan Richard Schwarz. 2026. Cognitive demand steering for adaptive meta-reasoning in large language models. arXiv preprint arXiv:2608.01319 on PithPith's reviewarXiv
  62. [116] Kulin Shah, Nishanth Dikkala, Xin Wang, and Rina Panigrahy. 2024. Causal language modeling can elicit search and reasoning capabilities on logic puzzles. arXiv preprint arXiv:2409.10502 on PithPith's reviewarXiv
  63. [117] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634--8652 on Pith
  64. [118] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314 on PithPith's reviewarXiv
  65. [119] Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri \`a Garriga-Alonso, et al. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transac... on Pith
  66. [120] Yuan Sui, Yufei He, Tri Cao, Sophia Simeng Han, Yulin Chen, and Bryan Hooi. 2026. Meta-reasoner: Dynamic guidance for optimized inference-time reasoning in large language models. In Findings of the Association for Computational Linguistics: ACL 2026, pages 13268--13286 on Pith
  67. [121] Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al. 2023. Challenging big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computati... on Pith
  68. [122] Xiangru Tang, Tianrui Qin, Tianhao Peng, Ziyang Zhou, Daniel Shao, Tingting Du, Xinming Wei, Peng Xia, Fang Wu, He Zhu, et al. 2025. Agent kb: Leveraging cross-domain experience for agentic problem solving. arXiv preprint arXiv:2507.06229 on PitharXiv
  69. [123] Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. 2026. Kimi k2. 5: Visual agentic intelligence. arXiv preprint arXiv:2602.02276 on PithPith's reviewarXiv
  70. [124] Gladys Tyen, Hassan Mansoor, Peter Chen, Tony Mak, and Victor C a rbune. 2023. Llms cannot find reasoning errors, but can correct them! arXiv preprint arXiv:2311.08516 on PithPith's reviewarXiv
  71. [125] Yuxuan Wan, Tianqing Fang, Zaitang Li, Yintong Huo, Wenxuan Wang, Haitao Mi, Dong Yu, and Michael R Lyu. 2026. Inference-time scaling of verification: Self-evolving deep research agents via test-time rubric-guided verification. arXiv preprint arXiv:2601.15808 on PithPith's reviewarXiv
  72. [126] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 on PithPith's reviewarXiv
  73. [127] Tianxin Wei, Noveen Sachdeva, Benjamin Coleman, Zhankui He, Yuanchen Bei, Xuying Ning, Mengting Ai, Yunzhe Li, Jingrui He, Ed H Chi, et al. 2025. Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory. arXiv preprint arXiv:2511.20857 on PithPith's reviewarXiv
  74. [128] Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, et al. 2024. Livebench: A challenging, contamination-free llm benchmark. arXiv preprint arXiv:2406.19314 on PithPith's reviewarXiv
  75. [129] Chunlong Wu and Zhibo Qu. 2025. Reflection-enhanced meta-optimization integrating textgrad-style prompt optimization with memory-driven self-evolution. arXiv preprint arXiv:2508.18749 on PithPith's reviewarXiv
  76. [130] Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. 2024. Os-copilot: Towards generalist computer agents with self-improvement. corr, abs/2402.07456, 2024. doi: 10.48550. arXiv preprint ARXIV.2402.07456 on PithPith's reviewarXiv
  77. [131] Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110 on PithPith's reviewarXiv
  78. [132] Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Botian Shi, et al. 2025. Learning on the job: An experience-driven self-evolving agent for long-horizon tasks. arXiv preprint arXiv:2510.08002 on PitharXiv
  79. [133] Jenny Zhang, Bingchen Zhao, Wannan Yang, Jakob Foerster, Jeff Clune, Minqi Jiang, Sam Devlin, and Tatiana Shavrina. 2026 a . Hyperagents. arXiv preprint arXiv:2603.19461 on PithPith's reviewarXiv
  80. [134] Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, et al. 2025 a . Agentic context engineering: Evolving contexts for self-improving language models. arXiv preprint arXiv:2510.04618 on PithPith's reviewarXiv

The first 100 of 104 references are listed here. All of them are in this paper's citation record.

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment