CoEvo-Mem: Co-Evolving Retrieval Policy and Memory Bank for LLM Agents

Bowen Ye; Junkai Ma; Wenzhao Li; Xiang Yin; Yongchao Xu; Zhijian Li

REVIEW 4 major objections 4 minor 24 references

CoEvo-Mem: Co-Evolving Retrieval Policy and Memory Bank for LLM Agents

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that long-term LLM agents should co-evolve the retrieval policy and the memory bank, because retrieval determines which memories receive credit while memory updates determine what later retrieval can find.

desk verdict Careful, reproducible system paper; the co-evolution result holds up but the SOTA claim and the un-isolated Q component need attention. read the letter →

arxiv 2608.01739 v1 pith:TG7CLBVW submitted 2026-08-03 cs.AI

Bowen Ye , Yongchao Xu , Zhijian Li , Xiang Yin , Junkai Ma , Wenzhao Li This is my paper · ORCID

classification cs.AI

keywords long-termmemoryLLMagentsretrieval-augmentedgenerationutilitylearningrelationalgraphqueryroutingco-evolvingretrievalandalternatingoptimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-term LLM agents need both to find the right past experiences and to keep those experiences useful. The paper's central claim is that these are not two separate problems: retrieval decides which memories receive feedback, and updates to memory values and relations change what later retrieval finds. It proposes CoEvo-Mem, which closes that loop by coupling a lightweight residual router to a typed relational memory graph, and alternates router updates with memory evolution to keep the pair stable. The paper reports the highest mean scores in all six Table 1 settings, gains up to 7.50 points over the strongest baseline, and first place on LoCoMo free-form QA, supporting the claim that retrieval and memory should co-evolve.

What carries the argument

The carrying object is a typed relational memory graph $G_t = (V_t,E_t)$ whose nodes store experience and a learned utility estimate $Q_t(m)$, coupled to a Q-weighted hybrid retriever. Retrieval fuses dense embedding similarity, BM25 sparse lexical matching, and the utility rank through reciprocal rank fusion, a standard rank-combination formula, with route weights produced by a residual router that corrects a frozen LLM's routing prior. On a memory-phase interaction, an exposed memory receives a temporal-difference-style residual $\delta_t(m) = R_t u_t(m) + \gamma \hat{Q}_t(m^{\mathrm{new}}_t) - Q_t(m)$, where $u_t(m)$ is the attributed contribution score and $\hat{Q}_t(m^{\mathrm{new}}_t)$ inherits the mean utility of the exposed set; credit then propagates across paths of typed edges with relation-specific attenuation, up to depth $D$. The retrieved exposure set $A_t$ is the only meeting point of the two learning processes, and alternating router passes with memory passes is the stabilizer that makes the coupled loop trainable.

What would settle it

Freeze the router, graph structure, and query rewrites from a trained run, replace every learned utility $Q(m)$ with a random permutation of the same values, and re-run the held-out evaluation. If performance does not drop by roughly the margin the paper attributes to memory evolution, the learned utility signal is not carrying the claimed load.

Watch

Extended reading notes

Core claim

The central claim is that memory access and memory evolution reinforce each other and should be optimized jointly. In CoEvo-Mem, the retrieved set is the coupling interface: a residual router, initialized to reproduce a frozen LLM's dense/sparse routing prior, is corrected online by task reward, and the same retrieved memories receive a temporal-difference-style utility update whose bootstrap value comes from the newly created memory. Typed dense, sparse, and temporal relations propagate the resulting outcome credit to neighboring memories, which changes the Q-valued ranking used by the hybrid retriever for later queries. Phase-wise alternating passes freeze one component while the other learns, limiting the non-stationarity that coupled updates create. Across seven benchmarks the paper reports top results, with improvements up to 7.50 points over the strongest baseline.

Load-bearing premise

The system's learned memory scores are reliable enough to rank retrieval even though each new score starts as the average of the scores of the memories that produced it and that same average then feeds the target those scores learn toward; the paper provides no convergence or stability analysis for this self-referential update.

Editorial extensions

If this is right

Separately optimizing retrieval and memory leaves a measurable gap; long-term agent memory systems should treat the two as one coupled learning problem.
Only a small residual router is learned; the answering LLM, the query rewriter, and the memory models stay frozen, so the approach does not require retraining a large language model.
Task rewards can be converted into per-memory utility updates by temporal-difference-style bootstrapping, with newly distilled memories inheriting value from the context that produced them.
Alternating one full pass per component is a more effective coordination schedule than updating both components simultaneously or optimizing them in two separate stages, based on the paper's ablations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the router is initialized to copy the frozen LLM's prior, the marginal value of the learned router could be tested directly by comparing the full system against the same system with the router ablated while keeping the graph and query rewrites intact; the paper's SR-QR ablation approximates this but also removes the route rewrites.
The bootstrap target is self-referential, so a direct stress test of whether memory evolution comes from the bootstrap is to run with $\gamma = 0$ and a constant inherited utility; the paper does not report this variant.
The same retrieved-set coupling should extend to multi-agent shared memory, where private routers read from and write to one graph and cross-agent credit is assigned through exposure; the paper lists this as future work.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

CoEvo-Mem is a serious engineering paper with a reproducible recipe for jointly adapting a retrieval router and a relational memory bank. The core result—that alternating co-evolution beats fixing one side—is supported by the ablations, and the appendix is unusually careful about failure semantics and hyperparameters. Worth a serious referee, but the paper oversells itself. The abstract claims SOTA across seven benchmarks; Table 4 shows the method is second to ElasticMem on both MC tasks, and the LoCoMo OpenQA comparison is not fully matched (imported baselines, category 5 removed). That needs fixing.

What is actually new: the combination of route-specialized rewriting, a residual REINFORCE router that corrects an LLM prior, a Q-weighted hybrid retriever, and relational TD credit propagation with alternating phases. Each ingredient has antecedents, but the closed-loop integration is original and clearly described. The experiments span seven task families, which is a lot of work, and the ablations show both main components matter: removing SR-QR or memory evolution drops macro-average by 4-5 points, and alternating beats simultaneous and two-stage.

Where the evidence is soft: the learned memory utility Q is never isolated. The 'w/o Memory Evolution' ablation removes typed relations, temporal expansion, and graph propagation together, so the 4.80-point drop cannot be attributed to Q. No experiment sets lambda_Q=0 while keeping the graph. That matters because the Q update is self-referential: Equation (6) initializes a new memory's value as the mean of the exposed set's Q, and Equation (7) uses gamma times that same mean as the bootstrap target for the exposed memories. Without a stability analysis, a miscalibrated Q could rank irrelevant memories above relevant ones; the system might still work because lambda_Q is small (0.15) and dense/sparse RRF dominates. The claim that memory evolution improves ranking is therefore untested as stated.

Minor but real: a dozen hyperparameters (lambda_Q, gamma, alpha_Q, rho's, etc.) are set without sensitivity analysis. That is acceptable in a first system paper but should be acknowledged.

Who this is for: anyone working on long-term memory for LLM agents. The paper gives a concrete, high-performing recipe with enough detail to reimplement. I would cite it as a system paper, not as evidence for a specific utility-learning mechanism.

Recommendation: engage with it—send it to peer review. But the authors need to either add an ablation where the Q-term is neutralized (lambda_Q=0 or all values equal) or soften the memory-evolution claim. Expect a conditional accept after revision.

Referee Report

4 major / 4 minor

Summary. The paper proposes CoEvo-Mem, a closed-loop framework that co-evolves a retrieval policy and a persistent memory bank for LLM agents. A frozen LLM produces route-specific query rewrites and a routing prior, and a lightweight residual router is trained online to adjust dense/sparse fusion weights. The memory side maintains a typed relational graph with learned utility values Q, updated by TD-style residuals and relational credit propagation. The two sides are trained in alternating phases, with one component fixed while the other updates. The paper evaluates CoEvo-Mem on seven benchmarks and reports large gains over baselines, with ablations showing degradation when either the query-rewriting/routing module or the memory-evolution module is removed.

Significance. If the central claims held as stated, the paper would make a useful empirical contribution: it identifies a feedback loop between retrieval and memory evolution and provides a concrete instantiation with careful experimental controls, including fixed splits, frozen evaluation, conservative failure semantics, and a detailed appendix. The ablation evidence that co-adaptation helps is nontrivial and generally well designed. However, the headline 'state-of-the-art across seven benchmarks' is inconsistent with the paper's own Table 4, and the specific mechanism attributed to learned memory utility is not isolated by any ablation. These issues do not invalidate the overall approach, but they currently prevent the paper from supporting the strength of its claims.

major comments (4)

[Abstract; Section 'Long-Term Conversational Memory Task'; Table 4] The abstract's claim that 'CoEvo-Mem achieves state-of-the-art performance' across seven benchmarks is contradicted by Table 4, where CoEvo-Mem ranks second to ElasticMem in every matched-backbone setting (e.g., 71 vs. 84 on LoCoMo-MC and 64 vs. 76 on LongMemEval-MC with Qwen2.5-7B-Instruct). The body text itself states that 'CoEvo-Mem ranks second in all matched-backbone comparisons.' The headline claim should be revised to specify the settings in which CoEvo-Mem is best, or the MC results should be explicitly reconciled with the SOTA statement.
[Section 'CoEvo-Mem Framework', Eq. (5); Table 3] The contribution of the learned memory utility Q to retrieval is never isolated. The 'w/o Memory Evolution' ablation removes typed relations, temporal expansion, and graph-based value propagation together, so the reported 4.80-point average drop cannot be assigned to the Q-weighted ranking term in Eq. (5). To support the specific claim that learned utilities improve retrieval rankings, the authors should add an ablation that sets lambda_Q = 0 while retaining the graph structure and propagation, or that varies lambda_Q; otherwise the gains from memory evolution could be driven entirely by temporal graph expansion or relational propagation rather than by value-based ranking.
[Section 'Retrieval-Aligned Relational Memory Evolution', Eqs. (6)-(7)] The value update is self-referential in a way that is not analyzed. Equation (6) initializes a new memory's Q as the mean of the exposed memories' Q values, and Eq. (7) uses gamma times that same mean as the bootstrap target for the exposed memories. Because Q also directly enters the retrieval ranking in Eq. (5), a miscalibrated Q could silently distort rankings. No convergence or stability analysis is provided, and the paper defers theoretical guarantees to future work. Please add an analysis or an experiment that checks Q calibration or compares against a non-bootstrapped utility estimator; without this, the link between memory evolution and retrieval improvement through Q is the weakest step in the argument.
[Appendix B.3; Table 2] Some rows in Table 2 are imported literature values rather than matched reruns, as disclosed in B.3 ('Imported values are system-level literature references, not paired reruns'). Because the main text does not mark which rows are imported, the 'highest overall score' comparison against StructMem and other baselines may be read as a matched comparison. Add explicit markers for imported results, or restrict the headline comparison to matched runs.

minor comments (4)

[Table 3] The first data row of Table 3 is formatted ambiguously: 'GPQA Diamond 68.3360.00 58.33' should be separated as '68.33 60.00 58.33', and the missing delimiter makes the ablation numbers hard to read.
[Table 4 caption] The caption refers to purple and peach highlighting, but these colors are not visible in a monochrome rendering; please add textual markers (e.g., asterisks or bold) for trainable model-integrated methods and for CoEvo-Mem.
[Section 'Route-Specialized Query Rewriting and Retrieval', Eq. (3)-(4)] The KL divergence in Eq. (4) uses Cat(pi_t) and Cat(\bar p^0_t), but it is not specified whether the prior used in the KL term is the smoothed prior from Eq. (3) or the raw p^0_t; please clarify the notation.
[Figure 3] The caption says 'Training success rate' while the y-axis is labeled 'Accuracy'; please make the metric name consistent, and indicate whether the plotted values are means over the five splits.

Circularity Check

1 steps flagged · score 3.0 of 10

Memory utility Q is updated from a bootstrap target equal to the mean of the very Q values it ranks; the central external benchmark claim is not circular.

self definitional [Section 'Retrieval-Aligned Relational Memory Evolution', Eqs. (6)-(7); retrieval use in Eq. (5).]
"Because a new memory has no reuse history, its initial utility is inherited from the local context that produced it: bQt(mnew t ) = |At|−1 P m∈At Qt(m), At̸=∅, Qinit, At=∅. (6) ... δt(m) = Rt ut(m)+γ bQt(mnew t )−Qt(m), m∈At, (7)"

Substituting Eq. (6) into Eq. (7) makes the bootstrap target for every exposed memory m equal to R_t u_t(m) + γ |A_t|^{-1} Σ_{m'∈A_t} Q_t(m') − Q_t(m). The new memory's utility is therefore defined as the mean of the very values being updated, and that same mean is then used as the 'downstream utility' bootstrap for those values. No independent successor-state value or separately measured future outcome enters the bootstrap term. Consequently the learned Q is partially self-referential: at a fixed point it is essentially an affine transform of the immediate reward-attribution terms, not an independently estimated discounted future utility. Since Eq. (5) ranks memories by Q, the Q-weighted ranking inherits this self-reference.

full rationale

CoEvo-Mem's central empirical claim—that jointly adapting retrieval routing and memory evolution outperforms fixed counterparts—is tested against external benchmarks and component ablations, and it does not reduce by construction to its inputs. Table 1 is a standard held-out comparison, and Table 3 removes SR-QR and memory evolution and shows degradation. No load-bearing self-citation chain or imported uniqueness theorem is used; MemQ and MemRL appear as baselines and protocol sources, not as justification of the proposed design. The one genuine circular step is in the memory-utility learning: Eq. (6) initializes the newly distilled memory's value as the mean of the exposed memories' Q values, and Eq. (7) uses γ times that same mean as the TD bootstrap target for those exposed memories. This makes the 'downstream utility' term in the update defined by the very values being updated, so the learned Q is not an independent estimate of future retrieval utility. The immediate reward-attribution term R_t u_t(m) is external, and the reported performance gains are empirically measured, so the paper's headline result is not forced by this self-reference. Score 3 reflects one self-referential mechanism inside an otherwise externally validated system.

Assumptions & free parameters 10 free parameters · 8 assumptions · 2 invented entities

The framework's performance rests on many hand-set hyperparameters and ad hoc design choices; only the residual router is trained, while the memory graph's update rule, thresholds, and propagation factors are specified without sensitivity analysis. The bootstrap value initialization in Eq. (6) is a particularly load-bearing design choice.

free parameters (10)

lambda_Q = 0.15
Utility-rank weight in hybrid score Eq. (5); hand-set, no sensitivity analysis reported.
gamma = 0.35
Bootstrap discount in TD residual Eq. (7); controls how much future utility is inherited.
alpha_Q = 0.3
Memory value update rate in Eq. (10); hand-set.
beta_KL = 0.1
KL penalty weight in Eq. (4); controls deviation from LLM routing prior.
kappa = 20
Beta concentration for exploration in router phase; affects exploration variance.
rho_d, rho_s, rho_t = (.8,.5,.6)
Relation-specific attenuation factors for credit propagation through graph paths (Eq. 8).
tau_d, tau_s = 0.6, 0.3
Edge creation thresholds for dense and sparse relations (Appendix A.5).
eta = 60
RRF reciprocal rank fusion constant in Eq. (5).
Q_min, Q_max = [-1,5]
Projection bounds for utility values in Eq. (10).
D = 4
Maximum path depth for credit propagation.

assumptions (8)

standard math Score-function (REINFORCE) estimator is unbiased (Williams 1992; Ahmadian 2024).
Used in Eq. (4) for router gradient through non-differentiable retrieval.
standard math Reciprocal rank fusion is a valid rank aggregation method (Cormack et al. 2009).
Used in Eq. (5) for hybrid retrieval.
standard math TD-style bootstrapping converges under contraction assumptions.
Eqs. (7)-(10) assume TD value iteration behavior; no contraction proof for this batch bootstrap is given.
domain assumption Task evaluator returns a reward in [0,1] that reflects true outcome quality.
Rt feeds Eq. (4) and Eq. (7); for LoCoMo OpenQA this is an LLM judge with no calibration.
domain assumption The frozen route LLM produces useful rewrites and a routing prior; the residual router only corrects small errors.
Eq. (3) initializes policy at the LLM prior, so performance relies on prior quality.
ad hoc to paper New memory value is initialized to the mean Q of the exposing context (Eq. 6).
Bootstrap initialization is a design choice; it makes the TD target depend on the same Q values being updated.
ad hoc to paper Alternating phases are claimed to mitigate coupling-induced non-stationarity (Section 'Alternating Co-Evolution').
No formal guarantee; supported only by the schedule ablation.
ad hoc to paper Credit propagates along graph paths with relation-specific attenuation (Eq. 8).
Hand-designed propagation rule without external validation.

invented entities (2)

Typed Relational Memory Graph independent evidence
purpose: Stores experiences as nodes with dense, sparse, and time edges matching retrieval channels; supports value propagation.
The graph is central to the method and is evaluated on benchmarks; its design is not externally verified beyond these experiments.
Prior-Guided Residual Router independent evidence
purpose: Learns a residual correction to the LLM's routing prior for dense and sparse retrieval.
A new learned component; its benefit is shown in ablations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CoEvo-Mem: Co-Evolving Retrieval Policy and Memory Bank for LLM Agents." pith.science (2026). https://pith.science/paper/TG7CLBVW

@misc{pith2026260801739,
  author       = {Pith},
  title        = {Pith review of: CoEvo-Mem: Co-Evolving Retrieval Policy and Memory Bank for LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TG7CLBVW}},
  note         = {Machine review of arXiv:2608.01739}
}

read the original abstract

As memories accumulate across tasks and sessions, the performance of long-term LLM agents depends jointly on query-specific retrieval and continual memory refinement. However, existing methods typically optimize either memory access, through iterative query refinement or adaptive retrieval policies, or memory evolution such as structural update. This separation overlooks a fundamental feedback loop: retrieval determines which memories receive usage signals, while updated memory bank reshape future retrieval. We propose \textbf{CoEvo-Mem}, a closed-loop framework for co-evolving the retrieval policy and memory bank. For each query, a frozen LLM generates route-specific query rewrites and a routing prior, which a lightweight residual router corrects online. The retrieved context serves as the coupling interface between the two learning processes: task outcomes assign credit to routing decisions, while trajectory-conditioned feedback updates memory values and graph relations. These updates alter how memories are ranked and selected for subsequent queries, thereby closing the feedback loop. To mitigate coupling induced non-stationarity, CoEvo-Mem alternates between updating the router with the memory bank fixed and evolving the memory bank with the retrieval policy fixed. Across seven diverse benchmarks, \textbf{CoEvo-Mem} achieves state-of-the-art performance, demonstrating the importance of retrieval-memory coevolution.

Figures

Figures reproduced from arXiv: 2608.01739 by the authors.

**Figure 1.** Three paradigms of Memory Systems. to retrieve and update it has become a central problem in the design of long-term agent systems. Existing work improves agent memory along two main directions. Memory-access methods use RAG, query reformulation, and adaptive retrieval to surface relevant experiences for each query (Lewis et al. 2020; Gao et al. 2023; Asai et al. 2024). Memory-centric methods instead improve how e… view at source ↗

**Figure 2.** Overview of CoEvo-Mem. Route-specialized retrieval and relational memory evolution form a closed feedback loop and are co-adapted through alternating training phases. addresses this coupled learning problem through phase-wise co-adaptation of retrieval and relational memory, as summarized in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Training success rate on LoCoMo free-form QA. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 10 canonical work pages

[1]

Backtoba- sics:RevisitingREINFORCE-styleoptimizationforlearning from human feedback in LLMs

Ahmadian,A.;Cremer,C.;Gallé,M.;Fadaee,M.;Kreutzer, J.;Pietquin,O.;Üstün,A.;andHooker,S.2024. Backtoba- sics:RevisitingREINFORCE-styleoptimizationforlearning from human feedback in LLMs. InProceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), 12248–12267. Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and ...

work page 2024
[4]

Santhanam, K.; Khattab, O.; Saad-Falcon, J.; Potts, C.; and Zaharia,M.2022

Now Publishers Inc. Santhanam, K.; Khattab, O.; Saad-Falcon, J.; Potts, C.; and Zaharia,M.2022. Colbertv2:Effectiveandefficientretrieval via lightweight late interaction. InProceedings of the 2022 ConferenceoftheNorthAmericanChapteroftheAssociation for Computational Linguistics: Human Language Technolo- gies, 3715–3734. Shinn, N.; Cassano, F.; Gopinath, A...

work page 2022
[5]

arXiv preprint arXiv:2501.06713

Minirag: Towards extremely simple retrieval-augmented generation. arXiv preprint arXiv:2501.06713. Fang, R.; Liang, Y.; Wang, X.; Wu, J.; Qiao, S.; Xie, P.; Huang,F.;Chen,H.;andZhang,N.2026. Memp:Exploring agentproceduralmemory. InFindingsoftheAssociationfor Computational Linguistics: ACL 2026, 17490–17502. Feng, T.; Ye, C.; Luo, T.; Xu, J.; Xu, X.; Zhang...

arXiv 2026
[7]

Hu, M.; Chen, T.; Chen, Q.; Mu, Y.; Shao, W.; and Luo, P

Ligh- trag: Simple and fast retrieval-augmented generation.arXiv preprint arXiv:2410.05779, 2(3). Hu, M.; Chen, T.; Chen, Q.; Mu, Y.; Shao, W.; and Luo, P

arXiv
[8]

InInternationalConference on Learning Representations, volume 2025, 58791–58831

Live- codebench: Holistic and contamination free evaluation of largelanguagemodelsforcode. InInternationalConference on Learning Representations, volume 2025, 58791–58831. Jin, B.; Zeng, H.; Yue, Z.; Yoon, J.; Arik, S.; Wang, D.; Zamani, H.; and Han, J

work page 2025
[9]

Kang,J.;Ji,M.;Zhao,Z.;andBai,T.2025

Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516. Kang,J.;Ji,M.;Zhao,Z.;andBai,T.2025. Memoryosofai agent. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 25972–25981. Karpukhin,V.;Oguz,B.;Min,S.;Lewis,P.;Wu,L.;Edunov, S.; Chen, D.; and Yih, W.-t

arXiv 2025
[11]

InProceedings ofthe 43rd International ACMSI- GIRconferenceonresearchanddevelopmentinInformation Retrieval, 39–48

Colbert: Efficient and effective passage search via contextualized late interaction over bert. InProceedings ofthe 43rd International ACMSI- GIRconferenceonresearchanddevelopmentinInformation Retrieval, 39–48. LangChainAI.2024.LangMem.https://langchain-ai.github. io/langmem/. Accessed: 2024-01-01. Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, ...

work page 2024
[12]

MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs.arXiv preprint arXiv:2605.08374. Maharana, A.; Lee, D.-H.; Tulyakov, S.; Bansal, M.; Bar- bieri, F.; and Fang, Y

work page Pith review arXiv

Show all 24 references

[13]

InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 13851–13870

Evaluating very long-term conversational memory of llm agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 13851–13870. memodb-io.2024. Memobase. https://github.com/memodb- io/memobase#get-started. Accessed...

2024
[14]

Rasmussen, P.; Paliychuk, P.; Beauvais, T.; Ryan, J.; and Chalef,D.2025

Rag-fusion: a new take on retrieval- augmented generation.arXiv preprint arXiv:2402.03367. Rasmussen, P.; Paliychuk, P.; Beauvais, T.; Ryan, J.; and Chalef,D.2025. Zep:atemporalknowledgegrapharchitec- ture for agent memory.arXiv preprint arXiv:2501.13956. Rein, D.; Hou, B. L.;...

2025 arXiv
[15]

Robertson, S.; and Zaragoza, H

Gpqa: A graduate-levelgoogle-proofq&abenchmark.arXivpreprint arXiv:2311.12022. Robertson, S.; and Zaragoza, H. 2009.The probabilistic relevance framework: BM25 and beyond, volume

2009 arXiv
[17]

Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, Y.; Fan, L.; and Anandkumar, A

Reflexion: Language agents with verbal re- inforcement learning.Advances in neural information pro- cessing systems, 36: 8634–8652. Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, Y.; Fan, L.; and Anandkumar, A. 2023a. Voyager: An open- ended embodied agent with la...

arXiv
[18]

Wang, Y.; Krotov, D.; Hu, Y.; Gao, Y.; Zhou, W.; McAuley, J.; Gutfreund, D.; Feris, R.; and He, Z

Memoryllm: To- wards self-updatable large language models.arXiv preprint arXiv:2402.04624. Wang, Y.; Krotov, D.; Hu, Y.; Gao, Y.; Zhou, W.; McAuley, J.; Gutfreund, D.; Feris, R.; and He, Z

arXiv
[19]

Williams,R.J.1992.Simplestatisticalgradient-followingal- gorithms for connectionist reinforcement learning.Machine learning, 8(3): 229–256

M+: Extend- ing MemoryLLM with scalable long-term memory.arXiv preprint arXiv:2502.00592. Williams,R.J.1992.Simplestatisticalgradient-followingal- gorithms for connectionist reinforcement learning.Machine learning, 8(3): 229–256. Wu, D.; Wang, H.; Yu, W.; Zhang, Y.; Chang, K.-...

1992 arXiv
[20]

Xu, B.; Chen, Y.; Fang, J.; Zhong, R.; Yao, Y.; Zhu, Y.; Du, L.; and Deng, S

Longmemeval: Benchmarking chat as- sistants on long-term interactive memory.arXiv preprint arXiv:2410.10813. Xu, B.; Chen, Y.; Fang, J.; Zhong, R.; Yao, Y.; Zhu, Y.; Du, L.; and Deng, S. 2026a. StructMem: Structured Memory for Long-Horizon Behavior in LLMs. InProceedings of th...

arXiv
[21]

Zhang, S.; Wang, J.; Zhou, R.; Liao, J.; Feng, Y.; Li, Z.; Zheng, Y.; Zhang, W.; Wen, Y.; Li, Z.; et al

Memgen: Weaving generative latent memory for self-evolving agents.arXiv preprint arXiv:2509.24704. Zhang, S.; Wang, J.; Zhou, R.; Liao, J.; Feng, Y.; Li, Z.; Zheng, Y.; Zhang, W.; Wen, Y.; Li, Z.; et al
[22]

Zhang, Z.; Dai, Q.; Bo, X.; Ma, C.; Li, R.; Chen, X.; Zhu, J.; Dong, Z.; and Wen, J.-R

Memrl: Self-evolving agents via runtime reinforcement learning on episodic memory.arXiv preprint arXiv:2601.03192. Zhang, Z.; Dai, Q.; Bo, X.; Ma, C.; Li, R.; Chen, X.; Zhu, J.; Dong, Z.; and Wen, J.-R

arXiv
[23]

Zhao,A.;Huang,D.;Xu,Q.;Lin,M.;Liu,Y.-J.;andHuang, G.2024

A survey on the memory mechanism of large language model-based agents.ACM Transactions on Information Systems, 43(6): 1–47. Zhao,A.;Huang,D.;Xu,Q.;Lin,M.;Liu,Y.-J.;andHuang, G.2024. Expel:Llmagentsareexperientiallearners. InPro- ceedings of the AAAI Conference on Artificial In...

2024
[24]

Zhong, W.; Guo, L.; Gao, Q.; Ye, H.; and Wang, Y

Lifelongagentbench: Evaluating llm agentsaslifelonglearners.arXivpreprintarXiv:2505.11942. Zhong, W.; Guo, L.; Gao, Q.; Ye, H.; and Wang, Y

arXiv
[2020]

InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 6769–6781

Dense passage retrieval for open-domain question answering. InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 6769–6781. Khattab, O.; and Zaharia, M

2020
[2023]

InPro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 3829–3846

Adapting language models to compress contexts. InPro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 3829–3846. Chhikara, P.; Khant, D.; Aryan, S.; Singh, T.; and Yadav, D

2023
[2024]

InInternational conference on learning rep- resentations, volume 2024, 9112–9141

Self-rag:Learningtoretrieve,generate,andcritiquethrough self-reflection. InInternational conference on learning rep- resentations, volume 2024, 9112–9141. Chevalier, A.; Wettig, A.; Ajith, A.; and Chen, D

2024
[2025]

Cormack, G

Mem0:Buildingproduction-readyaiagentswithscal- able long-term memory.arXiv preprint arXiv:2504.19413. Cormack, G. V.; Clarke, C. L.; and Buettcher, S

arXiv
[2026]

Formal, T.; Piwowarski, B.; and Clinchant, S

ElasticMem: Latent Memory as a Learnable Resource for LLM Agents.arXiv preprint arXiv:2605.30690. Formal, T.; Piwowarski, B.; and Clinchant, S

arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Backtoba- sics:RevisitingREINFORCE-styleoptimizationforlearning from human feedback in LLMs

Ahmadian,A.;Cremer,C.;Gallé,M.;Fadaee,M.;Kreutzer, J.;Pietquin,O.;Üstün,A.;andHooker,S.2024. Backtoba- sics:RevisitingREINFORCE-styleoptimizationforlearning from human feedback in LLMs. InProceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), 12248–12267. Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and ...

work page 2024

[2] [4]

Santhanam, K.; Khattab, O.; Saad-Falcon, J.; Potts, C.; and Zaharia,M.2022

Now Publishers Inc. Santhanam, K.; Khattab, O.; Saad-Falcon, J.; Potts, C.; and Zaharia,M.2022. Colbertv2:Effectiveandefficientretrieval via lightweight late interaction. InProceedings of the 2022 ConferenceoftheNorthAmericanChapteroftheAssociation for Computational Linguistics: Human Language Technolo- gies, 3715–3734. Shinn, N.; Cassano, F.; Gopinath, A...

work page 2022

[3] [5]

arXiv preprint arXiv:2501.06713

Minirag: Towards extremely simple retrieval-augmented generation. arXiv preprint arXiv:2501.06713. Fang, R.; Liang, Y.; Wang, X.; Wu, J.; Qiao, S.; Xie, P.; Huang,F.;Chen,H.;andZhang,N.2026. Memp:Exploring agentproceduralmemory. InFindingsoftheAssociationfor Computational Linguistics: ACL 2026, 17490–17502. Feng, T.; Ye, C.; Luo, T.; Xu, J.; Xu, X.; Zhang...

arXiv 2026

[4] [7]

Hu, M.; Chen, T.; Chen, Q.; Mu, Y.; Shao, W.; and Luo, P

Ligh- trag: Simple and fast retrieval-augmented generation.arXiv preprint arXiv:2410.05779, 2(3). Hu, M.; Chen, T.; Chen, Q.; Mu, Y.; Shao, W.; and Luo, P

arXiv

[5] [8]

InInternationalConference on Learning Representations, volume 2025, 58791–58831

Live- codebench: Holistic and contamination free evaluation of largelanguagemodelsforcode. InInternationalConference on Learning Representations, volume 2025, 58791–58831. Jin, B.; Zeng, H.; Yue, Z.; Yoon, J.; Arik, S.; Wang, D.; Zamani, H.; and Han, J

work page 2025

[6] [9]

Kang,J.;Ji,M.;Zhao,Z.;andBai,T.2025

Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516. Kang,J.;Ji,M.;Zhao,Z.;andBai,T.2025. Memoryosofai agent. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 25972–25981. Karpukhin,V.;Oguz,B.;Min,S.;Lewis,P.;Wu,L.;Edunov, S.; Chen, D.; and Yih, W.-t

arXiv 2025

[7] [11]

InProceedings ofthe 43rd International ACMSI- GIRconferenceonresearchanddevelopmentinInformation Retrieval, 39–48

Colbert: Efficient and effective passage search via contextualized late interaction over bert. InProceedings ofthe 43rd International ACMSI- GIRconferenceonresearchanddevelopmentinInformation Retrieval, 39–48. LangChainAI.2024.LangMem.https://langchain-ai.github. io/langmem/. Accessed: 2024-01-01. Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, ...

work page 2024

[8] [12]

MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs.arXiv preprint arXiv:2605.08374. Maharana, A.; Lee, D.-H.; Tulyakov, S.; Bansal, M.; Bar- bieri, F.; and Fang, Y

work page Pith review arXiv