REVIEW 3 major objections 5 minor 54 references
MESA:Task-Adaptive Multi-Structure Evidence Selection for Long-Horizon Agent Memory
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that a learned, query-adaptive selection among complementary memory structures outperforms both single-structure routing and full fusion for long-horizon agent memory.
desk verdict Useful formulation and extensive experiments, but the missing QA-type ablation leaves the main gain possibly explained by annotation metadata. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the selector policy $\rho$ that maps a query $q_i$ and observable context $c_i$ to a non-empty binary vector $z_i \in \{0,1\}^K$ over $K=5$ memory structures (summary, temporal store, knowledge graph, vector database, and raw episodic trace). The optimizer, called prior-guided harness optimization, keeps all builders, retrievers, composer, and the answer model fixed and searches only over executable selection policies generated under twenty prior directions organized into five axes: demand inference, backend value estimation, subset construction, evidence awareness, and robustness and cost shaping. Candidate policies are fitted on training traces, scored on validation with the regularized objective $\tilde{J}_{\mathrm{val}}(\rho)=J_{\mathrm{val}}(\rho)-\lambda C_{\mathrm{val}}(\rho)$, and a UCB rule $\mathrm{UCB}(a)=\bar{\tilde{J}}_{\mathrm{val}}(a)+\beta\sqrt{\log(N+2)/(n_a+1)}$ schedules which proposal direction the LLM proposer explores. This machinery converts sparse, answer-level correctness feedback into a concrete, inspectable routing rule.
What would settle it
Run the same experiments with a 1:1:8 train/validation/test split or with leave-one-out selection over episodes, and check whether the validation-selected policy still beats the all-structure configuration on the test set; if the advantage reverses or disappears, the central claim fails.
Extended reading notes
Core claim
The central claim is that the optimal memory configuration for a long-horizon agent is typically neither a single structure nor the full union, but an intermediate, query-dependent subset. In a controlled analysis on AMA-Bench, the best composition varied across domains and memory capabilities, so no fixed subset was universally optimal. MESA learns an explicit selection policy that maps a query and observable context to a non-empty subset of five structures, and this learned policy outperforms all baselines on AMA-Bench with two different answer-model backbones while also improving F1 on the conversational LoCoMo benchmark. The paper attributes the gains to composing complementary evidence views rather than to any single strong representation, since the strongest single structure alone reaches only 57.7% accuracy, while MESA reaches 65.1%.
Load-bearing premise
The load-bearing premise is that picking the policy with the best regularized score on a validation set of roughly 20 percent of episodes will keep its advantage on held-out episodes, even though the search only tries about 90 candidate policies.
Editorial extensions
If this is right
- If MESA is right, agent memory systems should expose multiple complementary structures and learn per-query selection instead of using fixed routing or full fusion.
- The 41.2% token reduction at higher accuracy indicates that adaptive selection is a cost-saving strategy, not a trade-off between quality and efficiency.
- Accuracy gains are concentrated in domains that require combining global state with exact identifiers and temporal locations, such as software engineering and text-to-SQL, suggesting that composition of complementary structures is the mechanism.
- The framework transfers from agent trajectories to long-term conversations, so the principle is not limited to episodic agent memory.
- Learned policies remain explicit, keyword-based rules, making memory-access decisions auditable and easy to inspect.
Reading between the lines
- Inference: The intermediate-subset finding suggests a general design principle: memory access can be treated as per-query feature selection, and new memory structures should be evaluated by their marginal contribution to selection rather than in isolation.
- Inference: Because MESA freezes memory builders and retrievers, jointly optimizing construction with selection could yield larger gains and may shift the optimal subset toward fewer or different structures than the five studied here.
- Inference: The validation-based selection protocol is the main risk; a natural stress test is nested cross-validation or a much larger validation budget to see whether the reported margin persists.
- Inference: The explicit rules learned by MESA could be distilled into a small deterministic router that runs without an LLM proposer at inference time, further reducing cost and latency.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MESA, a framework for query-adaptive selection among five pre-built memory structures (summary, temporal store, knowledge graph, vector database, raw episodic traces) for long-horizon agent memory. A learned selector, trained by an LLM-proposer harness search with UCB-guided scheduling and answer-level feedback, chooses a non-empty subset of structures per query; the selected evidence is fused and passed to a frozen answer model. On AMA-Bench, MESA reports 65.1% overall accuracy versus 56.6% for AMA-Agent, 63.7% for the all-structure configuration, and 52.6% for the long-context reader, while using 11.0k versus 18.7k evidence tokens per query. The paper also reports an exhaustive sweep over all 31 non-empty structure subsets, finding that the best subset is usually neither a single structure nor the full union and varies by task; additional experiments on LoCoMo and multiple ablation and robustness tables are included.
Significance. If the main result holds, the paper makes a useful contribution: it provides a controlled comparison of memory-structure composition strategies and shows that a learned, query-adaptive subset can beat both single-structure routing and full fusion while reducing token cost. Strengths include the exhaustive 31-subset sweep, experiments with two answer-model backbones, judge-robustness checks, a second benchmark (LoCoMo), and explicitly inspectable final policies, with a clear statement that test outcomes are not used during policy selection. The central risk is that the selector's context includes gold QA-type and capability annotations that are not available to baselines or in deployment; the paper does not ablate this input. A second concern is the small validation set used for policy selection. Both are testable and addressable within the manuscript's scope.
major comments (3)
- [§4.1, Fig. 5, Appendix D] The selector context c_i is defined to include 'task, domain, and QA-type information,' and the accepted policies in Fig. 5 explicitly branch on gold QA-type and capability labels (e.g., 'Causal inference / text-to-sql -> {S,T,R}', 'state/update -> {S,T,G,V}', 'abstract -> all five'). These QA-type and capability labels are benchmark annotations, not user-visible query text, and no baseline receives them. Because the controlled sweep in Sec. 3 shows that the winning subset varies by QA type, a selector that reads the gold type has a privileged signal that could explain part of the 8.5-point gain over AMA-Agent and the 1.4-point gain over the all-structure configuration. Table 9 partially addresses coarse domain/capability-conditioned lookup, but it does not isolate the QA-type channel. Please add an ablation that removes QA-type and capability labels from c_i and from the proposer's rule language, and report the resulting accuracy on the same five splits; this is the key test of whether the measured advantage is learned selection or annotation leakage.
- [Algorithm 1, §5.1] The final policy is chosen as the argmax of the regularized validation objective over an archive produced by 30 iterations with roughly three candidates each, while the validation split contains only about 20% of 208 episodes (approximately 42 episodes). The reported test numbers are therefore the result of validation-based selection over roughly 90 candidate policies. This is a realistic risk of selection bias. Please report the per-split relationship between validation and test scores for all archive policies, or compare the argmax selection rule with alternative rules (e.g., median validation policy, top-3 ensemble, or a stability criterion) to show that the 65.1% headline is not an artifact of choosing the best-looking validation policy.
- [Table 3] Several component-level claims are not supported by the reported variability. MESA (65.1 ± 0.8) versus the all-structure configuration (63.7 ± 1.0) and versus w/o UCB (64.3 ± 2.2) are within one standard deviation; the statements that removing UCB 'lowers accuracy' and that UCB 'stabilizes the search' rely on overlapping intervals. Please report paired per-split differences with significance tests or bootstrap confidence intervals before interpreting the ablation rows. This does not undermine the headline 8.5-point gain over AMA-Agent, but it does weaken the specific attribution of gains to the UCB component and the claimed accuracy advantage over full fusion.
minor comments (5)
- [Appendix D vs. §4.1] Appendix D says the selector may access 'the question, task metadata, and query-conditioned evidence previews' but does not mention QA-type, while §4.1 and Fig. 5 make clear that QA-type is used; please reconcile these descriptions.
- [Table 2 vs. Table 7] The LoCoMo gain of +1.8 F1 over EMem reported in Table 2 is within the five-fold standard deviations reported in Table 7 (MESA 49.0 ± 3.3, EMem 47.0 ± 4.6). Please state clearly whether the main LoCoMo comparison is on one fixed split and add a paired significance statement over the five conversation-level splits.
- [Appendix F.3] The last sentence of F.3 contains a typo: 'a acceptable' should be 'an acceptable'.
- [Fig. 5 caption] The label 'Accepted TEST (post-hoc)' in Fig. 5 is confusing; the footnote clarifies that these test accuracies were generated after optimization and never exposed during selection, so the caption should say so directly.
- [Appendix B.2] The hyperparameters λ and β are fixed across all settings; reporting a small sensitivity analysis (e.g., λ = 0 and β = 0.04/0.16) would make the calibration argument in B.2 more convincing.
Circularity Check
No circularity identified: MESA's claims are empirical comparisons with independent controls, not derivations that reduce to their inputs.
full rationale
MESA's load-bearing claims are empirical system comparisons rather than derivations, and the paper's own equations do not reduce any predicted quantity to a fitted input. The controlled analysis in Sec. 3 exhaustively sweeps all 31 non-empty subsets of the five structures, and Eq. (7) and Eq. (8) define a validation objective that Algorithm 1 optimizes by returning the argmax over the archive; the final policy is frozen and evaluated on held-out test splits (Sec. 5.1), so the headline 65.1% versus 56.6% comparison is not a refit of the reported result. The hyperparameters lambda and beta are fixed on development data, and the paper reports ablations (route-to-one, w/o priors, w/o UCB, and validation-selected fixed subsets in Table 9) that serve as independent controls for the selection mechanism. The reuse of AMA-Agent's memory schema, retrieval components, and judge prompt is openly declared as component reuse, and AMA-Bench and AMA-Agent are cited as external prior work rather than as an unverified self-supporting uniqueness claim; no evidence in the text establishes author overlap, so no self-citation circularity is asserted. The fact that the selector context may include QA-type labels (Sec. 4.1, Appendix D, Fig. 5) and that this input is not ablated is a legitimate correctness or leakage risk but not a case where the derivation is equivalent to its input by construction; accordingly, the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- Evidence-cost coefficient lambda =
1e-6
- UCB exploration coefficient beta =
0.08
- Fixed retrieval depths for structures G, V, R, and T fallback =
k_G=8, k_V=5, k_R=3, temporal top-5 fallback
assumptions (4)
- domain assumption LLM-as-judge binary correctness is a valid measure of answer quality on AMA-Bench.
- domain assumption The five frozen memory structures and their access mechanisms expose all evidence needed to answer queries.
- domain assumption A policy selected on roughly 20% of episodes as validation generalizes to held-out episodes within each of five random splits.
- domain assumption Freezing builders, retrievers, composer, and answer model isolates the effect of the selection policy.
Cite this review
Pith. "Pith review of MESA:Task-Adaptive Multi-Structure Evidence Selection for Long-Horizon Agent Memory." pith.science (2026). https://pith.science/paper/QD6EORJY
@misc{pith2026260810108,
author = {Pith},
title = {Pith review of: MESA:Task-Adaptive Multi-Structure Evidence Selection for Long-Horizon Agent Memory},
year = {2026},
howpublished = {\url{https://pith.science/paper/QD6EORJY}},
note = {Machine review of arXiv:2608.10108}
}
read the original abstract
Long-horizon agents accumulate trajectories spanning hundreds of interleaved reasoning, action, and observation steps, where answering a query may depend on evidence buried far back in the history. External memory stores such trajectories as structured representations, yet each structure provides a distinct and incomplete view. Existing multi-memory systems either read a fixed set of structures for every query, inflating context and introducing noise, or route each query to a single structure, preventing the composition of complementary evidence. A controlled analysis on AMA-Bench shows that the optimal memory configuration is typically neither a single structure nor the full union, but a tailored composition of multiple structural memories that varies with query and task demands. Motivated by these findings, we formulate structure-level dynamic selection: selecting and fusing a query-adaptive subset from a library of specialized memory structures. We propose MESA (a Multi-structure Evidence Selection framework for long-horizon Agent), which builds five complementary structure views of each trajectory and learns from end-to-end answer-level feedback to select and fuse a query-specific subset for a frozen answer model. To learn under this weak supervision, MESA employs harness optimization with prior-guided search and UCB-guided scheduling to balance exploration and exploitation. On AMA-Bench, MESA outperforms the strongest baseline by 8.5% while using 41% fewer evidence tokens than the all-structure alternative.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
Evaluating very long-term conversational memory of llm agents , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
-
[2]
arXiv preprint arXiv:2010.03768 , year=
Alfworld: Aligning text and embodied environments for interactive learning , author=. arXiv preprint arXiv:2010.03768 , year=
arXiv 2010
-
[3]
Plugmem: A task-agnostic plugin memory module for llm agents, 2026 , author=. URL https://arxiv. org/abs/2603.03296 , year=
arXiv 2026
-
[4]
StructMem: Structured Memory for Long-Horizon Behavior in LLMs , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=
-
[5]
International Conference on Learning Representations , volume=
Structrag: Boosting knowledge intensive reasoning of llms via inference-time hybrid information structurization , author=. International Conference on Learning Representations , volume=
-
[6]
S3Mem: Structured Spatiotemporal Scene-Event Memory for Long-Horizon Interactive Question Answering
S3Mem: Structured Spatiotemporal Scene-Event Memory for Long-Horizon Interactive Question Answering , author=. arXiv preprint arXiv:2605.28831 , year=
-
[7]
arXiv preprint arXiv:2512.12818 , year=
Hindsight is 20/20: Building agent memory that retains, recalls, and reflects , author=. arXiv preprint arXiv:2512.12818 , year=
-
[8]
arXiv preprint arXiv:2602.02474 , year=
MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents , author=. arXiv preprint arXiv:2602.02474 , year=
Show all 54 references
-
[9]
, author=
MemGPT: towards LLMs as operating systems. , author=. 2023 , publisher=
2023
-
[10]
Proceedings of the 36th annual acm symposium on user interface software and technology , pages=
Generative agents: Interactive simulacra of human behavior , author=. Proceedings of the 36th annual acm symposium on user interface software and technology , pages=
-
[11]
Advances in neural information processing systems , volume=
Reflexion: Language agents with verbal reinforcement learning , author=. Advances in neural information processing systems , volume=
-
[12]
arXiv preprint arXiv:2305.16291 , year=
Voyager: An open-ended embodied agent with large language models , author=. arXiv preprint arXiv:2305.16291 , year=
-
[13]
Proceedings of the AAAI conference on artificial intelligence , year=
Memorybank: Enhancing large language models with long-term memory , author=. Proceedings of the AAAI conference on artificial intelligence , year=
-
[14]
arXiv preprint arXiv:2504.19413 , year=
Mem0: Building production-ready ai agents with scalable long-term memory , author=. arXiv preprint arXiv:2504.19413 , year=
-
[15]
Advances in neural information processing systems , volume=
Hipporag: Neurobiologically inspired long-term memory for large language models , author=. Advances in neural information processing systems , volume=
-
[16]
arXiv preprint arXiv:2505.09388 , year=
Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=
-
[17]
arXiv preprint arXiv:2410.10813 , year=
Longmemeval: Benchmarking chat assistants on long-term interactive memory , author=. arXiv preprint arXiv:2410.10813 , year=
-
[18]
arXiv preprint arXiv:2306.02224 , year=
Auto-gpt for online decision making: Benchmarks and additional opinions , author=. arXiv preprint arXiv:2306.02224 , year=
-
[19]
arXiv preprint arXiv:2601.06411 , year=
Structured Episodic Event Memory , author=. arXiv preprint arXiv:2601.06411 , year=
-
[20]
Proceedings of the ACM Web Conference 2026 , pages=
Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation , author=. Proceedings of the ACM Web Conference 2026 , pages=
2026
-
[21]
arXiv preprint arXiv:2602.14038 , year=
Choosing how to remember: Adaptive memory structures for llm agents , author=. arXiv preprint arXiv:2602.14038 , year=
-
[22]
arXiv preprint arXiv:2602.03315 , year=
Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity , author=. arXiv preprint arXiv:2602.03315 , year=
-
[23]
Advances in Neural Information Processing Systems , volume=
A-mem: Agentic memory for llm agents , author=. Advances in Neural Information Processing Systems , volume=
-
[24]
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
Mem2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
-
[25]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Toolformer: Language Models Can Teach Themselves to Use Tools , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[26]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Tree of Thoughts: Deliberate Problem Solving with Large Language Models , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[27]
Transactions of the Association for Computational Linguistics (TACL) , year=
Lost in the Middle: How Language Models Use Long Contexts , author=. Transactions of the Association for Computational Linguistics (TACL) , year=
-
[28]
Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle=
-
[29]
Advances in Neural Information Processing Systems , year=
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , author=. Advances in Neural Information Processing Systems , year=
-
[30]
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing , year=
Dense Passage Retrieval for Open-Domain Question Answering , author=. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing , year=
2020
-
[31]
arXiv preprint arXiv:2603.16862 , year=
Chronos: Temporal-aware conversational agents with structured event retrieval for long-term memory , author=. arXiv preprint arXiv:2603.16862 , year=
-
[32]
URL https://arxiv
Synapse: Trajectory-as-exemplar prompting with memory for computer control, 2024 , author=. URL https://arxiv. org/abs/2306.07863 , year=
2024 arXiv
-
[33]
arXiv preprint arXiv:2403.05530 , year=
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. arXiv preprint arXiv:2403.05530 , year=
-
[34]
arXiv preprint arXiv:2602.22769 , year=
AMA-Bench: Evaluating long-horizon memory for agentic applications , author=. arXiv preprint arXiv:2602.22769 , year=
-
[35]
arXiv e-prints , pages=
E-mem: Multi-agent based Episodic Context Reconstruction for LLM Agent Memory , author=. arXiv e-prints , pages=
-
[36]
2025 , journal=
Memorag: boosting long context processing with global memory-enhanced retrieval augmentation , author=. 2025 , journal=
2025
-
[37]
arXiv preprint arXiv:2605.15701 , year=
H-Mem: A Novel Memory Mechanism for Evolving and Retrieving Agent Memory via a Hybrid Structure , author=. arXiv preprint arXiv:2605.15701 , year=
-
[38]
arXiv preprint arXiv:2601.20352 , year=
AMA: Adaptive Memory via Multi-Agent Collaboration , author=. arXiv preprint arXiv:2601.20352 , year=
-
[39]
arXiv preprint arXiv:2507.19457 , year=
Gepa: Reflective prompt evolution can outperform reinforcement learning , author=. arXiv preprint arXiv:2507.19457 , year=
-
[40]
arXiv preprint arXiv:2603.28052 , year=
Meta-harness: End-to-end optimization of model harnesses , author=. arXiv preprint arXiv:2603.28052 , year=
-
[41]
2026 , doi =
Pan, Wenbo and Liu, Shujie and Zhou, Xiangyang and Zhang, Shiwei and Shi, Wanlu and Xu, Mirror and Jia, Xiaohua , journal =. 2026 , doi =
2026
-
[42]
arXiv preprint arXiv:2604.20801 , year=
Synthesizing multi-agent harnesses for vulnerability discovery , author=. arXiv preprint arXiv:2604.20801 , year=
-
[43]
arXiv preprint arXiv:2606.08348 , year=
Bayesian-Agent: Posterior-Guided Skill Evolution for LLM Agent Harnesses , author=. arXiv preprint arXiv:2606.08348 , year=
-
[44]
Machine learning , volume=
Finite-time analysis of the multiarmed bandit problem , author=. Machine learning , volume=. 2002 , publisher=
2002
-
[45]
Annals of Mathematics and Artificial Intelligence , volume=
Analyzing bandit-based adaptive operator selection mechanisms , author=. Annals of Mathematics and Artificial Intelligence , volume=. 2010 , publisher=
2010
-
[46]
arXiv preprint arXiv:2605.28009 , year=
MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models , author=. arXiv preprint arXiv:2605.28009 , year=
-
[47]
arXiv preprint arXiv:2506.05176 , year=
Qwen3 embedding: Advancing text embedding and reranking through foundation models , author=. arXiv preprint arXiv:2506.05176 , year=
-
[48]
Proceedings of the 2023 conference on empirical methods in natural language processing , pages=
Llmlingua: Compressing prompts for accelerated inference of large language models , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=
2023
-
[49]
arXiv preprint arXiv:2606.24976 , year=
Diagnosing and Mitigating Compounding Failures in Agentic Persuasion via Taxonomic Strategy Retrieval , author=. arXiv preprint arXiv:2606.24976 , year=
-
[50]
arXiv preprint arXiv:2607.02770 , year=
Gemma 4 technical report , author=. arXiv preprint arXiv:2607.02770 , year=
-
[51]
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
-
[52]
2026 , publisher=
From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms , author=. 2026 , publisher=
2026
-
[53]
arXiv preprint arXiv:2504.15965 , year=
From human memory to ai memory: A survey on memory mechanisms in the era of llms , author=. arXiv preprint arXiv:2504.15965 , year=
-
[54]
arXiv preprint arXiv:2502.14802 , year=
From rag to memory: Non-parametric continual learning for large language models , author=. arXiv preprint arXiv:2502.14802 , year=
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.