REVIEW 2 major objections 7 minor 18 references
9B agent beats 397B on memory tasks by learning to navigate, not retrieve
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · glm-5.2
2026-07-08 23:43 UTC pith:OEVXU55A
load-bearing objection A 9B model trained with RL over a structured memory pyramid beats 10-40x larger untrained models on memory-intensive tasks — but the headline rests on in-domain benchmarks, and the OOD evidence is thin. the 2 major comments →
From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central finding is that a relatively small model (9B parameters) trained to actively navigate a structured memory pyramid—choosing among raw conversations, compact records, topic tracks, and user profiles via tool calls—can outperform models more than 40 times larger that use the same memory sources but without a learned navigation policy. The ablation showing that removing active navigation (reverting to passive retrieval over the same memory) drops the average score from 62.74 to 54.08, and that removing RL training (keeping the tools but using the untrained base model) drops it to 48.39, isolates the contribution of the learned navigation policy from the contribution of the存储.
What carries the argument
The multi-granularity memory pyramid with five navigation tools (search_conversations, get_conversations, search_records, get_records, read_file), trained via Group Relative Policy Optimization (GRPO) with a terminal reward that jointly optimizes answer correctness, format validity, and memory-tool usage.
Load-bearing premise
The RL reward function requires at least one memory-tool call for a trajectory to receive the highest reward, which means the training explicitly forces tool use. This makes the comparison between active navigation and passive retrieval not fully balanced, since the trained model was never rewarded for answering correctly without tools.
What would settle it
If a variant of the reward function that equally rewards correct answers without tool use were trained, and the active-navigation advantage over passive retrieval disappeared or shrank substantially, the paper's central claim that learned navigation is the source of the gains would be undermined.
If this is right
- If active memory navigation generalizes beyond the tested benchmarks, conversational agents could achieve strong personalization with much smaller models, reducing deployment cost for long-running assistant applications.
- The finding that RL training reduces unnecessary memory calls on non-memory tasks (from 34.5% to 6.9% on GPQA-D) suggests that learned memory-use policies can self-calibrate, avoiding the over-retrieval problem common in memory-augmented systems.
- The pyramid structure with provenance links between levels could serve as a template for other hierarchical information-access problems beyond user memory, such as codebase navigation or document review.
- The observation that larger models prefer top-down navigation (starting from profiles) while smaller models prefer bottom-up (starting from records) raises questions about whether navigation strategy should be conditioned on model capacity in deployed systems.
Where Pith is reading between the lines
- The reward function gives r=1 only to correct answers that use at least one memory tool, and r=0 to correct answers that skip tools. This means the RL training explicitly biases the policy toward tool use, which could inflate the apparent benefit of active navigation in the ablation comparison against passive retrieval. A reward that also rewards correct no-tool answers would provide a cleaner tes
- If the pyramid structure and navigation policy are both learnable end-to-end (rather than the pyramid being constructed by separate prompts), the system might discover memory granularities that do not correspond to the four hand-designed levels, potentially improving or simplifying the hierarchy.
- The four-tool-call budget per query may constrain the navigation depth for complex multi-hop reasoning; analyzing failure cases that exhaust the budget could reveal whether the policy is choosing well within its limits or hitting an artificial ceiling.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces NapMem, a framework that reframes long-term user memory from passive retrieval into a structured action space. The system organizes user history into a four-level memory pyramid (raw conversations, typed memory records, topic tracks, user profiles) connected by provenance links, and exposes these levels through five memory tools. A 9B base model is trained with GRPO to learn a navigation policy that selects which memory granularity to access based on the query and intermediate evidence. Experiments on three memory-intensive benchmarks (LoCoMo, LongMemEval, PersonaMem-v2) and three non-memory benchmarks (GPQA-D, BFCL-v3, V*Bench) show that the trained 9B model achieves competitive memory-task performance while preserving general capabilities. Ablations isolate the contributions of active navigation, memory granularity, and RL training.
Significance. The paper makes a genuine conceptual contribution by formulating memory use as a learnable action-space problem rather than a fixed retrieval pipeline. The multi-granularity pyramid design is well-motivated, and the tool-based navigation interface is a clean abstraction. Strengths include: (1) thorough ablations (Table 4) that test each component independently; (2) rigorous judge validation (99% human agreement, Cohen's κ=0.98, with inter-annotator agreement reported); (3) results averaged over three seeds with error bars throughout; (4) evaluation on non-memory tasks to verify that memory-tool training does not induce spurious tool calls (Table 2); (5) storage and inference efficiency analyses (Table 3, Figure 3) that address practical deployment concerns. The tool-use behavior analysis (Tables 5, Figure 4) provides useful qualitative insights about how navigation strategies vary with model scale and training.
major comments (2)
- §4.2, Table 1: The headline claim that the 9B RL-trained model outperforms the untrained 397B model (62.74 vs 59.85 average) is dominated by in-domain results. Two of the three benchmarks (LoCoMo and PersonaMem-v2) share their dataset distributions with the RL training data (user-level 60/20/20 splits). The only genuinely out-of-distribution benchmark is LongMemEval (100 test questions), where the gap is 80.33±0.47 vs 78.33±2.08. The binomial standard error for a proportion near 0.80 on n=100 is approximately 4 percentage points, so this 2-point difference is well within sampling noise. The paper should either (a) report a statistical test for the LongMemEval comparison, or (b) explicitly qualify that the 9B-vs-397B advantage rests on in-domain generalization and that OOD performance is statistically inconclusive at the current sample size. As written, the abstract and §4.2 present the 9
- §3.4, reward rubric: The reward assigns r=1 only when the trajectory uses at least one memory tool (U=1) and r=0 for correct answers without tool use (U=0). While the paper notes that training queries are 'memory-intensive,' this design choice means the RL policy is explicitly trained to always invoke tools, which could inflate the apparent benefit of active navigation in the ablations. The 'w/o navigation' ablation (Table 4: 54.08 vs 62.74) compares against passive retrieval, which is a valid test of the full system design. However, the paper would be strengthened by briefly justifying why this asymmetric reward does not bias the comparison—for instance, by reporting what fraction of training queries have answers recoverable without memory tools, or by noting whether a symmetric reward (r=1 for correct regardless of U) was tried and how it affected navigation behavior.
minor comments (7)
- Appendix C, first paragraph: 'We train PyraNav with multi-turn GRPO' — 'PyraNav' appears to be a leftover name from an earlier draft and should be changed to 'NapMem' for consistency.
- Table 2: The GPQA-D accuracy for 'w/o RL' (55.74) and 'w/ RL' (57.58) both exceed the base model (53.03), but no error bars are reported for non-memory tasks, unlike the memory tasks in Table 1. Adding error bars or noting the number of seeds would help assess whether these improvements are significant.
- §4.1: The paper states that non-agentic baselines are allowed up to 20 retrieved items per retrieval operation, while NapMem returns up to 5. This asymmetry is reasonable (NapMem can make multiple targeted calls), but the paper should briefly discuss whether this could disadvantage baselines that benefit from larger context windows.
- Figure 3: The scatter plot would benefit from error bars or confidence intervals on the latency and token-count estimates, since they are computed over 100 samples.
- §4.4, Table 4: The 'w/o navigation' row reports 48.37 for LoCoMo, but the main Table 1 reports 59.92 for the full system. The paper should clarify what 'passive retrieval' means in this ablation—specifically, whether it uses the same retrieval indices and top-k as the search tools, or a different retrieval configuration.
- The paper uses 'NapMem-9B w/ RL' in Table 1 but 'NapMem-9B w/ RL' and 'NapMem-9B' (without RL) in Table 5. Consistent naming across tables would improve readability.
- References: Several arXiv identifiers use future-dated preprints (e.g., 2607, 2605, 2602, 2601, 2604). If these are genuine preprints, the dates should be verified; if they are placeholders, they should be corrected before camera-ready.
Simulated Author's Rebuttal
We thank the referee for the careful reading and constructive feedback. Both major comments are well-taken, and we will revise the manuscript accordingly.
read point-by-point responses
-
Referee: §4.2, Table 1: The headline claim that the 9B RL-trained model outperforms the untrained 397B model (62.74 vs 59.85 average) is dominated by in-domain results. Two of the three benchmarks (LoCoMo and PersonaMem-v2) share their dataset distributions with the RL training data (user-level 60/20/20 splits). The only genuinely out-of-distribution benchmark is LongMemEval (100 test questions), where the gap is 80.33±0.47 vs 78.33±2.08. The binomial standard error for a proportion near 0.80 on n=100 is approximately 4 percentage points, so this 2-point difference is well within sampling noise. The paper should either (a) report a statistical test for the LongMemEval comparison, or (b) explicitly qualify that the 9B-vs-397B advantage rests on in-domain generalization and that OOD performance is statistically inconclusive at the current sample size.
Authors: The referee is correct on both the factual point and the statistical reasoning. We confirm that LoCoMo and PersonaMem-v2 use user-level 60/20/20 splits for training, validation, and testing, so their held-out test sets are in-domain. LongMemEval is the only OOD benchmark, and with n=100 and a proportion near 0.80, the binomial standard error is indeed approximately 4 percentage points, making the 2-point gap (80.33 vs 78.33) statistically inconclusive. We will revise the manuscript in two ways: (1) We will add an explicit qualification in §4.2 and the abstract that the 9B-vs-397B comparison is dominated by in-domain generalization, and that OOD performance on LongMemEval is not statistically significant at the current sample size. (2) We will report a two-proportion z-test for the LongMemEval comparison in a footnote or table note. We agree that the current framing overstates the OOD evidence and will adjust accordingly. revision: yes
-
Referee: §3.4, reward rubric: The reward assigns r=1 only when the trajectory uses at least one memory tool (U=1) and r=0 for correct answers without tool use (U=0). While the paper notes that training queries are 'memory-intensive,' this design choice means the RL policy is explicitly trained to always invoke tools, which could inflate the apparent benefit of active navigation in the ablations. The paper would be strengthened by briefly justifying why this asymmetric reward does not bias the comparison—for instance, by reporting what fraction of training queries have answers recoverable without memory tools, or by noting whether a symmetric reward (r=1 for correct regardless of U) was tried and how it affected navigation behavior.
Authors: This is a fair concern. The asymmetric reward does explicitly incentivize tool use during training, and we should justify why this does not invalidate the ablation comparisons. We offer two points. First, the 'w/o navigation' ablation (Table 4: 54.08 vs 62.74) compares active navigation against passive retrieval over the same memory sources, so both conditions have access to memory; the comparison tests whether agent-controlled navigation improves over system-level retrieval, not whether tool use per se helps. The reward asymmetry affects the trained model but not the passive-retrieval baseline, so if the asymmetry were merely inflating tool calls without improving evidence quality, we would expect comparable or worse performance—instead, the trained model achieves higher accuracy with fewer tool calls (Table 5: 2.15 vs 3.97 average tool calls) and a higher evidence-hit ratio (34.92% vs 20.66%), suggesting the policy learns selective, evidence-directed navigation rather than indiscriminate tool invocation. Second, the non-memory benchmark results (Table 2) show that the trained model reduces unnecessary memory calls from 34.51% to 6.90% on GPQA-D and to 0% on BFCL-v3 and V*Bench, indicating that the policy does not blindly invoke tools on all inputs. That said, we agree that the manuscript should address this more explicitly. We will add a paragraph to §3.4 or §4.4 noting the rationale for the asymmetric reward, reporting the fraction of training queries where the base model can answer correctly without memory tools (we can compute this from our existing logs), and acknowledging the limitation that we have not yet run a symmetric-reward ablation. We will also note this as a direction for future work in the Limitations section. revision: partial
Circularity Check
No significant circularity; reward design is a methodological choice, not a circular derivation.
full rationale
The paper's claims are empirical, not derived from first principles. The reward function in §3.4 assigns r=1 only when the trajectory uses at least one memory tool (U=1), and r=0 for correct answers without tools. This means the RL-trained model is explicitly incentivized to use tools during training. The paper then shows (Table 4) that the full system outperforms a 'w/o navigation' passive-retrieval ablation (54.08 vs 62.74 avg). While this creates a mild internal-validity concern — the trained model was never rewarded for non-tool trajectories, so the ablation is not fully balanced — this is a design choice, not circularity. The paper is transparent about it: 'Since RL training uses memory-intensive queries, memory-tool usage is treated as a desired behavior during training.' The paper does not claim to derive that tool use is beneficial from the reward function; it presents an empirical comparison. Furthermore, Table 2 shows that on non-memory tasks, the RL model actually reduces unnecessary memory calls (34.51% → 6.90%), demonstrating the learned policy is selective rather than blindly forced. Self-citations (e.g., Xu et al., 2026b) appear only as background context in the introduction and are not load-bearing for any derivation. No prediction or result reduces to its inputs by construction. The OOD generalization concern (LongMemEval gap within noise) is a correctness risk, not circularity.
Axiom & Free-Parameter Ledger
free parameters (7)
- Tool-call budget (max 4 steps) =
4
- Retrieval top-k (5 items per search) =
5
- Reward weights (r values: -1, 1, 0, -0.5, -1) =
as specified
- Topic track limit (20) =
20
- Profile length budget (2000 chars) =
2000
- Record flush schedule (1→2→4→5 turns) =
1,2,4,5
- Topic/profile update triggers (>20 records, >50 records) =
20, 50
axioms (4)
- domain assumption GRPO with terminal rewards can effectively optimize multi-step tool-use trajectories for memory navigation.
- domain assumption Four memory record types (fact, event, instruction, preference) sufficiently cover user memory dimensions.
- domain assumption LLM-based memory pyramid construction (extraction, reconciliation, topic writing, profile updating) produces faithful representations of user history.
- domain assumption Existing benchmarks (LoCoMo, LongMemEval, PersonaMem-v2) adequately represent real-world long-term user memory requirements.
invented entities (2)
-
Memory pyramid (4-level structure)
independent evidence
-
Five memory tools (search_records, search_conversations, get_records, get_conversations, read_file)
independent evidence
read the original abstract
Long-term user memory is essential for personalized conversational agents, yet many memory systems still expose memory through passive retrieval interfaces, making the model a consumer of pre-selected evidence. We introduce NapMem, a framework for learning to use long-term user memory as a structured action space rather than passively retrieved context. NapMem organizes user history into a linked multi-granularity memory pyramid, where raw conversations, typed memory records, topic tracks, and user profiles are connected through provenance relations, and exposes these levels through memory tools. The agent is trained to select memory according to the query and intermediate evidence, allowing it to inspect different memory granularities before answering. Experiments on PersonaMem-v2, LongMemEval, and LoCoMo show that a NapMem agent trained with memory-tool reinforcement learning is competitive across diverse memory-intensive tasks, while evaluations on non-memory tasks suggest that the learned policy largely preserves general reasoning and tool-use abilities. Additional analyses examine storage, inference cost, tool-use behavior, and ablations over navigation, memory granularity, and RL training. Our results suggest that long-term user memory benefits from coupling structured storage with a learned policy for using memory at the appropriate granularity.
Reference graph
Works this paper leans on
-
[1]
MemFlow: Intent-Driven Memory Orchestration for Small Language Model Agents
Jiayi Chen, Yingcong Li, and Guiling Wang. Memflow: Intent-driven memory orchestration for small language model agents.arXiv preprint arXiv:2605.03312,
work page internal anchor Pith review Pith/arXiv arXiv
-
[2]
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
URLhttps://arxiv.org/abs/2504.19413. Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang. Lightmem: Lightweight and efficient memory- augmented generation,
work page internal anchor Pith review Pith/arXiv arXiv
-
[3]
LightMem: Lightweight and Efficient Memory-Augmented Generation
URLhttps://arxiv.org/abs/2510.18866. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, et al. Memory in the age of ai agents.arXiv preprint arXiv:2512.13564,
work page internal anchor Pith review Pith/arXiv arXiv
-
[4]
Bowen Jiang, Yuan Yuan, Maohao Shen, Zhuoqun Hao, Zhangchen Xu, Zichen Chen, Ziyi Liu, Anvesh Rao Vijjini, Jiashu He, Hanchao Yu, Radha Poovendran, Gregory Wornell, Lyle Ungar, Dan Roth, Sihao Chen, and Camillo Jose Taylor. Personamem-v2: Towards personalized intelligence via learning implicit user personas and agentic memory, 2025a. URLhttps://arxiv.org/...
-
[5]
URLhttps://arxiv. org/abs/2506.06326. Jingyi Kang, Chunyu Li, Ding Chen, Bo Tang, Feiyu Xiong, and Zhiyu Li. Memreader: From passive to active extraction for long-term agent memory.arXiv preprint arXiv:2604.07877,
work page internal anchor Pith review Pith/arXiv arXiv
-
[6]
MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models
URLhttps://arxiv.org/abs/ 2505.22101. Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. Simplemem: Efficient lifelong memory for llm agents.arXiv preprint arXiv:2601.02553,
work page internal anchor Pith review Pith/arXiv arXiv
-
[7]
doi: 10.18653/v1/2024.acl-long.747
Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.747. URLhttps://aclanthology.org/2024.acl-long. 747/. Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. Memgpt: Towards llms as operating systems,
-
[8]
MemGPT: Towards LLMs as Operating Systems
URLhttps://arxiv.org/abs/2310.08560. Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. InForty-second International Conference on Machine Learning,
work page internal anchor Pith review Pith/arXiv arXiv
-
[9]
Zep: A Temporal Knowledge Graph Architecture for Agent Memory
URLhttps://arxiv.org/abs/2501.13956. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark.arXiv preprint arXiv:2311.12022,
work page internal anchor Pith review Pith/arXiv arXiv
-
[10]
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300,
work page internal anchor Pith review Pith/arXiv arXiv
-
[11]
LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
URLhttps://arxiv.org/abs/2410.10813. Penghao Wu and Saining Xie. V?: Guided visual search as a core mechanism in multimodal llms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13084–13094, June
work page internal anchor Pith review Pith/arXiv arXiv
-
[12]
Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents.Advances in Neural Information Processing Systems, 38:17577–17604, 2026a. Yue Xu, Qian Chen, Zizhan Ma, Dongrui Liu, Wenxuan Wang, Xiting Wang, Li Xiong, and Wenjie Wang. Toward personalized llm-powered agents: Foundations, evaluation, and futu...
-
[13]
URL https://arxiv.org/abs/2508.19828. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,
work page internal anchor Pith review Pith/arXiv arXiv
-
[14]
URLhttps://arxiv.org/abs/2601.01885. Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, and Wenya Wang. Memskill: Learning and evolving memory skills for self-evolving agents, 2026a. URLhttps://arxiv.org/ abs/2602.02474. Yingyi Zhang, Pengyue Jia, Derong Xu, Yi Wen, Xianneng Li, Yichao Wang, Wenlin Zhang, Xiaopeng Li, Weinan Gan...
work page internal anchor Pith review Pith/arXiv arXiv
-
[15]
MemoryBank: Enhancing Large Language Models with Long-Term Memory
URLhttps://arxiv.org/abs/2305.10250. 11 From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space A. Dataset Split Details LoCoMo (Maharana et al., 2024).LoCoMo contains long multi-session conversations between two speakers, with questions that test long-term conversational memory. We convert each two-speaker ...
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[16]
LongMemEval evaluates long-term interactive memory across question types such as single-session recall, multi-session reasoning, temporal reasoning, knowledge updates, and preference- related questions. We use LongMemEval only as a held-out test benchmark to evaluate out-of-distribution generalization, with 100 test questions in total. PersonaMem-v2.(Jian...
work page 2023
-
[17]
We evaluate text-based tool use on a 200-example executable subset of BFCL-v3, consisting ofexec_simple, exec_multiple, andexec_parallel examples. Each example provides a user request, executable function schemas, and ground-truth tool calls. The evaluator executes the model-generated tool calls with deterministic local backends and returns the results to...
work page 2022
-
[18]
Writing requirements. Each topic track should be written as a Markdown document with metadata, a short summary, a narrative description, and an evidence section. The narrative should integrate new and existing information coherently rather than appending records mechanically. Provenance. Every integrated memory must be grounded with its supporting memory ...
work page 2000
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.