{"id":"e08f8b18-1dcb-4a35-b1ad-2327c76baf52","arxiv_id":"2607.05794","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":7,"one_line_summary":"Training a 9B LLM agent with reinforcement learning to actively navigate a structured multi-granularity memory pyramid yields competitive performance on memory-intensive benchmarks while preserving non-memory capabilities.","lead":"The paper introduces NapMem, which trains an LLM agent to actively navigate a multi-level memory pyramid (raw conversations, records, topic tracks, profiles) using tools, rather than passively receiving retrieved context. A 9B model trained this way outperforms larger untrained models on memory-intensive tasks while preserving general reasoning abilities.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The headline claim that 9B-RL outperforms 397B rests primarily on in-domain benchmarks; the only OOD test (LongMemEval, 100 questions) shows a 2-point gap within statistical noise.","rationale":"The reader correctly identified data overlap as a secondary concern but focused primarily on the reward function's tool-use requirement as the weakest assumption. I find the reward function concern less load-bearing than the reader does. The RL training incentivizing tool use is part of the proposed method — the 'w/o navigation' ablation tests whether the full system (tools + RL + pyramid) outperforms passive retrieval over the same memory sources, which is a legitimate comparison regardless of reward design. The model also demonstrates calibrated tool use on non-memory tasks (Table 2: memory call rate drops from 34.51% to 6.90% on GPQA-D after RL), suggesting the reward does not simply force indiscriminate tool use. The more fundamental issue is evaluation scope: the headline result (9B > 397B) is not robustly demonstrated on out-of-distribution data. Two of three benchmarks are in-domain, and the only OOD benchmark has 100 questions where the 2-point gap is within noise. The paper is transparent about splits and acknowledges LongMemEval as OOD, but the framing of 'outperforms untrained 122B and 397B variants' does not adequately distinguish in-domain from OOD performance. The paper has real strengths: thorough ablations (Table 4), non-memory task evaluation (Table 2), efficiency analysis (Table 3, Figure 3), tool-use behavior analysis (Tables 5, Figures 4-6), and judge validation (Table 8). The memory pyramid design is well-motivated and the engineering appears solid. But the central empirical claim would be substantially strengthened by either a larger OOD benchmark or statistical testing confirming the LongMemEval difference is real. The name inconsistency (PyraNav in Appendix C vs NapMem elsewhere) and lack of code release are minor issues. The verdict should remain CONDITIONAL, as the reader assessed, but the primary concern should be reframed around OOD evaluation strength rather than reward function design.","tokens_in":17457,"tokens_out":5366,"duration_ms":395290,"concrete_test":"Bootstrap-resample the 100 LongMemEval questions (10,000 iterations, paired by question) to compute a 95% confidence interval for the difference between 9B-RL and 397B L-J accuracy. If the 95% CI includes zero, the OOD claim of outperforming 397B is not supported, and the headline should be qualified to note that the advantage is limited to in-domain benchmarks. Additionally, report per-question agreement: how many of the 100 questions does 9B-RL get right that 397B gets wrong, and vice versa.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim is that a 9B RL-trained model (avg 62.74) outperforms an untrained 397B model (avg 59.85). This average is computed across three benchmarks, but two (LoCoMo and PersonaMem-v2) share their dataset distributions with the RL training data. The paper uses user-level 60/20/20 splits to avoid direct leakage, but the model has still learned the task format, question patterns, and memory structure of these benchmarks during training. The only genuinely out-of-distribution benchmark is LongMemEval, with 100 test questions. On LongMemEval, 9B-RL scores 80.33±0.47 vs 397B's 78.33±2.08 — a 2-point difference on 100 questions. The binomial standard error for a proportion near 0.80 on n=100 is approximately 4 percentage points, so this gap is well within sampling noise. The 397B model's higher variance (±2.08) further underscores this. If the LongMemEval difference is not statistically significant, the claim that 9B-RL 'outperforms' 397B depends entirely on in-domain generalization. The reader's concern about the reward function (§3.4) requiring tool use (U=1 for r=1, vs r=0 for correct-without-tools) is a reasonable observation about internal validity, but it is less load-bearing: the RL training incentivizing tool use is part of the proposed method, and the 'w/o navigation' ablation (Table 4: 54.08 vs 62.74) is a valid test of whether the full system design outperforms passive retrieval. Even if the reward design is perfectly sound, the headline comparison with 397B is not robustly supported on OOD data.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper introduces NapMem, a framework that reframes long-term user memory from passive retrieval into a structured action space. The system organizes user history into a four-level memory pyramid (raw conversations, typed memory records, topic tracks, user profiles) connected by provenance links, and exposes these levels through five memory tools. A 9B base model is trained with GRPO to learn a navigation policy that selects which memory granularity to access based on the query and intermediate evidence. Experiments on three memory-intensive benchmarks (LoCoMo, LongMemEval, PersonaMem-v2) and three non-memory benchmarks (GPQA-D, BFCL-v3, V*Bench) show that the trained 9B model achieves competitive memory-task performance while preserving general capabilities. Ablations isolate the contributions of active navigation, memory granularity, and RL training.","tokens_in":17761,"tokens_out":3092,"duration_ms":145136,"significance":"The paper makes a genuine conceptual contribution by formulating memory use as a learnable action-space problem rather than a fixed retrieval pipeline. The multi-granularity pyramid design is well-motivated, and the tool-based navigation interface is a clean abstraction. Strengths include: (1) thorough ablations (Table 4) that test each component independently; (2) rigorous judge validation (99% human agreement, Cohen's κ=0.98, with inter-annotator agreement reported); (3) results averaged over three seeds with error bars throughout; (4) evaluation on non-memory tasks to verify that memory-tool training does not induce spurious tool calls (Table 2); (5) storage and inference efficiency analyses (Table 3, Figure 3) that address practical deployment concerns. The tool-use behavior analysis (Tables 5, Figure 4) provides useful qualitative insights about how navigation strategies vary with model scale and training.","major_comments":[{"comment":"§4.2, Table 1: The headline claim that the 9B RL-trained model outperforms the untrained 397B model (62.74 vs 59.85 average) is dominated by in-domain results. Two of the three benchmarks (LoCoMo and PersonaMem-v2) share their dataset distributions with the RL training data (user-level 60/20/20 splits). The only genuinely out-of-distribution benchmark is LongMemEval (100 test questions), where the gap is 80.33±0.47 vs 78.33±2.08. The binomial standard error for a proportion near 0.80 on n=100 is approximately 4 percentage points, so this 2-point difference is well within sampling noise. The paper should either (a) report a statistical test for the LongMemEval comparison, or (b) explicitly qualify that the 9B-vs-397B advantage rests on in-domain generalization and that OOD performance is statistically inconclusive at the current sample size. As written, the abstract and §4.2 present the 9","section":null},{"comment":"§3.4, reward rubric: The reward assigns r=1 only when the trajectory uses at least one memory tool (U=1) and r=0 for correct answers without tool use (U=0). While the paper notes that training queries are 'memory-intensive,' this design choice means the RL policy is explicitly trained to always invoke tools, which could inflate the apparent benefit of active navigation in the ablations. The 'w/o navigation' ablation (Table 4: 54.08 vs 62.74) compares against passive retrieval, which is a valid test of the full system design. However, the paper would be strengthened by briefly justifying why this asymmetric reward does not bias the comparison—for instance, by reporting what fraction of training queries have answers recoverable without memory tools, or by noting whether a symmetric reward (r=1 for correct regardless of U) was tried and how it affected navigation behavior.","section":null}],"minor_comments":[{"comment":"Appendix C, first paragraph: 'We train PyraNav with multi-turn GRPO' — 'PyraNav' appears to be a leftover name from an earlier draft and should be changed to 'NapMem' for consistency.","section":null},{"comment":"Table 2: The GPQA-D accuracy for 'w/o RL' (55.74) and 'w/ RL' (57.58) both exceed the base model (53.03), but no error bars are reported for non-memory tasks, unlike the memory tasks in Table 1. Adding error bars or noting the number of seeds would help assess whether these improvements are significant.","section":null},{"comment":"§4.1: The paper states that non-agentic baselines are allowed up to 20 retrieved items per retrieval operation, while NapMem returns up to 5. This asymmetry is reasonable (NapMem can make multiple targeted calls), but the paper should briefly discuss whether this could disadvantage baselines that benefit from larger context windows.","section":null},{"comment":"Figure 3: The scatter plot would benefit from error bars or confidence intervals on the latency and token-count estimates, since they are computed over 100 samples.","section":null},{"comment":"§4.4, Table 4: The 'w/o navigation' row reports 48.37 for LoCoMo, but the main Table 1 reports 59.92 for the full system. The paper should clarify what 'passive retrieval' means in this ablation—specifically, whether it uses the same retrieval indices and top-k as the search tools, or a different retrieval configuration.","section":null},{"comment":"The paper uses 'NapMem-9B w/ RL' in Table 1 but 'NapMem-9B w/ RL' and 'NapMem-9B' (without RL) in Table 5. Consistent naming across tables would improve readability.","section":null},{"comment":"References: Several arXiv identifiers use future-dated preprints (e.g., 2607, 2605, 2602, 2601, 2604). If these are genuine preprints, the dates should be verified; if they are placeholders, they should be corrected before camera-ready.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The paper is a solid systems contribution with thorough experimentation. The main issue is statistical: the OOD comparison that supports the headline '9B outperforms 397B' claim is underpowered. This is fixable with better framing and possibly additional OOD evaluation. The reward design concern is worth raising but is not a fatal flaw—it is a design choice that the authors should justify, not an error. I would not recommend major revision; the core framework and in-domain results are sound."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for the careful reading and constructive feedback. Both major comments are well-taken, and we will revise the manuscript accordingly.","responses":[{"response":"The referee is correct on both the factual point and the statistical reasoning. We confirm that LoCoMo and PersonaMem-v2 use user-level 60/20/20 splits for training, validation, and testing, so their held-out test sets are in-domain. LongMemEval is the only OOD benchmark, and with n=100 and a proportion near 0.80, the binomial standard error is indeed approximately 4 percentage points, making the 2-point gap (80.33 vs 78.33) statistically inconclusive. We will revise the manuscript in two ways: (1) We will add an explicit qualification in §4.2 and the abstract that the 9B-vs-397B comparison is dominated by in-domain generalization, and that OOD performance on LongMemEval is not statistically significant at the current sample size. (2) We will report a two-proportion z-test for the LongMemEval comparison in a footnote or table note. We agree that the current framing overstates the OOD evidence and will adjust accordingly.","revision_made":"yes","referee_comment":"§4.2, Table 1: The headline claim that the 9B RL-trained model outperforms the untrained 397B model (62.74 vs 59.85 average) is dominated by in-domain results. Two of the three benchmarks (LoCoMo and PersonaMem-v2) share their dataset distributions with the RL training data (user-level 60/20/20 splits). The only genuinely out-of-distribution benchmark is LongMemEval (100 test questions), where the gap is 80.33±0.47 vs 78.33±2.08. The binomial standard error for a proportion near 0.80 on n=100 is approximately 4 percentage points, so this 2-point difference is well within sampling noise. The paper should either (a) report a statistical test for the LongMemEval comparison, or (b) explicitly qualify that the 9B-vs-397B advantage rests on in-domain generalization and that OOD performance is statistically inconclusive at the current sample size."},{"response":"This is a fair concern. The asymmetric reward does explicitly incentivize tool use during training, and we should justify why this does not invalidate the ablation comparisons. We offer two points. First, the 'w/o navigation' ablation (Table 4: 54.08 vs 62.74) compares active navigation against passive retrieval over the same memory sources, so both conditions have access to memory; the comparison tests whether agent-controlled navigation improves over system-level retrieval, not whether tool use per se helps. The reward asymmetry affects the trained model but not the passive-retrieval baseline, so if the asymmetry were merely inflating tool calls without improving evidence quality, we would expect comparable or worse performance—instead, the trained model achieves higher accuracy with fewer tool calls (Table 5: 2.15 vs 3.97 average tool calls) and a higher evidence-hit ratio (34.92% vs 20.66%), suggesting the policy learns selective, evidence-directed navigation rather than indiscriminate tool invocation. Second, the non-memory benchmark results (Table 2) show that the trained model reduces unnecessary memory calls from 34.51% to 6.90% on GPQA-D and to 0% on BFCL-v3 and V*Bench, indicating that the policy does not blindly invoke tools on all inputs. That said, we agree that the manuscript should address this more explicitly. We will add a paragraph to §3.4 or §4.4 noting the rationale for the asymmetric reward, reporting the fraction of training queries where the base model can answer correctly without memory tools (we can compute this from our existing logs), and acknowledging the limitation that we have not yet run a symmetric-reward ablation. We will also note this as a direction for future work in the Limitations section.","revision_made":"partial","referee_comment":"§3.4, reward rubric: The reward assigns r=1 only when the trajectory uses at least one memory tool (U=1) and r=0 for correct answers without tool use (U=0). While the paper notes that training queries are 'memory-intensive,' this design choice means the RL policy is explicitly trained to always invoke tools, which could inflate the apparent benefit of active navigation in the ablations. The paper would be strengthened by briefly justifying why this asymmetric reward does not bias the comparison—for instance, by reporting what fraction of training queries have answers recoverable without memory tools, or by noting whether a symmetric reward (r=1 for correct regardless of U) was tried and how it affected navigation behavior."}],"tokens_in":17215,"tokens_out":1281,"duration_ms":56797,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The main thing to know: this paper builds a four-level user-memory pyramid (raw conversations, typed records, topic tracks, user profiles), exposes it through five granularity-specific tools, and trains a 9B model with GRPO to navigate it. The trained 9B model averages 62.74 across three memory benchmarks, beating an untrained 397B model at 59.85. That is a real result worth taking seriously — a small model with learned memory access outperforming a much larger model with passive retrieval, while preserving non-memory task performance (Table 2 is clean on this point). The ablations in Table 4 are thorough: each component contributes, the judge validation is solid (99% human agreement, κ=0.98), and results are averaged over three seeds with error bars. The tool-use behavior analysis (Tables 5, Figure 4) showing that RL makes navigation more selective and evidence-hit rates higher is genuinely useful empirical content. The framework itself — coupling structured multi-granularity storage with a learned navigation policy — is a legitimate new contribution not present in the cited prior work (AgeMem, Memory-R1, MemFlow all do RL for memory but not this specific pyramid-plus-tool-navigation design). The case studies in Appendix D are honest about failure modes, which I appreciate. Now the soft spots. The reader's circularity concern about the reward function (U=1 required for r=1) is real but I think less load-bearing than it sounds. The reward incentivizes tool use during training, yes, but the 'w/o navigation' ablation (Table 4: 54.08 vs 62.74) still tests whether the full system beats passive retrieval over the same memory sources — that comparison is valid as a system-level test. The more serious issue is the OOD evidence. Two of three benchmarks (LoCoMo, PersonaMem-v2) share training distributions; LongMemEval is the only held-out test, with 100 questions. On LongMemEval, 9B-RL scores 80.33±0.47 vs 397B's 78.33±2.08 — a 2-point gap on 100 questions, which is within binomial sampling noise (~4pp SE near p=0.80). So the headline '9B beats 397B' depends on in-domain generalization. The paper should be more upfront about this. The name inconsistency (PyraNav in Appendix C vs NapMem elsewhere) is minor but suggests rushed revision. No code or data released, which limits reproducibility. These are addressable concerns, not structural flaws. The core engineering is sound, the ablations are honest, and the framework is a genuine contribution to the memory-augmented LLM space. This paper deserves a serious referee who can push the authors to (1) add at least one more OOD benchmark or expand LongMemEval, (2) release code, and (3) soften the headline claim to reflect that the 9B-vs-397B advantage is primarily in-domain. Recommend conditional accept pending those revisions.","headline":"A 9B model trained with RL over a structured memory pyramid beats 10-40x larger untrained models on memory-intensive tasks — but the headline rests on in-domain benchmarks, and the OOD evidence is thin.","tokens_in":18647,"tokens_out":726,"would_cite":true,"duration_ms":124937,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"9B agent beats 397B on memory tasks by learning to navigate, not retrieve","keywords":["long-term memory","conversational agents","reinforcement learning","memory navigation","multi-granularity memory","tool use","personalization"],"falsifier":"If a variant of the reward function that equally rewards correct answers without tool use were trained, and the active-navigation advantage over passive retrieval disappeared or shrank substantially, the paper's central claim that learned navigation is the source of the gains would be undermined.","tokens_in":17688,"feed_emoji":"🧭","tokens_out":1106,"duration_ms":126417,"temperature":0.7,"pith_summary":"The paper argues that long-term user memory in conversational AI agents should be treated not as a passive retrieval problem—where a system pre-selects evidence and hands it to the model—but as an active navigation problem, where the agent itself decides which memory layer to consult, when to drill deeper, and when it has gathered enough evidence to answer. The authors instantiate this claim with NapMem, a framework that organizes each user's interaction history into a four-level memory pyramid (raw conversations, typed memory records, topic tracks, and a user profile) connected by provenance links, and exposes each level through dedicated tools. A 9B-parameter language model trained with reinforcement learning to use these tools achieves the best average score (62.74) across three memory-intensive benchmarks, outperforming untrained 122B and 397B variants of the same architecture, while largely preserving performance on non-memory reasoning and tool-use tasks. The central mechanism is the coupling of structured multi-granularity storage with a learned policy that selects the appropriate memory abstraction level per query, rather than relying on a fixed retrieval pipeline.","feed_headline":"9B agent beats 397B on memory tasks by learning to navigate, not retrieve","feed_subtitle":"A small model trained to actively explore a four-level memory pyramid outperforms models 40x larger, suggesting memory use is a learnable技能.","key_machinery":"The multi-granularity memory pyramid with five navigation tools (search_conversations, get_conversations, search_records, get_records, read_file), trained via Group Relative Policy Optimization (GRPO) with a terminal reward that jointly optimizes answer correctness, format validity, and memory-tool usage.","core_discovery":"The paper's central finding is that a relatively small model (9B parameters) trained to actively navigate a structured memory pyramid—choosing among raw conversations, compact records, topic tracks, and user profiles via tool calls—can outperform models more than 40 times larger that use the same memory sources but without a learned navigation policy. The ablation showing that removing active navigation (reverting to passive retrieval over the same memory) drops the average score from 62.74 to 54.08, and that removing RL training (keeping the tools but using the untrained base model) drops it to 48.39, isolates the contribution of the learned navigation policy from the contribution of the存储.","pith_inferences":["The reward function gives r=1 only to correct answers that use at least one memory tool, and r=0 to correct answers that skip tools. This means the RL training explicitly biases the policy toward tool use, which could inflate the apparent benefit of active navigation in the ablation comparison against passive retrieval. A reward that also rewards correct no-tool answers would provide a cleaner tes","If the pyramid structure and navigation policy are both learnable end-to-end (rather than the pyramid being constructed by separate prompts), the system might discover memory granularities that do not correspond to the four hand-designed levels, potentially improving or simplifying the hierarchy.","The four-tool-call budget per query may constrain the navigation depth for complex multi-hop reasoning; analyzing failure cases that exhaust the budget could reveal whether the policy is choosing well within its limits or hitting an artificial ceiling."],"forward_implications":["If active memory navigation generalizes beyond the tested benchmarks, conversational agents could achieve strong personalization with much smaller models, reducing deployment cost for long-running assistant applications.","The finding that RL training reduces unnecessary memory calls on non-memory tasks (from 34.5% to 6.9% on GPQA-D) suggests that learned memory-use policies can self-calibrate, avoiding the over-retrieval problem common in memory-augmented systems.","The pyramid structure with provenance links between levels could serve as a template for other hierarchical information-access problems beyond user memory, such as codebase navigation or document review.","The observation that larger models prefer top-down navigation (starting from profiles) while smaller models prefer bottom-up (starting from records) raises questions about whether navigation strategy should be conditioned on model capacity in deployed systems."],"fun_headline_variants":["9B model outperforms 397B by navigating structured memory instead of passively retrieving","Treating memory as action space lets 9B agent beat 40x larger models on memory tasks","Active memory navigation beats passive retrieval: 62.74 vs 54.08 with same storage","Learned memory navigation policy drives 9B model past 397B rivals on long-term recall","RL-trained memory navigation lifts 9B agent above models 40x its size"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The RL reward function requires at least one memory-tool call for a trajectory to receive the highest reward, which means the training explicitly forces tool use. This makes the comparison between active navigation and passive retrieval not fully balanced, since the trained model was never rewarded for answering correctly without tools.","fun_headline_variants_meta":{"raw":{"variants":["9B model outperforms 397B by navigating structured memory instead of passively retrieving","Treating memory as action space lets 9B agent beat 40x larger models on memory tasks","Active memory navigation beats passive retrieval: 62.74 vs 54.08 with same storage","Learned memory navigation policy drives 9B model past 397B rivals on long-term recall","RL-trained memory navigation lifts 9B agent above models 40x its size"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":713,"prompt_tokens":594,"completion_tokens":119,"prompt_tokens_details":null},"tokens_in":594,"tokens_out":119,"duration_ms":46516,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T23:43:50.182866+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If a variant of the reward function that equally rewards correct answers without tool use were trained, and the active-navigation advantage over passive retrieval disappeared or shrank substantially, the paper's central claim that learned navigation is the source of the gains would be undermined.","supporting_citations":[],"review_version":1}