Pith. sign in

REVIEW 4 major objections 5 minor 14 references

CMI-Mem claims that a memory manager trained with QA correctness plus a conditional-mutual-information reward transfers better across memory-use scenarios than QA-only training.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 03:34 UTC pith:SCOW3CAM

load-bearing objection CMI-Mem's core ablation is credible, but the headline OOD gain is mostly the retrieval-indexed architecture, not the CMI reward. the 4 major comments →

arxiv 2607.20553 v1 pith:SCOW3CAM submitted 2026-07-15 cs.AI cs.CL

CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning

Yubo Wang , Qiuyu Zhao , Zenghui Sun , Shichao Dong , Jinsong Lan , Xiaoyong Zhu , Haoyang Li , Bo Zheng
show 1 more author
Lei Chen
This is my paper
classification cs.AI cs.CL
keywords memory managementreinforcement learningconditional mutual informationLLM agentslong-term memoryintrinsic rewardreward shapingselective forgetting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that memory management for language agents can be trained with a criterion intrinsic to memory itself, rather than one derived entirely from sampled questions and a downstream reader. It proposes CMI-Mem, which augments a QA-correctness reward with a conditional-mutual-information reward that scores, at every memory operation, how much informational novelty a proposed fragment adds relative to the current dialogue and existing memory. The claim is that this hybrid signal generalizes across memory-use scenarios—recall, temporal reasoning, preference grounding, selective forgetting—and that per-operation feedback also stabilizes training and shortens rollouts. If correct, it would give agent memory a usable intrinsic quality signal instead of making memory value hostage to whichever questions happen to be generated. The paper itself flags that its evidence is limited to 4B/8B backbones, that multimodal quality is bounded by the backbone, and that remaining QA-centric benchmarks may under-demonstrate the method.

Core claim

The paper's central claim is that memory value can be scored before a future question is known. It defines the value of an action as the information it contributes about the evolving memory state beyond what is already stored, R_t = I(M_t; A_t | M_{t-1}), and implements this as the conditional mutual information I(C_t; m_new | M_local) between the dialogue context and the candidate memory fragment, conditioned on a local set of existing memories. The CMI term is estimated by residual projection in embedding space (exact under a jointly Gaussian assumption), clamped, Gaussian-shaped, and blended with a QA-correctness reward. In the experiments the hybrid reward is reported to outperform QA-on

What carries the argument

The load-bearing object is the action-conditioned CMI reward r^CMI_t = I(C_t; m_new | M_local), estimated by residual projection: the current dialogue embedding and the candidate fragment embedding are each linearly regressed out of the local memory matrix, and their residual cosine similarity serves as the CMI estimate. The estimate is clamped, passed through a Gaussian shaping function, and blended with QA reward via a mixing weight alpha. Operation-specific scoring makes ADD/MERGE reward absolute information content and REPLACE/UPDATE reward the net gain over the old fragment, so a single estimator routes all memory actions.

Load-bearing premise

The load-bearing premise is the unproved equivalence between the information an action adds to the memory state, I(M_t; A_t | M_{t-1}), and the residual similarity of the dialogue to the policy's own proposed fragment, I(C_t; m_new | M_local); if that equivalence fails, the reward is a semantic-similarity heuristic rather than a true information-theoretic criterion.

What would settle it

Train the full CMI-Mem pipeline twice, once with the true local conditioning set M_local and once with conditioning removed entirely (unconditional cosine similarity), keeping QA reward and all other components fixed; if out-of-distribution gains persist without conditioning, the conditional-information mechanism is not doing the work. On fixed dialogues, also compare the residual-projection values with a plug-in entropy estimate of I(C_t; m_new | M_{t-1}); a low rank correlation would show the estimator is not measuring the named quantity.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Hybrid-trained memory managers should transfer to memory uses that were not covered by the training questions, since the intrinsic term does not condition on a sampled query.
  • Per-operation CMI feedback makes credit assignment denser, so training should converge with shorter rollouts and smaller memory banks.
  • The CMI signal should favor updating or replacing stale facts, since repeating already-stored information has low conditional mutual information.
  • CMI alone is not a sufficient objective; the ablation result says it needs the QA anchor, so the two rewards are complements, not substitutes.
  • The retrieval-indexed memory design and the reward are separable contributions; the paper claims each adds on top of the other.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the residual-projection estimator truly tracks conditional mutual information, the same reward could be applied to other agent decisions where value is the novelty of an action relative to an evolving state—tool selection, context summarization, or knowledge-base updates.
  • Because the estimator's information is measured in a fixed embedding space, the 'information' is only what that encoder can represent; swapping the encoder and observing whether the transfer gap changes would be a direct test of that dependence.
  • The paper's own flagged limitations—no 14B+ training, backbone-bounded multimodal performance, and QA-centric benchmarks—suggest its strongest evidence is currently on text-only, moderately sized models.
  • An appendix note records that an early Core action-space variant included a SKIP action later removed, so readers reproducing the system should rely on the final three-action Core interface; the paper also excludes a counterfactual multi-hop subset, arguing answer-model world knowledge confounds memory fidelity, which leaves reader-independent memory metrics as an open need.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CMI-Mem, an RL-trained memory manager that combines a session-level QA reward with a per-operation intrinsic reward derived from conditional mutual information (CMI) between the current dialogue and the proposed memory fragment given local stored memory. The manager uses a four-slot structured memory and retrieval-indexed provenance; it is trained with GRPO on LongMemEval and evaluated on LongMemEval, LoCoMo, and MemoryAgentBench. The main empirical claims are: (i) CMI+QA training outperforms QA-only and CMI-only training (Table 3: 68.5/67.0 vs 56.5/45.0 on LongMemEval); (ii) CMI-Mem-4B transfers better than MemBuilder-RL-4B to out-of-distribution MemoryAgentBench (+11.0); and (iii) per-operation CMI feedback shortens rollouts and reduces memory bank size. The paper also releases code and a checkpoint.

Significance. The in-domain complementarity result, if robust, is a useful contribution: it questions the sufficiency of QA-only memory rewards and offers a query-independent shaping signal. Strengths include the public code/checkpoint, the attempt to isolate the retrieval-indexed architecture through Ours-P, the detailed ablation hierarchy, transparent documentation of metric changes, and qualitative case studies linking reward design to memory behavior. However, the OOD transfer claim is currently confounded: the reward-isolated comparison is run only on LongMemEval, and on MemoryAgentBench Ours-P (same architecture, no RL) is only 0.3 points below CMI-Mem. The information-theoretic derivation is also asserted rather than derived. With the missing decoupled ablations and a corrected interpretation of Eqs. (4)-(5), the paper would be significantly stronger.

major comments (4)
  1. [§3.3, Eqs. (4)–(9)] The transition Eq. (4) → Eq. (5) is called 'exact' but no equality proof is given. Eq. (4) conditions on the full previous state M_{t-1} and measures I(M_t; A_t | M_{t-1}); Eq. (5) replaces the updated state by the current dialogue C_t and conditions on a hand-picked subset M^local. Determinism of Apply does not imply these quantities are equal, and A_t includes the action type (ADD/REPLACE/...) that Eq. (5) drops. Moreover, Eqs. (7)–(9) compute a cosine between residuals without centering or an intercept, so the asserted identity with Pearson partial correlation in Eq. (6) does not hold in general; Eq. (6) is exact only under joint Gaussianity of the raw variables, not of L2-normalized embeddings. The paper should either prove the equality under explicit assumptions or present Eqs. (4)–(5) as a shaped semantic-similarity heuristic, and should add a control ablation that removes the cond
  2. [§4.2.2, Table 2] The +11.0 gain of CMI-Mem-4B over MemBuilder-RL-4B on MemoryAgentBench is attributed to the CMI reward, but the comparison is confounded by the retrieval-indexed memory (RM) architecture. CMI-Mem and Ours-P share RM, while the MemBuilder-RL reimplementation is described without it. Ours-P (Qwen3-4B, prompting only, RM) scores 45.8, only 0.3 below CMI-Mem-4B (46.1); RM alone therefore accounts for nearly the entire gap. The only reward-isolated ablation (Table 3) is on LongMemEval, not on the OOD benchmark. To support the title-level transfer claim, report CMI+QA and QA-only results on MemoryAgentBench with and without RM (plus Ours-P/RL combinations).
  3. [§3.3, Eq. (5) and Appendix F] The CMI reward is computed on the policy's own proposed fragment, and the local conditioning set is retrieved using that same fragment as part of the query (Eq. (17), w_new=0.5). This makes the reward partially self-referential and gameable: the policy can increase r^CMI by emitting text that is similar to the dialogue but far from the retrieved local set. The Gaussian shaper only prefers mid-range values and the QA term anchors the total reward, but the paper's mechanistic interpretation ('high CMI indicates novelty relative to stored memories') is not directly validated. Since CMI-only training collapses to 45.0 in Table 3, the intrinsic signal is clearly not a stable objective by itself; the authors should test possible reward hacking, e.g., by checking correlation between r^CMI and human judgments of memory informativeness.
  4. [Appendix J; Table 2] The MemoryAgentBench evaluation deviates from the official protocol: most columns use an LLM judge instead of substring exact match, some tasks keep Recall@5/ROUGE-L, and the FC-MH subset is excluded. These choices are reasonable and well documented, but unless every baseline in Table 2 was re-scored under exactly this protocol, comparisons to published numbers (e.g., Memory-R1, MemBuilder-RL-8B) are not apples-to-apples. The text states that all systems use Qwen3.7-Max as answer/judge, but it does not explicitly confirm that the published baselines were re-run rather than copied. Please state this explicitly and, if feasible, report the official-metric aggregate as a secondary row for the main systems and Ours-P.
minor comments (5)
  1. [Table 4] Notation is inconsistent: Table 4 lists γ as the Gaussian shaper center and σ_shape as the width, while Section 3.3 uses µ and σ. Align the notation.
  2. [Appendix H.2] The paragraph explaining that CORE:SKIP is a 'stale leftover' from an earlier draft should be removed or moved to a design-notes section; in a published prompt template, such meta-commentary is confusing.
  3. [§4.2.1] The citation to Wang et al. (2026) (AgRAG) appears unrelated to the sentence about RAG methods underperforming; either connect it to the memory discussion or delete the citation.
  4. [Table 3] The row label 'Qwen3-4B' is ambiguous; clarify that it is the base LLM with the 4-partition memory structure but no RL/RM.
  5. [Eq. (12)] The 'per-step reward' mixes a per-operation CMI signal with a session-level QA scalar; state explicitly that the QA term is broadcast across the session's steps.

Circularity Check

0 steps flagged

No significant circularity: the CMI reward is an empirically evaluated training signal, not a prediction forced by its own definition; the only self-citation is peripheral and not load-bearing.

full rationale

The paper's central derivation chain is not circular in the sense of reducing a predicted result to its own inputs. Equation (4) defines a target information-theoretic quantity, and Equation (5) is an explicitly stated implementation/approximation of that quantity using a different variable set (Ct and a local memory subset). Whether this substitution is mathematically justified is a correctness and validity concern, not a circularity concern: the reward is defined, used for RL training, and then evaluated on downstream QA benchmarks that are external to the reward definition. The ablation in Table 3 directly tests the reward's contribution against QA-only and CMI-only variants on LongMemEval, so the claimed complementarity is empirically supported rather than definitionally forced. The only self-citation is the peripheral AgRAG reference (Wang et al., 2026) in Section 4.2.1, used to interpret why RAG is competitive on the LRU subset; it does not carry the paper's central argument. The OOD attribution concern raised by the reader—that the +11.0 gain over MemBuilder-RL on MemoryAgentBench may be largely explained by the retrieval-indexed architecture rather than the CMI reward—is a legitimate experimental-confounding critique, but it is not a circularity: the numbers are reported, not derived from the method's equations. No fitted parameter is renamed as a prediction, and no load-bearing result depends on a self-citation. Score 1 reflects the presence of one peripheral non-load-bearing self-citation; no circular step meets the evidentiary bar for a higher score.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 0 invented entities

The central claim rests on an asserted information-theoretic identity (Eq. 4 -> Eq. 5), a strong Gaussian assumption on embeddings, and multiple hand-tuned reward/retrieval parameters (Gaussian mu/sigma, alpha, lambda, retrieval weights/sizes). Entity-wise nothing new is invented; the CMI reward is a function rather than a new physical or model entity.

free parameters (7)
  • Gaussian shaping center mu = 0.35
    Calibrated via iterative reward-distribution monitoring across multiple training runs to maximize stable convergence (Section 4.1, Eq. 11).
  • Gaussian shaping width sigma = 0.15
    Same calibration procedure as mu; controls the margin between redundant and spuriously novel fragments.
  • Reward mixing weight alpha = 0.3
    Blends CMI and QA rewards in Eq. 12; ablation shows alpha>0.3 degrades performance, so the value is task-tuned.
  • Tikhonov regularization lambda = 1e-4
    Prevents ill-conditioning in residual projection (Eqs. 7-8); chosen by hand.
  • Local conditioning retrieval weights w_C, w_new, w_old = 0.5, 0.5, 0.3
    Weights in Eq. 17 for constructing M_local; 'tuned per-operation type' (Appendix F).
  • Local conditioning set sizes = recent=15, top-sim=5, local top-k=8, kr=2
    Counts for temporal recency and topical similarity in the CMI conditioning set (Appendix F, Table 7).
  • Curriculum difficulty weights w_s, w_c, w_q = not reported
    Tunable weights in Eq. 14 controlling session-depth, fragment-count, and question-count contributions; exact values not stated in the main text.
axioms (4)
  • domain assumption Joint Gaussian assumption on sentence embeddings
    Eqs. 6-9 equate the CMI to -1/2 log(1-rho^2) using the residual cosine as a partial correlation. Text embeddings are not jointly Gaussian; this is an unproven distributional assumption that grounds the 'information-theoretic' reward.
  • ad hoc to paper Eq. (4) equals Eq. (5): I(M_t; A_t | M_{t-1}) is implemented as I(C_t; m_new | M_local)
    Section 3.3 replaces the updated memory state M_t with the dialogue context C_t and introduces a local conditioning subset, labeling the result 'exact'. No derivation shows these quantities are equal; the entire reward definition rests on this asserted identity.
  • domain assumption Embedding cosine similarity in Qwen3-Embedding space captures semantic information relevant to memory utility
    The CMI estimator, retrieval, and residual projections all assume that L2-normalized embeddings of dialogues and memory fragments faithfully represent the information relevant to future memory use.
  • domain assumption Conditional mutual information is a valid normative criterion for memory management
    The paper's thesis that high CMI favors relevant/novel content and low CMI signals redundancy is assumed as the design principle; it is not derived from downstream utility, and QA reward is retained precisely because CMI alone lacks an outcome anchor.

pith-pipeline@v1.3.0-alltime-deepseek · 27632 in / 14360 out tokens · 132411 ms · 2026-08-02T03:34:49.150579+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning." pith.science (2026). https://pith.science/paper/SCOW3CAM

@misc{pith2026260720553,
  author       = {Pith},
  title        = {Pith review of: CMI-Mem: Toward Generalizable Long-Term Memory Management via CMI-Augmented Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SCOW3CAM}},
  note         = {Machine review of arXiv:2607.20553}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Memory Manager models are pivotal in agent systems. Existing methods rely predominantly on LLM-judged synthetic question-answer (QA) pairs, making memory valuation dependent on sampled queries and the downstream reader. To address this limitation, we propose \textbf{CMI-Mem}, a reinforcement learning(RL)-based lightweight memory manager model with a hybrid reward that combines downstream QA correctness and intrinsic Conditional Mutual Information (CMI). CMI evaluates the information contributed by new conversational inputs relative to the current memory state without conditioning on a sampled QA query, thereby complementing rather than replacing QA grounding. Our codes are available at: https://github.com/Wyb0627/CMIMem , and the CMI-Mem-4B model checkpoint is available at: https://www.modelscope.cn/models/wyb0627/CMIMem-4B

Figures

Figures reproduced from arXiv: 2607.20553 by Bo Zheng, Haoyang Li, Jinsong Lan, Lei Chen, Qiuyu Zhao, Shichao Dong, Xiaoyong Zhu, Yubo Wang, Zenghui Sun.

Figure 1
Figure 1. Figure 1: The two reward pathways in CMI-Mem. The right panel isolates the added CMI pathway; the com￾plete training objective retains the QA pathway on the left and combines both rewards. Kang et al., 2025). Stronger API models raise serving costs, making compact managers prefer￾able for online deployment. Most existing Mem￾ory Managers rely on heuristic rules or synthetic Question-Answer (QA) pairs evaluated throu… view at source ↗
Figure 2
Figure 2. Figure 2: An example of CMI reward calculation in our [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: CMI-Mem’s training framework combines downstream QA reward with an intrinsic CMI reward [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The training dynamics of policy gradient loss [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Per-agent reward progression for each memory [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Average response length during training. The [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 3 linked inside Pith

  1. [1]

    Examine all messages thoroughly to extract EVERY detail about the user's preferences, personal information, and vital facts

  2. [2]

    operation

    Decide on ONE operation: - APPEND : Add new information to existing block (if <90% full) - REPLACE: Update specific outdated or incorrect information - REWRITE: Reorganize and consolidate the entire block (if >90% full or major updates needed) Return JSON with ONE of these operations: {"operation": "APPEND", "content": "..."} {"operation": "REPLACE", "old...

  3. [3]

    If memories mention specific dates/times, use those for time questions

  4. [4]

    If memories contain contradictory information, prefer the most recent

  5. [5]

    Not answerable

    Focus on memory content, not exact word matches. For factual questions (What/When/Where/Who): - Answer based on direct information in the memories. - If the specific fact is not mentioned, respond: "Not answerable". For inference/reasoning questions (Would/Could/Likely): - Make reasonable inferences based on related information. When to say "Not answerabl...

    2025

  6. [6]

    [Optional] Supporting Original Conversations: {chunk_context} The current date/time is {current_time}

    {memory_2} ... [Optional] Supporting Original Conversations: {chunk_context} The current date/time is {current_time}. Use this as the reference point when answering questions about relative time. Question: {question} Instructions:

  7. [7]

    Carefully analyze the retrieved memories to find relevant information

  8. [8]

    Consider synonyms and related concepts

  9. [12]

    Not answerable

    GOLD = "Not answerable": The generated answer is CORRECT if it indicates unavailability via any equivalent phrasing ("no information", "cannot be determined")

  10. [13]

    Not answerable

    GOLD is a SPECIFIC answer: A generated "Not answerable" is WRONG. If the answer attributes the correct fact to the WRONG person/entity, it is also WRONG

  11. [14]

    Not answerable

    CRITICAL: "Not answerable" can ONLY be CORRECT when GOLD is also "Not answerable". Do NOT be misled by reasoning -- focus on whether the answer actually provides the requested information. Question: {question} Gold answer: {gold_answer} Generated answer: {generated_answer} First provide a one-sentence explanation, then return the label as JSON: {"label": ...

  12. [2018]

    lifelong learner

    Correlation coefficients: appropriate use and interpretation.Anesthesia & analgesia, 126(5):1763– 1768. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open lan- guage models.arXiv preprint arXiv:2402.03300. Zhiyu Shen, Z...

    Pith/arXiv arXiv 2024

  13. [2024]

    InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 13851– 13870

    Evaluating very long-term conversational memory of llm agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 13851– 13870. Martin Marko, Drahomír Michalko, Adam Kubinec, and Igor Rie ˇcansk`y. 2024. Measuring semantic mem- ory using associative and dissociative retrieval tasks. Ro...

    Pith/arXiv arXiv 2024

  14. [2025]

    InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25972–25981

    Memory os of ai agent. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25972–25981. Amirhossein Kazemnejad, Milad Aghajohari, Eva Porte- lance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. 2025. Vineppo: Refining credit assignment in rl training of llms. In F orty-second Internationa...

    Pith/arXiv arXiv 2025