Pith.

Artificial Intelligence · arXiv:2610.20022 v2 · submitted 14 Oct 2026

Memorizing Who, Knowing How: Benchmarking Personal and Procedural Memory for Persistent Assistants

Maosheng Qin, Qi Zhang, Mingxuan Xia, Jinglei Chen, Jiexiang Wang, Sheng Guo, Cheng Peng, Haobo Wang

The paper says

“Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act.”

more of the abstract

Crucially, in harness-based personal assistance, procedures or skills are inherently user-conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change. Existing benchmarks typically emphasize either personal or procedural memory, leaving their interaction underexplored. We introduce PraxisMemBench, a harness-based benchmark for persistent personal assistance across daily-life and work trajectories, where personal and procedural memory evolve together and later tasks depend on both. Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single-type to interdependent dependencies (84.6% to 77.9%). More broadly, across existing methods, relation-related failures increase from early to late sessions as interaction history grows (59.8% to 72.9%), revealing a continual-maintenance challenge: writing correct memories is not enough if their evolving content and organization cannot be preserved for later retrieval. To address this, we propose PraxisMem, which uses privileged joint-state supervision to internalize the ability to maintain memory content and organization into the memory manager, with location-aware GRPO and dynamic residual-routed OPSD. PraxisMem reaches 40.78% overall Session Success Rate, compared with 20.60% for Claude Code's native memory. Our code is publicly available at https://github.com/QMMMS/PraxisMemBench.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 15 Oct 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A careful benchmark that genuinely fills the personal-plus-procedural-memory niche, but the headline findings are installed by the construction protocol rather than discovered in real data. This line is wrong

4 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

The paper says, at Section 4.3 (Finding 2), Appendix F.3 (Figure 13)

The review did not quote this passage.

Pith says: Objection 1 of 4 · major

The headline degradation claim (84.6% to 77.9%, abstract) is not isolated from task difficulty. Figure 13 already contains the needed control: on the same Task Completion metric, the Oracle — which receives the exact required memories — drops from a 96.95% average on single-type tasks (Personal-only 98.8, Procedural-only 95.1) to 92.8% on joint tasks.

more of objection 1

DualMem's corresponding drop is from 84.6% to 79.2% (Figure 13; pooled from the Coordinated 80.1 and Interdependent 77.9 values in Figure 6). The excess of DualMem's drop over Oracle's is therefore only about 1.2 points on pooled joint tasks, so most of the observed degradation could reflect the intrinsic difficulty of joint executable tasks (more constraints, longer procedures) rather than the separation of the two memory stores. The interdependent-specific gap (6.7 points) needs the same control. Please report Oracle completion separately for Coordinated and Interdependent tasks alongside Figure 6, and attribute to memory maintenance only the excess over the Oracle gradient, or otherwise demonstrate that joint and single-type tasks are matched in difficulty.

The paper says, at Section 2.2, Appendix C.2, Algorithm 1

The review did not quote this passage.

Pith says: Objection 2 of 4 · major

The central scientific inference — that joint personal-procedural maintenance is a distinct failure mode of persistent assistance (Findings 2-3, Conclusion) — is drawn from a distribution whose construction rules install the phenomenon. Section 2.2 states 'By construction, the task requires personal memory and procedural memory to jointly determine a measurable outcome' and that in later dependent sessions the relevant details …

more of objection 2

… 'are omitted from the public request'; Appendix C.2 excludes recently disclosed information from the historical target set; and Algorithm 1 synthesizes per-session user events that change personal state, invalidating previously stored procedures. These three choices produce exactly the measured effects (84.2% joint tasks; the 59.8% to 72.9% relation-failure shift). The authors transparently acknowledge that the synthetic personas may carry biases (Ethics statement), but no paragraph calibrates the rate parameters (omission frequency, preference-change rate, association volatility) against real interaction data, and the Findings are worded as properties of persistent assistance rather than of the benchmark distribution. This is a correctness-risk concern with a concrete test: measure these three rates in real assistant interaction logs or a small user study and report the benchmark's settings against them; absent such calibration, rephrase Findings 2-3 and the Conclusion as claims about the constructed distribution.

The paper says, at Appendix D, Eq. (40)-(41)

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

The Session SR definition is not fully auditable. The text states that 'a small number of sessions are excluded from SR aggregation after final human validation to ensure consistent scoring,' but the number of excluded sessions, their task families, and the exclusion criteria are not reported anywhere.

more of objection 3

Since the abstract headline (40.78% versus 20.60%) is a pooled rate over the eligible set E_t, and since all 14 methods were scored before any exclusion, the reader cannot verify that the exclusions were method-independent or that the criteria were fixed in advance. Please report the exclusion counts per family, the exact criteria used in final human validation, and a recomputation of the Session SR columns in Table 2 with the excluded sessions counted as failures.

The paper says, at Table 2, Eq. (40), Table 11

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

The Memory QA family barely discriminates at session level: all non-oracle baselines score between 0.00% and 15.71% Session SR despite moderate Recall (10.35-43.37) and F1 (13.05-46.88), and Hermes — the best harness-native method overall — reaches only 10.71%. Because Eq.

more of objection 4

(40) requires the conjunction F1 >= 0.75, P >= 0.60, R >= 0.60, it is unclear whether these near-zero rates reflect recall failure, precision failure, or judge behavior, especially since the human-human QWK for the QA precision rubric is the lowest in Table 11 (0.783). Please report a per-method breakdown of which conjunct fails, or justify the session criterion with a sensitivity analysis; otherwise this family contributes little signal beyond the component scores.

6 minor objections

The paper says, at Abstract, Section 4.3, Figure 6

Pith says: minor objection 1 of 6

The abstract's 84.6% is the pooled average of the two single-type values (84.7% personal-only, 84.5% procedural-only, Figure 6 left panel); state this pooling explicitly next to the number so the headline is reproducible from the figure.

The paper says, at Figure 6, right panel; Section 4.3 (Finding 3)

Pith says: minor objection 2 of 6

The 59.8% to 72.9% statistic pools MemOS, Hermes, and PraxisMem per the figure caption, but the abstract says 'across existing methods'; specify the method pool and the session-stage definition in the main text.

show 4 more

The paper says, at Table 2, Tables 18-19

Pith says: minor objection 3 of 6

Add an overall pooled Session SR column to Table 2 (the values already appear in Tables 18-19) so that the abstract's headline comparison of 40.78% versus 20.60% can be verified directly from the main results table.

The paper says, at Appendix D, Table 11

Pith says: minor objection 4 of 6

The human-human QWK for QA Precision (0.783) is markedly lower than for the other three rubrics; add a sentence interpreting the reliability of this rubric, since Eq. (40) makes Memory QA Session SR depend on a precision threshold.

The paper says, at Appendix C.3, Table 10; Appendix E.1

Pith says: minor objection 5 of 6

With an average trajectory length of 663.6K tokens and a 128,000-token Long Context budget that truncates oldest history first, state how much history remains available to Long Context in late sessions, so that the time-distance degradation in Figure 6 (middle) is interpretable for this baseline.

The paper says, at Reproducibility statement

Pith says: minor objection 6 of 6

The promised release of 'fixed benchmark splits' would benefit from explicit versioning and a contamination-control protocol for future works that extend the trajectories, given the long-horizon nature of the benchmark.

What would settle it

Restatement ablation: regenerate the 20 test trajectories with the omission rule disabled for a randomly selected half of the joint Tool/Artifact tasks — the public request naturally restates the single critical personal+procedural constraint it currently omits — keeping events, fixtures, SOPs, gold checks, and validators identical. Re-run DualMem and PraxisMem in the Scheduled setting. If the joint-task completion drop (84.6%→77.9%) and the relation-failure rise (59.8%→72.9%) persist under restatement, the external-validity concern is answered; if the gap collapses, the headline findings measure the construction's omission policy rather than an inherent joint-maintenance failure mode of persistent assistants.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 15 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 2 min 16 s · 47,603 tokens read, 16,148 written
  3. Editor wrote the verdict above 32 s · 48,619 tokens read, 3,824 written
  4. Referee major revision: 4 major objections, 6 minor 2 min 52 s · 47,947 tokens read, 19,115 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Read this one for the benchmark, not the method headline. PraxisMemBench genuinely fills a gap: no prior benchmark puts personal and procedural memory in the same harness with cross-session state dependence, and the construction is careful. Sequential trajectory building, blind independent validation per session, programmatic tool and artifact checks, and a judge-agreement audit reported with raw numbers (human-human QWK 0.934, GLM-5 at 0.793) all hold up. The 25-session trajectories at roughly 663K history tokens are a real stress test. The failure case studies in Appendix F.5 are the best part of the paper; the fragmented-relation and competing-memory examples ring true and will be useful to anyone building memory systems.

The main thing to be careful about: the 84.6-to-77.9 completion drop and the 59.8-to-72.9 relation-failure rise are properties of a distribution the authors constructed. Section 2.2 states that joint dependence holds "by construction," later requests omit historical details by design, and the event synthesizer deliberately invalidates old procedures. Findings 2 and 3 are demonstrations that the construction rules produce the predicted failure pattern, not discoveries about real persistent assistance. The ethics statement acknowledges the synthetic personas, but the load-bearing rates — omission frequency, preference-change rate, association volatility — are never calibrated against real interaction data. That bounds the external claim; it does not break the benchmark.

Three smaller items. The 40.78 versus 20.60 headline is one run per method with no variance estimate; a 20-point gap that large probably survives noise, but "probably" is doing work. The paper says a small number of sessions are excluded from Session SR after human validation without giving a count or criterion — that needs to be exact, or it invites the wrong inference. And PraxisMem is trained on the benchmark's own canonical states; the disjoint training and test domain pools mitigate transfer concerns, but the interaction structure is shared, so the method's advantage is partly tuned to this construction protocol.

This is a paper for researchers building long-horizon agent memory systems and for anyone designing executable benchmarks; the construction methodology is worth copying even if you never use the benchmark itself. The right referee will push on the calibration question and the reporting gaps, and the paper will be better for it. Send it to review.

The strongest objection

The central scientific claim — that joint personal–procedural maintenance is a distinct, dominant failure mode in persistent assistance (Findings 2–3) — is inferred from a distribution whose construction rules install the phenomenon. Section 2.2 states that "By construction, the task requires personal memory and procedural memory to jointly determine a measurable outcome," and that in later dependent sessions the relevant details are omitted from the public request; Appendix C.2 further excludes recently disclosed information from the historical target set, and the event synthesizer deliberately changes personal state in ways that invalidate previously stored procedures (Algorithm 1). These three choices — forcing 84.2% of executable tasks to be joint, universally omitting disambiguating context, and engineering association volatility — are precisely the conditions under which the reported 84.6%→77.9% DualMem degradation and the 59.8%→72.9% rise in relation-related failures appear. The computation is internally consistent, and the Oracle controls, ablation, judge audit, and backbone sensitivity are genuine support; what is load-bearing is the inference from these measured numbers to persistent assistants in general. The paper acknowledges the synthetic personas (Ethics statement) but offers no calibration of its key rates — omission frequency, preference-change rate, association volatility — against real interaction data. The unspecified Session-SR exclusions (Section D) and single-run reporting (Appendix E.1) are real but secondary: they bound the headline's precision, not the construction-dependence of the phenomenon.

The reader's weakest_assumption and my concern coincide, so I mark agree; my pass sharpens it into a specific mechanism (the request-omission rule, Section 2.2 and Appendix C.2) and a decisive experiment. I weighed the two other conditions before settling. The single-run issue (Appendix E.1) threatens the magnitude of the 40.78% versus 20.60% headline but not its direction: the ablation (Table 3), backbone sensitivity (Table 17), and monotone training curves (Figure 7) independently corroborate that the method and its components help within this benchmark, so even a proper variance estimate would not flip the qualitative claim. The unspecified Session-SR exclusions (Section D) are a transparency defect, but the rule applies across methods and is described as covering a small number of sessions, which bounds its effect on the between-method gap. The manufactured-phenomenon concern is different in kind: it attacks the link between the benchmark numbers and the paper's central conclusion. If request omission and forced joint dependence are the mechanisms producing the degradation, the paper has demonstrated that a joint-maintenance failure mode emerges under a deliberately difficulty-calibrated construction — a legitimate stress-test contribution — but it has not established that this mode dominates real persistent assistance, where users may restate constraints and procedures may evolve less abruptly. The proposed restatement ablation settles exactly this: if the headline effects vanish when the critical constraint is restated, the findings are artifacts of the omission policy; if they persist, the external-validity objection is substantially answered. I credit the benchmark's independent blind validation, programmatic verifiers, disjoint held-out domains, and judge-agreement audit (Table 11) as genuine evidence of internal reliability, and I checked the headline arithmetic (303/360 = 84.2% joint; the abstract's 84.6% is the mean of 84.7/84.5; Table 18 matches the abstract's 40.78/20.60; Table 15 counts sum to 349) without finding an inconsistency. The verdict remains CONDITIONAL, with the restatement ablation as the natural test of its external-validity clause.

The referee's report

The benchmark construction is methodologically careful: executable sessions are validated by independent testers (Appendix C.2), tool and artifact outcomes are programmatic, memory dependencies must map to observable outcomes, judge agreement is audited, and the training and test splits use disjoint domain pools (Appendix C.1), which materially reduces the risk that PraxisMem's gains come from memorizing training domains. The ablations in Table 3 and Figure 7 show a clean progression (Combined Completion 66.2 to 83.4) with each added component, and the backbone-sensitivity and cost-latency analyses (Tables 17-19) support the practical claims. The finding that relation-related failures, rather than access failures, dominate late sessions, backed by a dual-annotator failure attribution (Appendix F.5), is the paper's most interesting result. The authors also scope their own contribution honestly in places, noting that the Markdown schema is an implementation choice and not part of the benchmark (Appendix B.1). However, the headline degradation and the generalization of the findings to persistent assistance in general are not yet established, for the reasons in the major comments.

Pith's reading of the paper (a machine's summary, not the authors')

The paper argues that in a persistent personal assistant, procedures are inherently user-conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change, so personal and procedural memory must be maintained as one evolving, linked structure rather than two separate stores. To test this, the authors build PraxisMemBench, a harness-based benchmark with 25-session trajectories across 40 application domains in which 303 of 360 executable tasks require both memory types and later requests omit the historical conditions they depend on. They report that a dual-store baseline's completion falls from 84.6% on single-type tasks to 77.9% on interdependent tasks, and that across all methods the share of relation-related failures rises from 59.8% to 72.9% from early to late sessions, which they read as a continual-maintenance problem rather than a retrieval problem. Their proposed manager, PraxisMem, uses the canonical joint memory state as privileged training supervision, combining location-aware GRPO with dynamic residual-routed OPSD, and reaches 40.78% overall Session Success Rate versus 20.60% for Claude Code's native memory. If the claim is right, long-horizon assistant memory should be designed, trained, and evaluated as one user-conditioned structure whose organization, not just content, must be maintained.

What carries the argument. The load-bearing object is the canonical joint memory state $\mathcal{M}^{\star}_n = (\mathcal{M}^{\star,\mathrm{pers}}_n, \mathcal{M}^{\star,\mathrm{proc}}_n, \mathcal{I}^{\star}_n)$, realized in training as a structured Markdown store: user profiles, dated event files, per-domain skill files, and an index file that connects them into an association structure. That canonical state acts as privileged supervision, and the discrepancy between it and the manager's realized state, recomputed at each update chunk, is the dynamic residual that says what is missing, misplaced, conflicting, or redundant and where it lives. Two training objectives carry the argument: location-aware GRPO, whose reward uses a lexicographic matching that prioritizes location correctness over semantic similarity when scoring memory updates, and dynamic residual-routed OPSD, which activates a privileged teacher only for criteria that every rollout in a group missed and routes its token-level guidance only to the chunks where the defect persists.

Core claim. On the paper's own terms, the discovery is that the association between personal and procedural memory is itself part of what must be remembered: the memory state is modeled as $\mathcal{M}_n = (\mathcal{M}^{\mathrm{pers}}_n, \mathcal{M}^{\mathrm{proc}}_n, \mathcal{I}_n)$, where $\mathcal{I}_n$ is an evolving structure linking user context to the procedures it conditions. Because users rarely restate the historical conditions behind their requests, correct action requires recovering the right personal context and the right procedure together, and keeping that pairing intact across repeated updates. The authors further argue that scalar task rewards cannot say which part of the joint state is wrong, whereas a canonical joint state available during training can, and that this privileged information can be internalized by the manager so it is not needed at deployment. Their experiments show that writing correct content is not enough: location-aware supervision raises correct-location recall from 41.1% to 61.3% over training, and on the controlled eight-trajectory ablation subset, interdependent-task completion rises from 54.9% to 82.1%.

The premise it rests on. The benchmark's central phenomenon is built in by its own construction rule: annotators deliberately couple each later task to both a personal fact and a procedure, and the authors acknowledge the personas are synthetic and may carry biases, so if real persistent assistance is not structured this way the benchmark results do not transfer to real users.

If the paper is right (Pith's inferences, not the authors')

Benchmarks for persistent assistants should treat joint personal–procedural dependence as a first-class axis; PraxisMemBench supplies 25-session trajectories, 40 application domains, and 2,185 annotated memory dependencies for that purpose.

Dual-store designs will systematically underperform wherever the current user state selects or instantiates the applicable procedure, even when each store retrieves its own evidence correctly.

Long-horizon memory failure is dominated by fragmented and competing personal–procedural relations, so evaluation must track how relations degrade across trajectory stages, not just how accurately memories are written.

A memory manager can learn where to place information, not merely what to write, when training exposes it to a canonical reference state, and that organizational skill transfers to task success without the canonical state at deployment.

If user-conditioning transfers to real deployments, skill stores should be organized so that each procedure carries explicit links to the user-state conditions that select it, and retrieval should be conditioned on current user state; a testable prediction is that such consolidated stores will beat modular skill libraries on long-horizon personal tasks.

The 59.8%→72.9% shift toward relation failures is measured on trajectories whose joint dependencies were pre-audited by annotators; a naturalistic replication on organic user logs would show whether the effect is a property of real persistent assistance or of the construction rule.

The training results imply that reorganizing existing entries, moving, merging, and re-linking them after a preference change, is as important as writing new ones; a practical extension would test whether the gains persist when user preferences change weekly rather than monthly.

The headline gap of 40.78% versus 20.60% Session SR is measured on fictional personas with no privacy friction; real deployments with retention, consent, and access controls may compress the memory that can be maintained and thus change the ordering.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act. Crucially, in harness-based personal assistance, procedures or skills are inherently user-conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change. Existing benchmarks typically emphasize either personal or procedural memory, leaving their interaction underexplored. We introduce PraxisMemBench, a harness-based benchmark for persistent personal assistance across daily-life and work trajectories, where personal and procedural memory evolve together and later tasks depend on both. Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single-type to interdependent dependencies (84.6% to 77.9%). More broadly, across existing methods, relation-related failures increase from early to late sessions as interaction history grows (59.8% to 72.9%), revealing a continual-maintenance challenge: writing correct memories is not enough if their evolving content and organization cannot be preserved for later retrieval. To address this, we propose PraxisMem, which uses privileged joint-state supervision to internalize the ability to maintain memory content and organization into the memory manager, with location-aware GRPO and dynamic residual-routed OPSD. PraxisMem reaches 40.78% overall Session Success Rate, compared with 20.60% for Claude Code's native memory. Our code is publicly available at https://github.com/QMMMS/PraxisMemBench.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-15

The authors' main claim, as the classifier picked it out 0.62

“Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single-type to interdependent dependencies (84.6% to 77.9%).”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.18 · states limitation 0.39 · human participants 0.20 · controlled comparison 0.90 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act. background 1.00 0.07 0.09 asserted 0.98
Crucially, in harness-based personal assistance, procedures or skills are inherently user-conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change. background 0.98 0.11 0.08 asserted 1.00
Existing benchmarks typically emphasize either personal or procedural memory, leaving their interaction underexplored. gap 1.00 0.06 0.03 asserted 1.00
We introduce PraxisMemBench, a harness-based benchmark for persistent personal assistance across daily-life and work trajectories, where personal and procedural memory evolve together and later tasks depend on both. contribution 0.98 0.85 0.10 asserted 0.99
Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single-type to interdependent dependencies (84.6% to 77.9%). result 1.00 0.94 0.26 asserted 1.00
More broadly, across existing methods, relation-related failures increase from early to late sessions as interaction history grows (59.8% to 72.9%), revealing a continual-maintenance challenge: writing correct memories is not enough if their evolving content and organization cannot be preserved for later retrieval. interpretation 0.52 0.63 0.24 asserted 1.00
To address this, we propose PraxisMem, which uses privileged joint-state supervision to internalize the ability to maintain memory content and organization into the memory manager, with location-aware GRPO and dynamic residual-routed OPSD. method 0.95 0.80 0.10 asserted 0.97
PraxisMem reaches 40.78% overall Session Success Rate, compared with 20.60% for Claude Code's native memory. result 1.00 0.93 0.21 asserted 1.00

What kind of paper the classifier reports this is: dataset or benchmark 0.99 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-15: 1 request, 6,418 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (28)
Figure 1 of the paper
Figure 1. Representative personal, procedural, and joint
Figure 2 of the paper
Figure 2. PraxisMemBench trajectory construction. Each session is instantiated from the evolving
Figure 3 of the paper
Figure 3. PraxisMemBench evaluation lifecycle. Across sessions, the memory method continu
Figure 4 of the paper
Figure 4. Memory-dependency structure in the test split. From left to right: (a) Distribution of
Figure 5 of the paper
Figure 5. Overview of PRAXISMEM. (1) Criteria missed by all rollouts are activated. (2) They are routed to chunks where the corresponding memory defects persist. (3) A privileged teacher receives the dynamic residual between current and target memory states to provide token-level guidance. (4) GRPO trains the full rollout, while OPSD supervises only routed chunks. relation to the relevant user context or procedure after subsequent updates. We therefore extend the soft semantic state matching of DeltaMem (Zhang et al., 2026) with location-aware supervision, using the canonical joint state to…
Figure 6 of the paper
Figure 6. Memory use across dependency types and persistent interaction. From left to right: (a)
Figure 7 of the paper
Figure 7. Correct-location Recall across training variants. rent personal context. DualMem preserves the two memory contents separately but leaves this association implicit, making individually plausible but jointly incompatible memories more likely. Finding 3: Persistence turns memory coupling into a continual-maintenance problem. The middle panel of
Figure 8 of the paper
Figure 8. Memory QA task example. 26
Figure 9 of the paper
Figure 9. Tool-use task example. 27
Figure 10 of the paper
Figure 10. Artifact task example. 28
Figure 11 of the paper
Figure 11. Training dynamics of four memory-training variants, measured by Correct-location Re
Figure 12 of the paper
Figure 12. Routing and stability diagnostics for the two OPSD variants over training steps.
Figure 13 of the paper
Figure 13. Task Completion by memory dependency type.
Figure 14 of the paper
Figure 14. DualMem fragmented-relation failure. Separate retrieval surfaces the escalation format
Figure 15 of the paper
Figure 15. Hermes fragmented-relation failure. The agent recovers the return workflow and sepa
Figure 16 of the paper
Figure 16. OpenClaw retrieval failure after roughly nine months.
Figure 17 of the paper
Figure 17. PraxisMem update failure. Personal context from the source session is stored in
Figure 18 of the paper
Figure 18. MemOS competing-memory failure. Retrieved procedural structure yields a polished
Figure 19 of the paper
Figure 19. Effectiveness and runtime trade-offs. (a) Session SR versus inference cost for completing
Figure 20 of the paper
Figure 20. Prompt template for expanding a PersonaHub seed into a structured stable persona.
Figure 21 of the paper
Figure 21. Prompt template for synthesizing a persona value change and the event that explains it.
Figure 22 of the paper
Figure 22. Prompt template for the user simulator’s next turn.
Figure 23 of the paper
Figure 23. System and user prompt templates for PRAXISMEM memory update. System message You are a long-term memory retrieval module. Use tools to inspect the Markdown memory store and find evidence for the current request. The final response must contain only concise bullets in the form - label: text. Do not include headings, explanations, hidden labels, internal identifiers, source IDs, or irrelevant history. 48
Figure 24 of the paper
Figure 24. System and user prompt templates for PRAXISMEM memory retrieval. You are a strict evaluator for personalized memory points reflected in tool arguments. For each item, compare the actual memory-bearing open-text tool arguments against the gold memory point. Each item is one personalized memory point. Multiple memory points may be reflected in the same tool argument. Scoring First identify every required semantic component in the gold memory point. 2: The actual arguments cover all required semantic components. Exact wording is not required. 1: The actual arguments cover a strict…
Figure 25 of the paper
Figure 25. Judge prompt for personal-memory points in tool arguments.
Figure 26 of the paper
Figure 26. Judge prompt for semantic personal-memory slots in artifacts.
Figure 27 of the paper
Figure 27. Judge prompt for Memory QA recall. You are a strict memory precision and accuracy evaluator. The program has already parsed the assis￾tant’s final summary into candidate bullets. Do not split, merge, or invent candidate units. Evaluate each candidate bullet exactly as one memory candidate. For each candidate Decide whether its field belongs to the target gold memory field set, and whether its content is supported by the gold memory points. Accuracy score 2: The candidate’s core content is supported by the gold memory points and has no contradictory detail. 1: The candidate’s core…
Figure 28 of the paper
Figure 28. Judge prompt for Memory QA precision. 51

Figures are the authors', reproduced from the paper.

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment