{"id":"de3df526-6432-442a-a62c-4d21a38ba0c9","arxiv_id":"2607.18975","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Mi-Memory proposes a four-role lifecycle framework for personal AI memory with an audit contract of typed evidence, traces, strategy artifacts, and rollback records; modules are benchmarked separately, but the contract itself is not jointly validated.","lead":"This report proposes Mi-Memory, a framework that treats personal AI memory as an auditable lifecycle across structure, evidence, evolution, and deployment. Its modules post strong benchmark numbers, but the shared audit contract that is the main contribution is not tested end-to-end.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The audit contract's central value is untested end-to-end: §8.2 concedes that module compatibility is not a joint ablation, so cross-module traceability and regression resistance—the core of the lifecycle claim—remain unvalidated.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing gap: the audit contract is the paper's primary contribution, and it is never tested as an integrated whole. The paper is unusually honest about this in §8.2, but honesty does not supply the evidence. I considered alternative concerns—the non-comparability of the MemStack 93.59% and E2MEND 94.74% LoCoMo numbers, and the absence of released code—but these are either explicitly bounded by the paper or secondary to the central claim. The most central risk is that the shared artifact families may add process overhead without improving failure localization or preventing regressions; until a joint ablation or closed-loop trace-accuracy measurement is run, the framework's integrating value is underdetermined. This supports the CONDITIONAL verdict rather than rejection: the module-level evidence is real and the design is coherent, but acceptance of the central contribution should be conditional on the missing end-to-end validation. No change to the reader's verdict is needed.","tokens_in":41865,"tokens_out":2898,"duration_ms":28565,"concrete_test":"Run a closed-loop benchmark on one dataset (e.g., LoCoMo) with all four modules connected: MemFuse outputs typed payloads into MemStack; E2MEND performs gated strategy updates; LiteMem serves a subset of queries. Measure (1) trace accuracy—the fraction of failing questions where the diagnostic trace's earliest-loss stage matches a manually annotated or independently derived ground-truth stage; and (2) regression resistance—the held-out category regression rate across accepted strategy updates. Compare against an ablative baseline with artifact families disabled (e.g., no typed payloads, no gate/rollback records, no diagnostic traces). If trace accuracy or regression resistance does not improve over the baseline, the audit contract's central claim is not demonstrated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper defines a valid memory system in §3.1 via three operational requirements: evidence preservation, stage-local diagnosis, and auditable evolution. The central contribution is that these hold across roles through shared artifact families. But every empirical result is module-level: MemStack (§4.3), MemSense (§5.3), E2MEND (§6.3), and LiteMem (§7.3) are evaluated in separate harnesses with different benchmarks and backbones. No experiment feeds MemFuse payloads into MemStack, evolves a strategy under E2MEND, and checks whether the audit contract makes failures localizable or regressions reversible in a closed loop. Section 8.2 states this explicitly: 'These compatibility observations are not a substitute for joint ablation' and 'marginal contribution still requires a joint ablation benchmark.' Thus the strongest claim—that the audit contract integrates algorithmic, systems, and governance aspects—is currently supported by design and interface compatibility, not by evidence that the shared artifacts improve diagnosis or prevent silent regressions. This is a load-bearing missing evaluation rather than an internal inconsistency; the paper's own limitation statement makes the gap clear. Secondary but related: trace correctness itself is not independently validated, since diagnostic traces are emitted by the same pipeline that is being audited.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Mi-Memory, a lifecycle memory framework for personal AI organized around four roles: Structure (MemStack), Expansion (MemSense/MemFuse), Evolution (D2ACCI/E2MEND), and Deployment (LiteMem). Its central proposal is a \"lifecycle audit contract\" carried by four artifact families: typed evidence payloads, diagnostic traces, strategy artifacts, and gate/rollback records. The paper argues that this contract makes evidence movement traceable across module boundaries, localizes failures to pipeline stages, and bounds policy evolution. Empirical evidence is role-specific and explicitly calibrated: MemStack is evaluated as a controlled reference on LoCoMo, PersonaMem-V2, and LongMemEval; MemSense is module-level on Mem-Gallery; MemFuse is preliminary/internal on MemFuseBench; E2MEND is a descriptive offline strategy-search result on LoCoMo; LiteMem is transfer-feasibility evidence in a repository-native setting. The paper is unusually careful in labeling these evidence boundaries, and Section 8 centralizes a claim ledger with explicit limitations.","tokens_in":42203,"tokens_out":4711,"duration_ms":48038,"significance":"If the lifecycle audit contract holds, it would give the community a useful organizing vocabulary for memory-system research, separating serving correctness from update correctness and making memory-policy changes inspectable and reversible. The paper's strengths include: an explicit evidence-level taxonomy, controlled-reference comparisons where possible, artifact-level reproducibility claims for strategy evolution, and unusually candid limitation statements. The formulation itself is valuable even before a full end-to-end validation. However, the main integrative claim—that the shared audit contract improves cross-module audibility, failure localization, and regression resistance—is not yet supported by evidence; it is supported by design and interface compatibility. The paper's own limitation statement in Section 8.2 acknowledges this gap. Because the framework is proposed as a systems contribution, the missing joint evaluation is the key obstacle to acceptance.","major_comments":[{"comment":"The central claim that the lifecycle audit contract integrates the four roles is left unevaluated. The paper states explicitly: \"These compatibility observations are not a substitute for joint ablation\" and \"marginal contribution still requires a joint ablation benchmark.\" No experiment feeds MemFuse payloads into MemStack, evolves a strategy under E2MEND, and checks whether the shared artifacts localize failures or enable rollback in a closed loop. Since the audit contract is the paper's primary contribution (§1, §3.1), this is a load-bearing missing evaluation. A concrete remedy would be a joint ablation with and without the shared artifact families, measuring failure-localization accuracy and recovery from induced regressions.","section":"§8.2"},{"comment":"The E2MEND improvement from 75.58% to 94.74% on LoCoMo (+19.16pp) is a single descriptive run in which strategy search, gate decisions, and checkpoint selection all operate on the same benchmark. No held-out split, repeated-run variance, or confidence interval is reported; the paper itself labels the result descriptive. This makes the headline number vulnerable to benchmark overfitting and does not support a generalization claim. At minimum, the authors should evaluate on a validation split or provide per-category repeated-run statistics, and state the acceptance thresholds before reporting the result as evidence of bounded evolution.","section":"§6.3, Table 10"},{"comment":"The MemSense comparison is confounded: the 89.15% Mem-Gallery score uses GPT-4.1-mini with a GPT-4o-mini judge, while the cited baselines use a unified Qwen-2.5-VL-7B backbone. The paper correctly disclaims direct numerical ranking, but the operational conclusion \"IKB should be the primary evidence source\" is not supported without a same-backbone ablation isolating the IKB contribution. Similarly, MemFuseBench is an internal benchmark scored by the authors' judge pipeline, with no human upper bound or inter-annotator agreement; the +4.7pp margin over mem0 is preliminary and should not be read as cross-device fusion superiority beyond descriptive evidence.","section":"§5.3, Tables 6–7, Figure 7"},{"comment":"The audit contract's second requirement, stage-local diagnosis, is never independently validated. Diagnostic traces are emitted by the same pipeline that is being audited, and the evidence-preservation ladder in Eq. (14) assumes knowledge of ground-truth evidence. No experiment measures trace-label accuracy against human-annotated failure causes or independent instrumentation. Without such validation, the core diagnostic property of the contract is assumed rather than demonstrated. A trace-label accuracy study, even on a small sample, would materially strengthen the claim.","section":"§3.1, Appendix E, Eq. (14)"}],"minor_comments":[{"comment":"The LoCoMo margin over MemBrain is reported as +0.36% in Figure 4 and +0.34pp in Section 8.1. These should be reconciled or explicitly described as different quantities (percentage vs. percentage-point change).","section":"§4.3 vs §8.1"},{"comment":"The provenance-relevance trade-off weight λ is introduced without any value or sensitivity analysis. Since the objective is used to justify the Evidence Admission contract, a brief statement of the chosen value and its stability would help.","section":"§5.1, Eq. (5)"},{"comment":"The LiteMem transfer evidence uses a LoCoMo-aligned setting with a modular TypeScript harness. The paper should state explicitly that the 90.81% score is not directly comparable to the MemStack LoCoMo result because of differences in harness, retrieval surface, and evaluation prompts, even though the retention calculation is transparent.","section":"§7.3"},{"comment":"The LiteMem scoring equation uses four λ weights and two decay timescales, but the paper does not report the configured values or any sensitivity analysis. These are implementation-level parameters, but a brief configuration note would improve reproducibility.","section":"Appendix G, Eq. (20)"}],"recommendation":"major_revision","confidential_remarks":"The paper's candid evidence labeling is a real strength, and the missing joint ablation is acknowledged rather than hidden. I recommend major revision rather than rejection because the audit-contract formulation is useful and the missing end-to-end validation is within the paper's stated scope. The editor should ensure that the E2MEND +19.16pp number is not treated as a leaderboard claim, and that the revised version either adds a joint ablation or explicitly narrows the central contribution to a formulation/architecture proposal rather than an empirically validated integration mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, its real contribution is the lifecycle audit contract itself: the four artifact families (typed evidence, diagnostic traces, strategy artifacts, gate/rollback records) and the four-role decomposition give the memory-systems community a clean shared vocabulary for talking about provenance, diagnosis, and governed change. Second, the paper's central claim—that this contract makes memory systems more auditable and regression-resistant across modules—is never tested end-to-end. Section 8.2 says it plainly: the compatibility observations are “not a substitute for joint ablation,” and marginal contribution “still requires a joint ablation benchmark.” That is a load-bearing gap, and the authors know it.\n\nWhat the paper does well: it is unusually disciplined about evidence boundaries. The evidence-level taxonomy (controlled reference, module-level, preliminary/internal, transfer-feasibility, design-only) is a real improvement over the typical technical report that lumps everything into one leaderboard. MemStack's controlled-reference comparison on LoCoMo, PersonaMem-V2, and LongMemEval is a reasonable anchor. The D2ACCI rounds with paired comparisons and archived rejected directions are a model of honest engineering narrative, even if the statistical support is mostly descriptive.\n\nThe soft spots are in proportion to how soft they actually are. The E2MEND +19.16pp LoCoMo gain is a benchmark-search result, not a prediction; the paper calls it descriptive and does not overclaim statistical strength. MemFuseBench is internal, judge-scored, and lacks external validation or a human upper bound; again, the paper labels it preliminary. Trace correctness is not independently validated—the diagnostic traces are emitted by the same pipeline being audited—and that is a subtle but real circularity. No code or data are shipped, which limits reproducibility, though the project homepage exists.\n\nWho this is for: anyone working on personal-AI memory architectures, especially on provenance or governance aspects. It is a framework paper with module-level evidence, not a closed-loop demonstration. It should get a serious referee: the framing is valuable, the honesty is rare, and the missing joint ablation is a well-defined task rather than an unfixable flaw. My recommendation is to send it to peer review with a clear request for an end-to-end ablation or a narrowing of the central claim to contract design plus per-module compatibility.","headline":"A genuinely useful organizing framework with unusually honest claim boundaries, but the central cross-module audit-contract claim is openly unvalidated and needs a joint ablation before it can be taken as demonstrated.","tokens_in":42761,"tokens_out":1570,"would_cite":true,"duration_ms":17937,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that personal-AI memory should be redesigned as a governed lifecycle, kept auditable by a shared contract of typed evidence, diagnostic traces, strategy artifacts, and rollback records.","keywords":["personal AI","memory lifecycle","audit contract","evidence provenance","memory governance","multimodal memory","edge deployment","strategy evolution"],"falsifier":"A concrete falsifier: run a joint ablation where the audit contract artifacts (typed payloads, diagnostic traces, strategy artifacts, gate/rollback records) are removed one at a time from an integrated Mi-Memory pipeline, and measure whether failure localization accuracy, silent-regression frequency, or rollback success degrade. If removing the contract artifacts does not worsen any of these metrics (or if the traces cannot predict the true failing stage better than chance), the central claim of the framework is unsupported.","tokens_in":41718,"feed_emoji":"🧠","tokens_out":5191,"duration_ms":39382,"temperature":0.7,"pith_summary":"The report argues that long-term memory for personal assistants—across phones, cars, homes, and wearables—should be built as an auditable lifecycle, not a passive store of past chats. It proposes a shared 'lifecycle audit contract' that ties together four roles: structuring durable state, expanding evidence beyond dialogue, governing policy changes, and deploying under edge constraints. Four artifact families carry the contract: typed evidence payloads with provenance, diagnostic traces that localize where evidence is lost, versioned strategy artifacts, and gate/rollback records. Concrete modules instantiate each role, and the report reports controlled-reference accuracy on three memory benchmarks, module-level results on multimodal and cross-device evidence, a gated strategy-evolution gain on LoCoMo, and transfer-feasibility on a repository-native lightweight substrate. The paper is careful to bound each claim's evidence level and explicitly notes that cross-module joint ablation is still missing.","feed_headline":"Personal-AI memory becomes an auditable lifecycle under one contract","feed_subtitle":"A four-role framework keeps every answer traceable and every memory-policy change reversible.","key_machinery":"The central object is the lifecycle audit contract: a set of four artifact families (typed evidence payloads that preserve source identity and provenance; diagnostic traces that record where evidence moved or disappeared; versioned strategy artifacts that make memory-policy changes explicit; and gate/rollback records that bound accepted evolution). It works by requiring every memory-dependent answer to be traceable back through context and evidence, and every policy update to carry a versioned diff, fixed harness, gate decision, and rollback point. This contract is what links the four roles—Structure, Expansion, Evolution, Deployment—and what makes failures localizable and changes reversible","core_discovery":"The central discovery is that the 'lifecycle audit contract' can serve as the integration primitive for Personal AI memory. Rather than optimizing a single memory store, the framework defines three operational requirements for a valid memory system: evidence preservation (every answer traces back to source observations), stage-local diagnosis (a failure trace identifies the earliest loss point in ingestion, storage, retrieval, filtering, packing, or generation), and auditable evolution (every policy change carries a versioned strategy diff, a fixed harness, a gate decision, and a rollback point). The contract is carried across modules by four artifact families—typed evidence payloads, diagno","pith_inferences":["If the contract holds, the next natural target is a standardized cross-system memory interface—analogous to tool-use or function-calling schemas—so different memory architectures can be jointly ablated and composed; the paper itself lists this as an open direction.","The paper's own Section 8.2 admits that cross-module compatibility observations are 'not a substitute for joint ablation'; until such an ablation shows that the contract improves failure localization or prevents regressions in a closed loop, the framework's main added value remains an untested hypothesis.","A testable extension: instrument the diagnostic traces to quantify how often the earliest-loss stage matches the true root cause (relative to oracle labels) in a joint pipeline; the paper reports stage labels but not a measure of attribution accuracy.","The E2MEND gain (75.58%→94.74% on LoCoMo) is descriptive, not statistical; a repeated-run study with paired significance tests and category-level confidence intervals would substantiate or weaken the claim that gated strategy search beats manual iteration."],"forward_implications":["If the audit contract is adopted, memory-system failures can be attributed to a specific stage (ingestion, retrieval, filtering, packing, generation) rather than hidden in a black-box RAG pipeline.","Memory-policy changes become versioned, gated, and reversible, reducing silent regression risk during iteration.","The same contract can transfer across deployment substrates: a repository-native Markdown/Git implementation retained 90% of the service-side improvement over a no-memory baseline in a LoCoMo-aligned setting.","Non-dialogue evidence (images, device events, cross-device links) can enter memory as provenance-preserving payloads instead of being discarded or mis-fused.","The framework separates serving correctness from update correctness, making 'was this answer grounded?' and 'is this policy change safe?' distinct, auditable questions."],"fun_headline_variants":["Memory becomes an auditable lifecycle for Personal AI","One audit contract ties Personal AI memory together","Personal AI memory tracks evidence, versions, and rollbacks","A four-role framework turns memory into an auditable chain","Lifecycle memory: evidence-gated, reversible, deployment-aware"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the shared audit contract actually improves cross-module auditability, diagnosability, and regression-resistance in a closed-loop setting; the paper's own Section 8.2 acknowledges this premise is untested ('marginal contribution still requires a joint ablation benchmark').","fun_headline_variants_meta":{"raw":{"variants":["Memory becomes an auditable lifecycle for Personal AI","One audit contract ties Personal AI memory together","Personal AI memory tracks evidence, versions, and rollbacks","A four-role framework turns memory into an auditable chain","Lifecycle memory: evidence-gated, reversible, deployment-aware"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000164,"raw_usage":{"total_tokens":1105,"prompt_tokens":791,"completion_tokens":314,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":236}},"tokens_in":535,"tokens_out":314,"duration_ms":50882,"temperature":1.0,"reasoning_tokens":236,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T13:47:43.169381+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier: run a joint ablation where the audit contract artifacts (typed payloads, diagnostic traces, strategy artifacts, gate/rollback records) are removed one at a time from an integrated Mi-Memory pipeline, and measure whether failure localization accuracy, silent-regression frequency, or rollback success degrade. If removing the contract artifacts does not worsen any of these metrics (or if the traces cannot predict the true failing stage better than chance), the central claim of the framework is unsupported.","supporting_citations":[],"review_version":1}