{"id":"39e06383-7a99-4655-8b2e-08fb53a0f7cf","arxiv_id":"2607.02881","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Offline practice on historical user actions builds dual experiential memory (patterns plus bias alerts) that improves LLM long-horizon behavior prediction over compression baselines.","lead":"PraMem turns a user's long action history into offline practice problems so an LLM can distill that user's habits and its own prediction biases before the next guess. The resulting dual memory beats standard compression and retrieval baselines on video, live, ads, shopping, and movie-rating tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Self-review may pass LLM-plausible but non-causal proposals; gains could be generic memory rather than practice-derived patterns.","rationale":"The reader correctly flags the weakest assumption: that practice samples plus LLM self-audit produce transferable patterns/biases rather than noise or self-consistency artifacts. That assumption is load-bearing for the paradigm-shift claim versus context-compression baselines. My concern sharpens the same point: the only quality gate is LLM-on-LLM, and no control isolates practice+self-review from “just give the model more good text.” Existing ablations and the case study (Appendix F) are consistent with the claim but do not falsify the alternative. The proposed matched-length non-practice and no-filter controls would settle it with one experiment. Because the paper is otherwise clear, well-ablated, and shows transfer across backbones, I leave the verdict at CONDITIONAL rather than REJECT; the concern does not invent a new flaw, it makes the reader’s weakest assumption operationally testable. No circular math or formal error; the risk is empirical attribution.","tokens_in":22568,"tokens_out":702,"duration_ms":6717,"concrete_test":"On a fixed OmniBehavior user subset, build three memories of matched token length: (A) full PraMem, (B) PraMem with SelfReview always returning Pass (no filter), (C) a non-practice control that asks the same LLM once to write pattern+bias bullets from the full history without trial/label comparison. Evaluate all three with the same truncated suffix and both backbones. If (A) does not beat (B) and (C) by a statistically significant margin on overall F1 (and especially Ads F1), the self-review/practice claim does not hold.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that iterative practice on later historical segments, filtered by groundedness (perturbation) and generalizability (virtual-scene) self-review plus consensus every T=5 rounds, yields transferable pattern and bias-alert experience that explains the large lifts over Mem0/MemOS/ProEx (Table 1: overall ACC 84.7 / F1 31.6 vs ProEx 77.7 / 26.9 on GPT-OSS-120B). The load-bearing step is that SelfReview (Eqs. 6–7, §3) and consensus (Eq. 5) reject sample-specific noise and LLM self-consistency artifacts rather than merely selecting fluent, model-preferred statements. Both review stages are LLM-judged on LLM-generated perturbations/virtual scenes; nothing independently verifies that a “Pass” proposal is causally tied to the user’s true next-action distribution. Ablations (Table 2, Figs. 5–6) remove components but never replace self-review with a non-LLM oracle or a frozen random/generic memory of matched length. Thus the headline superiority could be driven by (i) extra high-quality natural-language context that any strong summarizer would produce, or (ii) self-reinforcing LLM priors, rather than practice-derived, user-specific experience. The 80-round schedule and offline cost amplify this risk: more rounds improve scores (Fig. 4) but also give more opportunities for the same LLM to polish its own narrative.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes PraMem, a training-free framework that reframes long historical user sequences as practice resources rather than a context burden. It iteratively samples later temporal segments as labeled practice tasks, has an LLM predict under a current experiential memory (pattern experience of user habits plus bias-alert experience of model-intrinsic failure modes), reflects to generate revision proposals, filters them with a two-stage self-review (perturbation-based groundedness and virtual-scene generalizability), and periodically applies only consensus-supported operations. On OmniBehavior (four scenarios) and MovieLens-1M, with GPT-OSS-120B and Qwen3.5-35B-A3B, PraMem reports large gains over long-context, truncation, RAG, summary, Mem0, MemOS, and ProEx (e.g., overall ACC 84.7 / F1 31.6 vs ProEx 77.7 / 26.9 on GPT-OSS-120B), with ablations of deep thinking, reflection, consensus, each experience type, and each review stage, plus evolution plots and a case study.","tokens_in":22975,"tokens_out":1346,"duration_ms":23717,"significance":"If the gains are genuinely attributable to practice-derived, user-specific pattern and bias-alert experience rather than merely richer natural-language side context, the work offers a useful paradigm shift for LLM-based sequential recommendation and behavior simulation: offline practice can convert long histories into compact, transferable memory that improves prediction under short recent windows and transfers across backbones. Strengths include a clear systems design, multi-scenario and multi-backbone evaluation, component ablations, memory-length evolution analysis, released code, and an explicit efficiency report. The bias-alert memory idea is a distinctive complement to profile/extraction baselines that only manage historical facts.","major_comments":[{"comment":"§3, Eqs. (6)–(7): Both self-review stages are LLM-judged on LLM-generated perturbations and virtual scenes. There is no independent check that a “Pass” proposal is causally tied to the user’s next-action distribution rather than fluent, self-consistent model statements. Because the central narrative is that SelfReview plus consensus (Eq. 5) reject sample-specific noise and self-consistency artifacts, this circularity is load-bearing; a non-LLM oracle, human audit of a proposal sample, or frozen/random proposal baseline would substantially strengthen the claim.","section":"§3 Self-review Mechanism, Eqs. (6)–(7)"},{"comment":"§4.2, Table 1 and §4.4: Headline superiority over Mem0/MemOS/ProEx is large, but there is no length- or quality-matched control that injects generic, non-practice, or randomly sampled natural-language “experience” of similar token budget. Without that control, gains could be driven by extra high-quality structured text rather than practice-derived, temporally grounded patterns. A matched-context ablation is needed to support the paradigm claim that practice (not just better memory text) is the operative mechanism.","section":"§4.2 Main Results, Table 1"},{"comment":"§4.3, Figure 4 and Implementation Details: Performance rises monotonically over 80 rounds while memory length stabilizes after ~60, which is encouraging, but free parameters (n=10, m=7, T=5, q=1, k, 80 rounds) are fixed without sensitivity analysis. Given that more rounds also give the same LLM more opportunities to polish its own narrative, reporting sensitivity of ACC/F1 to T, n, and round budget—and preferably a diversity/novelty metric on accepted experiences—is necessary to show that later rounds add transferable signal rather than self-reinforcing polish.","section":"§4.3 Evolution of Experiential Memory, Figure 4"},{"comment":"§4.1–4.2, Figure 3: MovieLens-1M is evaluated only via confusion matrices and a single F1 callout for PraMem; quantitative ACC/macro-F1 for all baselines under the same backbone and truncation setting are not tabulated. For a multi-class claim of “stronger diagonal dominance,” a full numeric table (and the same backbone split as OmniBehavior) is needed so the MovieLens result can be compared on equal footing with Table 1.","section":"§4.2 Main Results, Figure 3"}],"minor_comments":[{"comment":"Dataset name is inconsistently spelled “OminiBehavior” vs “OmniBehavior” (abstract/intro vs Table 1 caption and related work).","section":"Abstract / Table 1"},{"comment":"Bias-alert experience is also called “Calibration Experience” / “deficiency experience” in prompts and appendices; unify terminology with the main-text “Bias-alert Experience (Bia. E.).”","section":"§2.2 and Appendices A–C"},{"comment":"Figure 1 and Figure 2 are dense; a short legend defining Pat. E., Bia. E., proposal pool, and the Pass/Out outcomes would improve readability.","section":"Figures 1–2"},{"comment":"Appendix G efficiency table is useful; consider also reporting online prediction latency (all methods share the same short ˜H_t window) so offline cost is clearly separated from serving cost.","section":"Appendix G"},{"comment":"Related work on experiential agent memory is appropriate; a brief contrast with diagnostic profile methods (e.g., DGDPO cited in §5) on whether they include model-bias alerts would sharpen the novelty claim.","section":"§5 Related Work"}],"recommendation":"major_revision","confidential_remarks":"Solid empirical systems paper with released code and multi-backbone results; suitable for a strong venue after controls that isolate practice from generic memory quality. The dialectical-philosophy framing is rhetorical and can be shortened without loss. Scope fits cs.CL / recommendation / agent memory tracks. No integrity red flags from the text alone."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: PraMem turns the long history into offline practice samples, builds a dual memory (user patterns + model bias alerts), filters proposals with groundedness/generalizability self-review, and only commits consensus updates. On OmniBehavior it lifts overall ACC/F1 from the best advanced baseline (ProEx ~77.7/26.9) to 84.7/31.6 with GPT-OSS-120B, transfers to Qwen3.5, and shows cleaner MovieLens diagonals. Ablations on deep thinking, reflection, consensus, each memory type, and each review stage all move the needle in the expected direction, and memory length plateaus instead of exploding.\n\nWhat is actually new relative to Mem0/MemOS/ProEx and the agent experiential-memory line is the closed practice–reflect–consensus loop plus the two-check self-review (perturb history for groundedness; virtual scenes for generalizability). That is a coherent engineering package, not just another summarizer. Code is linked; temporal order of practice samples is respected; evaluation is held-out next-action prediction. For a recsys / long-horizon agent reader this is real, usable progress.\n\nSoft spots, in proportion: the load-bearing filter is still LLM-judged on LLM-generated perturbations and scenes. Nothing replaces self-review with a non-LLM oracle or a length-matched generic memory, so part of the gain could be high-quality natural-language context rather than strictly practice-derived causal patterns. Offline cost is higher (~56 min/user vs ~25–34 for baselines) and the free parameters (80 rounds, n, m, T, q, k) are fixed rather than stress-swept. Those are real caveats for claiming a general memory paradigm; they do not erase the table gains or the ablations.\n\nThis is for people building LLM recommenders and user simulators who already care about memory design. Math is light (prompted procedures, not theorems); citations cover the right baselines; no circular derivation. I would send it to peer review. Worth reading and citing if you work in this subfield; not a foundational rewrite of memory.","headline":"Solid systems paper: practice-derived dual memory plus self-review beats strong baselines; main open question is whether self-review is causal or just polished LLM text.","tokens_in":23620,"tokens_out":545,"would_cite":true,"duration_ms":5693,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Treating long user histories as practice material yields experiential memory that beats compression-based methods on next-action prediction.","keywords":["long-horizon behavior prediction","experiential memory","large language models","practice-based memory","bias-alert experience","self-review","sequential recommendation","user modeling"],"falsifier":"On held-out users or a new domain, run the same 80-round practice schedule and check whether final pattern-plus-bias memory still lifts next-action accuracy over strong compression baselines; if the lift disappears or memory length explodes without stabilizing, the central claim fails.","tokens_in":23452,"feed_emoji":"🧠","tokens_out":663,"duration_ms":5530,"temperature":0.7,"pith_summary":"Long-horizon behavior prediction asks a model to forecast a user’s next action from a long sequence of past scene–action pairs. Large language models struggle both to extract latent patterns scattered across that sequence and to avoid their own systematic biases. Prior memory systems treat the history mainly as a context burden to compress or retrieve from, so the model still has to induce patterns on the fly and still inherits its biases. This paper reframes the history as a stock of labeled practice samples. By running iterative practice—trial under current memory, reflective proposals, consensus-driven updates, and a self-review filter that checks groundedness and generalizability—it builds a compact experiential memory of two kinds: pattern experience that states the user’s habits and bias-alert experience that warns the model about its own error tendencies. That memory is then supplied as assisted input at prediction time. Experiments on multi-scenario platform behavior and movie ratings show clear gains over truncation, retrieval, summarization, and advanced memory baselines, and the same memory helps a second backbone without being rebuilt.","feed_headline":"Practice on user history beats compression for next-action prediction","feed_subtitle":"PraMem builds pattern and bias-alert memory offline and lifts accuracy across scenarios and models","key_machinery":"PraMem’s iterative practice loop: existing-experience trial with deep thinking on labeled historical segments, reflective proposal generation, consensual experience adjustment every fixed number of rounds, and a self-review that keeps only proposals that fail a history-perturbation groundedness test and pass a virtual-scene generalizability test.","core_discovery":"The paper claims that beforehand practice on segments of a user’s historical sequence can construct a user-specific, time-evolving experiential memory—pattern experience plus bias-alert experience—that resolves the two core failures of LLM long-horizon prediction better than context-compression memory methods, producing higher accuracy and F1 on OmniBehavior and stronger diagonal dominance on MovieLens-1M, with the memory remaining effective when transferred to another backbone.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Practice on history builds experiential memory that beats compression","PraMem turns long user sequences into practice-derived pattern memory","Offline practice memory lifts LLM next-action accuracy over compression","History practice resolves LLM biases better than context compression","PraMem experiential memory outperforms compression on long-horizon tasks"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That later historical segments used as practice samples, plus the model’s own perturbations and virtual scenes, produce proposals whose consensus reliably captures transferable user patterns and model biases rather than sample-specific noise or self-consistency artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Practice on history builds experiential memory that beats compression","PraMem turns long user sequences into practice-derived pattern memory","Offline practice memory lifts LLM next-action accuracy over compression","History practice resolves LLM biases better than context compression","PraMem experiential memory outperforms compression on long-horizon tasks"]},"model":"grok-4.5","effort":"low","cost_usd":0.004262,"raw_usage":{"total_tokens":1239,"prompt_tokens":745,"num_sources_used":0,"completion_tokens":61,"cost_in_usd_ticks":42620000,"prompt_tokens_details":{"text_tokens":745,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":433,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":745,"tokens_out":61,"duration_ms":4358,"temperature":1.0,"reasoning_tokens":433,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T06:24:23.659226+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On held-out users or a new domain, run the same 80-round practice schedule and check whether final pattern-plus-bias memory still lifts next-action accuracy over strong compression baselines; if the lift disappears or memory length explodes without stabilizing, the central claim fails.","supporting_citations":[],"review_version":1}