Artificial Intelligence · arXiv:2610.24453 v2 · submitted 17 Oct 2026
Hippo: Training-Free Agent Self-Evolution via Hippocampal Replay
The paper says
“Motivated by recursive self-improvement (RSI), we study how LLM agents can learn through external memory without updating model parameters.”
more of the abstract
Extracting useful experience from a single attempt is difficult: flawed reasoning can produce success, while poor execution can obscure a sound strategy. Preserving its value over time is also difficult: our accumulation baseline without consolidation initially improves but eventually performs below a no-memory baseline. Our key idea is to compare attempts at the same task to identify reusable decisions and refine when they apply. We introduce Hippo, a memory framework inspired by selective hippocampal encoding and replay that learns from these comparisons when shortcomings are diagnosed. It compares the resulting experience with existing memory to merge overlaps and resolve conflicts through explicit applicability conditions. This allows prior guidance to be revised without storing every lesson as a separate entry. Experiments on SWE-bench Verified, WebArena, and Mind2Web show performance gains in all three settings and substantially fewer retained experiences than the accumulation baseline. The gains extend across the evaluated base models of varying capabilities, indicating that memory-based adaptation remains useful even for highly capable models.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 20 Oct 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up. Well-executed empirical paper on memory-based self-evolution for LLM agents; the core mechanism is novel and the evaluation is thorough, but the stress-test concern about unvalidated memory quality is real and acknowledged by the authors. This line is wrong
3 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the references say.
The paper says, at Section 4.2, Figure 4
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The long-horizon evaluation plots success rates over 400 tasks without any indication of variability. The main benchmark results report mean ± standard deviation over three complete runs, but Figure 4 appears to present a single run. Since learning dynamics can be sensitive to task order and stochasticity, the authors should either report error bars (e.g., over multiple seeds or shuffled task orders) or explicitly state if the figure reflects one run and discuss the potential variance.
more of objection 1
Without this, the robustness of the sustained gains claim is difficult to assess.
The paper says, at Section D.4, Table 15
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The human audit validates only the binary surprise/not-surprise classification of the judge. The quality, relevance, and correctness of the L1 reflections and L2 consolidated experiences—the content that actually drives agent behavior—are not systematically evaluated. While Section H acknowledges shared blind spots as a limitation, the central learning mechanism depends on the quality of these generated entries.
more of objection 2
The case studies (Section F) are illustrative but not representative. A more systematic assessment (e.g., human ratings of a stratified sample of L1/L2 entries for relevance, correctness, and actionability) would strengthen the claim that performance gains are due to the quality of memory content rather than to increased context or inference calls. The authors should either provide such an analysis or explicitly discuss its feasibility and why the current evidence suffices.
The paper says, at Table 5, memory update strategies
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The repeated-rewrite variant shows substantially lower performance (54.57%) than the append-only variant (70.10%) and the proposed add-and-delete (72.90%). This suggests that the rewrite policy's instructions may be suboptimal or that the LLM revises entries incorrectly when given broad freedom. While the paper compares policies, the exact rewrite prompt is only partially described in Appendix G.3.
more of objection 3
Providing the full rewrite prompt and analyzing a few examples of what went wrong (e.g., excessive modification, loss of useful details) would clarify why this policy fails and further justify the proposed restricted deletion policy.
6 minor objections
The paper says, at Abstract (line 1)
Pith says: minor objection 1 of 6
Typo: 'exmaple' should be 'example'.
The paper says, at Section 3.3, paragraph 2
Pith says: minor objection 2 of 6
The paper mentions 'evaluated basedline performance' — 'basedline' is a typo for 'baseline'.
show 4 more
The paper says, at Figure 5 caption
Pith says: minor objection 3 of 6
The figure caption says 'exmaple' — should be 'example'.
The paper says, at Table 2 caption
Pith says: minor objection 4 of 6
Table 2 is referenced in text before Table 1 is fully discussed; reordering may improve flow.
The paper says, at General
Pith says: minor objection 5 of 6
The paper uses both 'L1' and 'L2' in figures and text without consistently defining the abbreviations at first use (they are defined in Section 3.1 and 3.3, but could be more prominent).
The paper says, at Section 4.2, Figure 4
Pith says: minor objection 6 of 6
The y-axis label 'Success rate (%)' would benefit from stating 'on SWE-bench tasks' for clarity.
What would settle it
Human-annotate a random sample of 200 L2 memory entries across all benchmarks for correctness, relevance, and absence of misleading guidance. Then replace Hippo's L2 bank with a version where only entries judged as 'correct and applicable' are retained, and rerun the main experiments (Tables 1-2, Mind2Web). If performance drops significantly (more than one standard deviation from the original), it indicates that the LLM-generated memory quality is critical and that the current audit is insufficient.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 20 Oct 2026
- Reader holds up
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee minor revision: 3 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
This paper does something genuinely useful: it addresses the performance degradation problem in unconsolidated agent memory by introducing surprise-driven selective reflection, multi-rollout contrastive induction, and consolidation with applicability-based merging. The empirical work is strong—three benchmarks, four base models, error bars, and a long-horizon study showing sustained gains while baselines degrade below no-memory. The insight that more attempts alone improve success (9.3 pp absolute, 28.8% normalized) is a practical baseline contribution. The ablation study cleanly separates the contributions of surprise detection, cross-rollout comparison, and memory update strategy.
The weakest point, as the stress-test flags, is that the only human audit (Section D.4, 194 traces) validates the judge's surprise classification but not the quality of the L1 reflections or L2 consolidated experiences. The authors acknowledge shared blind spots (Section H) but do not systematically check whether the generated guidance is correct, generalizable, or non-misleading. The case studies (Appendix F) are illustrative but not representative. Given that the same model serves as agent, judge, and memory writer, this is a real concern—though it doesn't undermine the main result, since the overall performance gains are clear. The paper would be stronger with a broader audit of memory entries.
Minor points: the free parameters (N=8, K=5) are reasonable but not optimized; the cost increase (12.4%) is modest but may not capture all overhead. The paper is honest about these.
Overall: this is a solid, novel contribution that deserves a serious referee. The central claim holds up; the soft spots are acknowledged and do not invalidate the results. I would assign it to an experienced reviewer who can push for a broader audit of memory quality but will recognize the empirical rigor otherwise.
The strongest objection
The paper's weakest point is that the only human audit (Section D.4) checks the judge's binary surprise/not-surprise classification on 194 WebArena traces, but does not evaluate the correctness, relevance, or generalizability of the L1 reflections or the L2 consolidated experiences that actually drive the agent's behavior. The memory formation pipeline (surprise-driven reflection, contrastive experience induction, memory consolidation) all use the same base model that serves as the agent, creating shared blind spots (Section H, b683). Without a systematic quality check on the generated memory entries, it is possible that the observed performance gains are not primarily due to the quality of the extracted guidance, but rather to other factors such as increased context or the additional inference calls. The case studies (Section F) are illustrative but not representative; a broader audit is needed to ensure the central learning mechanism works as intended.
The reader correctly identifies the reliability of LLM components as the weakest assumption. However, the specific gap is that the human audit covers only the judge's binary classification, not the content of the memories that drive agent behavior. The paper acknowledges shared blind spots and provides qualitative case studies, but does not systematically validate memory quality. Despite this gap, the overall empirical evidence across multiple benchmarks, models, ablations, and long-horizon evaluation is strong and consistent. The concern does not overturn the verdict, but highlights a specific area where additional validation would further strengthen the central claim.
The referee's report
The paper addresses a central challenge in building self-improving LLM agents: how to accumulate and refine experience without catastrophic forgetting or degradation. Hippo's approach of selective encoding based on surprise, contrastive experience induction from multiple rollouts, and conflict-resolving consolidation through applicability conditions is well-motivated by cognitive science and clearly operationalized. The empirical evaluation is unusually thorough, covering three diverse benchmarks (coding, interactive web, offline next-action), four base models including very capable ones like GPT-5.6 Sol, three independent runs with standard deviations, and ablations that validate each component. The long-horizon experiment (400 tasks) directly shows that naive accumulation leads to performance collapse while Hippo's consolidation avoids it, which is a significant empirical result. The paper also contributes a practical insight (multiple attempts alone improve performance) that should become a standard baseline. If the results hold, Hippo provides a principled and cost-effective way to equip LLM agents with persistent, reusable memory.
Pith's reading of the paper (a machine's summary, not the authors')
This paper argues that LLM agents can improve their own task performance by comparing multiple attempts at the same task and selectively retaining only the useful experience in an external memory, without any parameter updates. The key idea is that a single attempt is insufficient: a correct outcome may conceal flawed reasoning, while a failed attempt may reflect poor execution of a valid plan. By running multiple rollouts and contrasting successes and failures, Hippo distills reusable decisions and explicit applicability conditions. It then merges or revises existing memory entries to avoid duplication and conflict. The result is a persistently improving agent that stores far fewer entries than prior approaches and avoids the performance degradation observed with simple accumulation.
What carries the argument. The central machinery is the memory evolution engine, which operates in two steps. First, contrastive experience induction compares N parallel rollouts (default 8) of the same task to extract a candidate experience: a decision together with its conditions of use. The comparison handles four cases depending on the ratio of verified successes. Second, memory consolidation compares this candidate against related existing memories via an LLM comparator, which discards duplicates, merges overlaps by making applicability conditions explicit, and deletes replaced entries. This two-stage process ensures that memory remains compact and non-contradictory, avoiding the performance collapse observed when experience is simply appended.
Core claim. Hippo introduces a two-stage memory framework inspired by hippocampal encoding and replay. In the first stage, a 'surprise' judge identifies rollouts that fail or succeed for the wrong reasons; only those produce episodic reflections. In the second stage, the memory evolution engine compares multiple rollouts of the same task to derive a candidate experience, then compares this candidate against existing memory to merge overlaps and resolve conflicts through explicit applicability conditions. This process allows prior guidance to be revised without storing every lesson as a separate entry. On SWE-bench Verified, WebArena, and Mind2Web, Hippo improves task performance across all evaluated base models, including strong ones like GPT-5.6 Sol, while retaining substantially fewer experiences than the accumulation baseline. The method sustains gains over 400 tasks without the inverted-U degradation seen in baselines.
The premise it rests on. The LLM-based judge correctly identifies 'surprise' (failures and successes for wrong reasons), and the same model used for reflection and consolidation reliably produces accurate and generalizable guidance from multi-rollout comparisons, even when all attempts fail or a trace is mislabeled.
If the paper is right (Pith's inferences, not the authors')
Hippo provides a recipe for gradient-free self-improvement that works across diverse task domains (coding, web navigation, offline prediction).
The selective encoding and consolidation mechanism suggests that memory quality, not quantity, drives agent improvement.
Even strong models like GPT-5.6 benefit from memory-based adaptation, indicating that experience reuse remains valuable as base models improve.
The long-horizon results show that Hippo avoids the catastrophic forgetting observed in prior memory methods, enabling sustained learning over hundreds of tasks.
The surprise judge's reliance on the same model as the agent creates a potential blind spot: if the model cannot recognize its own flawed reasoning, incorrect lessons may enter memory. A separate validation model could mitigate this.
The framework could be extended to hierarchical consolidation over longer episodes, addressing the paper's acknowledged limitation on long individual tasks.
The explicit applicability conditions extracted by the LLM comparator may serve as a form of symbolic abstraction that could be transferred to other agents.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Motivated by recursive self-improvement (RSI), we study how LLM agents can learn through external memory without updating model parameters. Extracting useful experience from a single attempt is difficult: flawed reasoning can produce success, while poor execution can obscure a sound strategy. Preserving its value over time is also difficult: our accumulation baseline without consolidation initially improves but eventually performs below a no-memory baseline. Our key idea is to compare attempts at the same task to identify reusable decisions and refine when they apply. We introduce Hippo, a memory framework inspired by selective hippocampal encoding and replay that learns from these comparisons when shortcomings are diagnosed. It compares the resulting experience with existing memory to merge overlaps and resolve conflicts through explicit applicability conditions. This allows prior guidance to be revised without storing every lesson as a separate entry. Experiments on SWE-bench Verified, WebArena, and Mind2Web show performance gains in all three settings and substantially fewer retained experiences than the accumulation baseline. The gains extend across the evaluated base models of varying capabilities, indicating that memory-based adaptation remains useful even for highly capable models.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-20
The authors' main claim, as the classifier picked it out 0.87
“Experiments on SWE-bench Verified, WebArena, and Mind2Web show performance gains in all three settings and substantially fewer retained experiences than the accumulation baseline.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.32 · human participants 0.03 · controlled comparison 0.96 · randomized assignment 0.02 · states quantitative result 0.11 · compares with prior results 0.96 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Motivated by recursive self-improvement (RSI), we study how LLM agents can learn through external memory without updating model parameters. | background 0.54 | 0.38 | 0.04 | asserted 0.57 |
| Extracting useful experience from a single attempt is difficult: flawed reasoning can produce success, while poor execution can obscure a sound strategy. | background 0.75 | 0.20 | 0.02 | asserted 0.98 |
| Preserving its value over time is also difficult: our accumulation baseline without consolidation initially improves but eventually performs below a no-memory baseline. | result 0.69 | 0.90 | 0.04 | asserted 1.00 |
| Our key idea is to compare attempts at the same task to identify reusable decisions and refine when they apply. | contribution 0.53 | 0.30 | 0.10 | asserted 0.95 |
| We introduce Hippo, a memory framework inspired by selective hippocampal encoding and replay that learns from these comparisons when shortcomings are diagnosed. | contribution 0.71 | 0.79 | 0.10 | asserted 0.99 |
| It compares the resulting experience with existing memory to merge overlaps and resolve conflicts through explicit applicability conditions. | method 0.99 | 0.81 | 0.05 | asserted 1.00 |
| This allows prior guidance to be revised without storing every lesson as a separate entry. | method 0.90 | 0.68 | 0.07 | asserted 0.89 |
| Experiments on SWE-bench Verified, WebArena, and Mind2Web show performance gains in all three settings and substantially fewer retained experiences than the accumulation baseline. | result 1.00 | 0.96 | 0.50 | asserted 1.00 |
| The gains extend across the evaluated base models of varying capabilities, indicating that memory-based adaptation remains useful even for highly capable models. | interpretation 0.76 | 0.93 | 0.19 | hedged possibility 0.81 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-20: 1 request, 6,784 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (6)
Figures are the authors', reproduced from the paper.
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment