Pith. sign in

REVIEW 3 major objections 5 minor 56 references

Memory consolidation creates value through cross-domain recombination, not rehearsal, in two architecturally unrelated artificial systems.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 09:42 UTC pith:AQK6YZI6

load-bearing objection Real signal in the neural arm (gold-answer GSM8K transfer, shuffle null), but the abstract keeps a retracted symbolic headline and the 'not rehearsal' claim is untested at the same rank. the 3 major comments →

arxiv 2607.16256 v2 pith:AQK6YZI6 submitted 2026-06-28 cs.LG cs.AIcs.IRcs.NE

Discovery by Dreaming: Cross-Domain Recombination in Artificial Memory

Oliver Zahn , James Evans , David Eagleman This is my paper
classification cs.LG cs.AIcs.IRcs.NE
keywords memory consolidationcross-domain recombinationLoRA fine-tuningcontinual learningsleep-inspired AIsymbolic knowledge replaydiscovery mechanismstructural analogy
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that memory consolidation should be reframed as a discovery mechanism, not an anti-forgetting device. It isolates recombinatory replay and builds it into two systems that share no architecture: a LoRA fine-tuning pipeline and a symbolic knowledge engine. Both produce the same asymmetry: consolidation that juxtaposes knowledge from different domains improves performance or surfaces novel connections, while within-domain rehearsal is null. The neural effect appears only above a capacity threshold, and the symbolic effect depends on structured claims rather than flat text. If true, this would mean reading the literature teaches recall, but producing discovery requires a separate offline phase that recombines knowledge across domains—the computational analog of dreaming.

Core claim

The paper claims that replaying knowledge across domain boundaries produces measurable value in artificial learners, while replaying within a single domain does not. In the neural system, a LoRA fine-tune on an 8B-parameter model at rank 256 improves held-out accuracy by +5.64±2.31 percentage points (5/5 seeds, p=0.0055), with gains concentrated in cross-domain transfer tasks and reaching +14.5pp on unseen math reasoning. In the symbolic system, cross-domain replay of structured knowledge objects surfaces connections an embedding-similarity baseline misses, verified by a scramble control and by placement of known historical bridges in the extreme tail of cross-field similarity. The authors c

What carries the argument

The load-bearing operation is cross-domain replay: in the neural pipeline it is synthetic training data that juxtaposes examples from different domains during fine-tuning; in the symbolic engine it is deliberate co-presentation of knowledge objects from distant fields through an LLM extraction pass. LoRA (low-rank adaptation, a parameter-efficient fine-tuning method) rank acts as the capacity gate—the effect emerges at rank 192 and saturates at 256—and an adversarial shuffle shows that the cross-domain bridge structure, not extra tokens, carries the gain. An informal information bound, G ≤ I(DA;DB|θ) − red(DA;θ) − red(DB;θ), frames why within-domain consolidation cannot produce positive gain

Load-bearing premise

The symbolic arm's central evidence assumes the language model derives cross-domain bridges from the juxtaposed knowledge objects and does not retrieve memorized versions of those discoveries from pretraining; the paper itself flags this as a limitation.

What would settle it

Compare the symbolic engine against a strict temporal holdout: use only pre-cutoff knowledge objects (and an extracting model trained only on pre-cutoff data), then test whether it predicts documented post-cutoff cross-domain discoveries above an embedding-similarity baseline. If it does not, the recombination claim fails. A complementary neural falsifier would be a pre-registered hippocampal-recording study in which within-domain and cross-domain replay events produce statistically indistinguishable transfer coefficients.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If this pattern holds, consolidation phases in lifelong learning should be engineered for cross-domain novelty, not faithful replay of prior data.
  • Fine-tuning at small adapter capacity may silently foreclose consolidation gains; the effect appears only above a rank threshold.
  • The effect is a property of weights, not prompts: prepending cross-domain material to a frontier-scale model reversed the gain, so discovery requires offline restructuring.
  • Knowledge-augmented systems should recombine stored knowledge objects rather than treat stores as passive retrieval targets.
  • Within-domain rehearsal is not a generally effective consolidation strategy; its null result is consistent across base models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A decisive open test is a temporal holdout: train the extracting model only on pre-cutoff knowledge and test whether it predicts documented post-cutoff cross-domain discoveries; if it does not, the symbolic arm's recombination claim would collapse.
  • The capacity-threshold result suggests a practical design rule: below a certain adapter size, cross-domain consolidation can actively harm performance (the paper's 'confusion zone'); this is testable as a deliberate curriculum principle.
  • The information-theoretic bound implies consolidation gain should be predictable from a domain-distance metric computed before training; corpus-level novelty scores could schedule which domain pairs to replay.
  • The paper's hippocampal-recording prediction—that within a single replay event, representational distinctness should correlate with transfer strength at r>0.4—would give biology a concrete marker distinguishing recombination from rehearsal.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that offline memory consolidation is a discovery mechanism: cross-domain recombination during replay creates value, while within-domain rehearsal does not. It presents two implementations: DREAMS, a LoRA fine-tuning pipeline with synthetic replay, and SAPIENCE, a symbolic knowledge-object engine with LLM extraction. The load-bearing neural result is a +5.64pp accuracy gain on Llama-3.1-8B at LoRA r=256 (5/5 seeds, p=0.0055), with null cross-domain effects at lower rank and null within-domain effects at r=128. The symbolic arm reports bridge-surfacing via embedding-distance and scramble-control evidence, plus post-hoc placement of historical discoveries in OpenAlex tails. The paper includes a provenance table retracting earlier single-seed claims and an explicit audit trail.

Significance. If correct, the claim that consolidation creates value through cross-domain recombination, rather than preserving memory, would reframe continual learning, sleep-inspired ML, and CLS theory. The paper has real methodological strengths: multi-seed matched-conditions neural experiments, a gold-answer external transfer check on GSM8K/MMLU-Pro with no LLM judge, an adversarial shuffle null isolating bridge structure, a published adapter-hash reproducibility protocol, and an unusually transparent provenance table. These make the narrow neural effect credible. The broad 'not rehearsal' conclusion and the symbolic recombination mechanism, however, are not yet established at the same standard.

major comments (3)
  1. [§3.2, §3.4, Table 5] The central asymmetry claim — cross-domain consolidation creates value while within-domain rehearsal does not — is not tested at the same adapter capacity in the neural arm. The within-domain null is measured at LoRA r=128 (Δ=-1.8±4.4pp, n=3×150), while the positive cross-domain result is measured at r=256 (Δ=+5.64±2.31pp). Because the cross-domain effect is also null at r=128 (+1.17±3.06pp), rank is the operative variable, and a within-domain rehearsal condition at r=256 is required to attribute the effect to domain crossing rather than capacity. The single-domain control in Table 5 is a structural negative control by construction (both cells share the same corpus, Δ=0.00) and cannot serve as a behavioral rehearsal arm. The adversarial shuffle null (+1.74±0.89pp at r=256) shows that additional tokens alone yield some gain, so a same-rank within-domain rehearsal condition could plausibly
  2. [Abstract vs. §4.3 and Table 6] The abstract retains the 85.7% symbolic headline ('The symbolic arm surfaces novel cross-domain connections at 85.7%, a +21pp gain over baseline'), but Table 6 explicitly retracts this number and §4.3 explains that the 85.7%/64.3% pair came from incompatible per-model generation-and-self-judge runs and that judged connection rates are ceilinged across all generation conditions (all McNemar p=1.0). A retracted load-bearing number cannot appear in the abstract. The symbolic arm should be summarized only with the surviving evidence: the embedding-distance gap, the matched scramble control, and the OpenAlex placement.
  3. [§4.3, §6.3, §6.5] The symbolic arm's mechanism claim is partly self-referential. As the paper acknowledges in §4.3, the evaluating LLM was pretrained on literature containing the historical breakthroughs, so the extraction pass may retrieve memorized patterns rather than deduce bridges from juxtaposed KOs. The external OpenAlex validation is post-hoc: it shows known historical bridges lie in the extreme similarity tail, but it does not demonstrate that SAPIENCE would have surfaced them without memorization. A temporal holdout — training on pre-cutoff KOs and predicting post-cutoff discoveries, as proposed in §6.6 — is necessary to separate recombination from latent retrieval. Without such a test, the two-system convergence claim rests on one neural configuration plus post-hoc placement, and the symbolic arm should be framed accordingly.
minor comments (5)
  1. [Abstract] The '+14.5 pp' gain is described as occurring on 'subtasks explicitly requiring cross-domain transfer,' but GSM8K is an external gold-answer benchmark, not one of the four held-out task families. Rephrase to distinguish external transfer from the internal per-task decomposition.
  2. [Table 2] The r=512 row is labeled 'Positive (hi var)' with p≈0.13 and n=3; 'direction-consistent, underpowered' would be more accurate and less likely to be read as a replication.
  3. [§5.4] The verbal definition of red(D;θ) as 'the expected information θ already encodes about samples from D' suggests mutual information I(D;θ), but the proof of Corollary 1 substitutes red(D;θ)=H(D|θ). These are different quantities; the notation and operational definition should be made consistent or the corollary should be presented only as a heuristic.
  4. [References [30] and §5.8] The author name is misspelled as 'Büzsáki'; the standard spelling is Buzsáki.
  5. [Figure 14 / §5.2] The calibration set has n=32 pairs and the out-of-sample set only n=4; the paper is appropriately cautious in the text, but the figure and caption should explicitly state that the n=4 OOS subset cannot support inference on its own.

Circularity Check

2 steps flagged

Neural headline is independent, but the theory corollary is definitional and the symbolic LLM judge may retrieve memorized discoveries.

specific steps
  1. self definitional [§5.4, Theorem 1, Corollary 1 and Assumptions (v)]
    "Under the operational definition red(D;θ) = H(D|θ), substitution gives G(D, D;θ)≤H(D|θ)−2 red(D;θ) = −red(D;θ)≤0. ... (v) the operational equality H(D|θ) = red(D;θ) in Corollary 1"

    The corollary 'within-domain consolidation cannot have positive gain' is obtained by defining red(D;θ) to be H(D|θ), so the inequality is an algebraic identity under the definition, not a derived prediction. The paper lists the equality as assumption (v), meaning the conclusion is imported into the theorem rather than following from independent information-theoretic content. Because the section is labeled interpretation and the paper says the load-bearing claim is empirical, this is a self-definitional step that lends a formal veneer to the within-domain null without independently proving it.

  2. fitted input called prediction [§4.3, §6.5]
    "because the evaluating LLM was pre-trained on literature encompassing these breakthroughs, it may be retrieving memorized patterns rather than strictly deducing them from the juxtaposed knowledge objects"

    The symbolic arm's 'discovered' cross-domain bridges are identified by an LLM extraction pass whose pretraining already contains the historical breakthroughs. If the LLM retrieves memorized patterns, the output is a function of the evaluator's fitted weights rather than of the cross-domain juxtaposition being tested. The paper itself concedes that a strict temporal holdout is required to 'definitively separate true discovery from latent retrieval,' but does not perform it. The retraction of the 85.7% judged-accuracy headline and the reliance on LLM-judged scramble-control components leave the symbolic discovery claim partially self-referential, even though the external OpenAlex percentile placement is an independent post-hoc consistency check.

full rationale

The central neural-arm result is not circular: the +5.64±2.31pp headline is a paired contrast against a no-consolidation baseline, it transfers to gold-answer GSM8K/MMLU-Pro with no LLM judge, and the shuffled-pairing adversarial null isolates the cross-domain bridge structure from extra tokens. The external OpenAlex percentile placement of known bridges is explicitly described by the paper as 'consistent with rather than proof of the mechanism,' so it is a post-hoc validation, not a fitted prediction. The two genuine circular elements are local and largely acknowledged by the authors. First, Corollary 1's claim that within-domain consolidation cannot have positive gain follows by defining red(D;θ)=H(D|θ); the paper lists this equality as an assumption and calls the section interpretation, so it is not an independent derivation. Second, the symbolic extraction LLM was pre-trained on literature containing the historical discoveries, so the 'surfaced' bridges may be latent retrievals rather than novel recombinations; the paper itself concedes a temporal holdout is needed. These issues undercut the symbolic/interpretive arms but do not reduce the central neural evidence. The r=128 within-domain null versus r=256 cross-domain positive is a capacity-confounded comparison rather than a circularity, and the Table 5 single-domain control is explicitly a by-construction zero, so it cannot carry the claim that within-domain rehearsal is null at the effective rank. Overall, partial circularity in secondary derivations and evaluation, with an independent core empirical result.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 1 invented entities

The central empirical claim rests on a small number of hand-selected operating points (r=256, iter=141), a judge whose absolute scale is unstable, and an external corpus that is acknowledged to risk contamination. The information-theoretic theorem is explicitly non-load-bearing and uses a definitional equality. No new physical entities are introduced; the one new theoretical construct, red(D;θ), is weakly operationalized.

free parameters (3)
  • LoRA rank threshold r=256 = 256 (effect emerges at r=192, saturates at r=256)
    The load-bearing neural result exists only at this rank on Llama-3.1-8B; r=128 is null across three base models. Rank was swept and the saturated value retained as headline (§3.4).
  • Training-iteration 'sweet spot' iter=141 = 141
    The effect disappears at iter=282 and at intermediate sweeps (iter=170/200/230/260 are null); 141 was selected as the empirical effect window for the headline (§5.3).
  • Empty-output seed-exclusion threshold = ≥5% empty outputs
    Seed 123 at r=512 was excluded under this rule; without an externally registered protocol, the threshold is a hand-set criterion affecting reported magnitude (§3.4).
axioms (5)
  • ad hoc to paper red(D;θ)=H(D|θ)
    Operational equality used to make Corollary 1 (within-domain gain ≤0) follow by substitution; the paper calls it verbal rather than measure-theoretic (§5.4, Assumptions iii and v).
  • domain assumption LLM judge (Qwen-72B) scores are a valid measure of held-out accuracy
    The headline depends on this judge; cross-judge Cohen's κ=0.229 and the Sonnet re-judge shift magnitude by ~2pp (§6.5).
  • domain assumption CLS theory predicts replay value comes from novel recombination rather than faithful rehearsal
    Used to frame the neural and symbolic tests; cited from the CLS literature rather than established here (§5.6).
  • domain assumption OpenAlex cross-field embedding distance approximates domain distinctness and novelty
    Used for calibration and for the 50K-paper validation; the paper notes it is one member of a family of distance instruments and needs replication under citation-graph or hypergraph-surprise distance (§4.4, §6.5).
  • domain assumption The two systems are independent enough to support a convergence claim
    They share no code or evaluation metric, but they use the same LLM substrate (Sonnet/Qwen) and the same research group; §6.5 explicitly requests independent replication.
invented entities (1)
  • redundancy term red(D;θ) no independent evidence
    purpose: Quantifies how much a trained model already encodes about a domain; appears in Theorem 1's consolidation-gain bound and in the empirical linear-probe framing.
    No independent measurement outside this paper; operationalized via a linear-probe proxy and by definition set equal to H(D|θ) in the theorem (§5.4).

pith-pipeline@v1.3.0-alltime-deepseek · 33241 in / 18354 out tokens · 171748 ms · 2026-08-02T09:42:26.519633+00:00 · methodology

0 comments
read the original abstract

Dreams splice together people, places, and times that never met. Neuroscience suggests this recombination is not noise, but a function driving insight and creative discovery. This reframes memory consolidation: rather than merely defending against forgetting, its measurable value lies in recombining knowledge across experiences that have not yet co-occurred. We test this directly by isolating the recombinatory-replay mechanism and implementing it in two architecturally unrelated systems: a LoRA fine-tuning pipeline (DREAMS) and a symbolic engine replaying structured knowledge objects (SAPIENCE). Both systems converge on the same finding: cross-domain consolidation creates value, while within-domain rehearsal does not. The symbolic arm surfaces novel cross-domain connections at 85.7%, a +21 percentage point (pp) gain over baseline. The neural arm improves overall by +5.64 pp, but on subtasks explicitly requiring cross-domain transfer (like unseen math reasoning on GSM8K), gains reach +14.5 pp. This effect is a genuine property of the weights--not a prompt artifact--as prepending the same material in-context to a 671B-parameter model actually reverses the gain. We validate this prediction against documented discoveries across 50,000 real papers and state a falsifiable hippocampal-recording prediction to distinguish recombination from rehearsal. Ultimately, this principle is substrate-general, tracking real discovery at scale. Reading the literature teaches a model to recall what it has seen, but producing discovery requires a separate offline phase that recombines knowledge across domains--the computational analog of dreaming. Consolidation is not for remembering, but for discovering.

Figures

Figures reproduced from arXiv: 2607.16256 by David Eagleman, James Evans, Oliver Zahn.

Figure 1
Figure 1. Figure 1: Neural (Dreams) vs symbolic (Sapience) results. (a) Matched-conditions DREAMS at r=128 is null across three base models (Llama-3.1-8B ∆ = +1.17±3.06, Qwen-2.5-72B +0.62/+0.61, Llama-3.3-70B −0.10 ± 1.24 pp; McNemar p > 0.5). (b) Symbolic cross-domain replay surfaces bridges at greater embedding distance than within-domain pairs (field distance vs bridge rank ρ = 0.54, n=32; natural-bridge Cohen’s d = 2.47)… view at source ↗
Figure 2
Figure 2. Figure 2: One held-out evaluation item from eval_summary.json (robustness family, Llama-3.1- 8B r=256, full_dreams, seed 42). Each record pairs a prompt with a gold expected field, the model’s generated output, the judge score, a binary correct flag, and a model_info.path that the reproducibility protocol (Appendix A) cross-checks against the declared base model and rank. The +5.64 pp headline is the paired differen… view at source ↗
Figure 3
Figure 3. Figure 3: Per-seed phase transition along LoRA rank (Llama-3.1-8B, n=5/rank, n=3 at r=512). Dots: per-seed paired ∆; black diamonds: pooled mean. At r=64 and r=128 some seeds occupy the confusion zone (red shading, ∆ < 0); at r=192 all seeds are non-negative; at r=256 and r=512 all are strongly positive. The transition is sharp at the seed level and masked by the pooled mean. model artifact and was excluded) sit wit… view at source ↗
Figure 4
Figure 4. Figure 4: Gold-answer external task transfer: r=128 control vs. r=256 consolidation. For each benchmark, the light-gray bar is the r=128 control cell and the saturated bar is the r=256 consolidation cell (bars: pooled mean across 5 seeds; CI and per-seed dots overlaid on the r=256 bars). The contrast is the point: the r=128 control is near-null (+3.60 pp GSM8K, +1.60 pp MMLU-Pro) while r=256 shows a large GSM8K effe… view at source ↗
Figure 5
Figure 5. Figure 5: Adversarial null: shuffled pairings fail to reproduce the real-bridges effect. At Llama-3.1-8B r=256 with matched training-token count, real bridges produce ∆ = +5.64 ± 2.31 pp (n=5) vs shuffled ∆ = +1.74 ± 0.89 pp (n=3). The +3.90 pp gap is attributable specifically to the cross-domain pairing structure [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Where consolidation reshapes the model. Cosine similarity between full_dreams ∆W and no_consolidation ∆W, per module type, at three LoRA ranks. Error bars are standard error across seeds. The cross-condition cosine is lowest at the query projection (qproj, ∼ 0.66) and highest at the value projection (vproj, ∼ 0.74), with the same ordering preserved at all three ranks. Consolidation training primarily resha… view at source ↗
Figure 7
Figure 7. Figure 7: Per-task decomposition: accuracy gain and degeneracy reduction are partially decoupled across task types. For each of the four task families in the held-out evaluation set, the x-axis is the consolidation-vs-control accuracy gain and the y-axis is the loop-rate reduction. Robustness (the largest accuracy gain) is also near-largest on degeneracy reduction; retention shows the largest degeneracy reduction wi… view at source ↗
Figure 8
Figure 8. Figure 8: Joint phase transition: accuracy and degeneracy emerge together at r ≥ 192. Left: per-seed paired accuracy delta ∆ = ∆acc(full_dreams − no_consolidation) vs LoRA rank. Right: per-seed paired degeneracy reduction ∆loop = loop_rate(no_consolidation) − loop_rate(full_dreams). Both metrics show a “confusion zone” (red shading, ∆ < 0) at r ≤ 128 and a clean positive regime at r ≥ 192. The two effects co-emerge … view at source ↗
Figure 9
Figure 9. Figure 9: LoRA rank dose-response for cross-domain consolidation (Llama-3.1-8B, 5 seeds/rank, 3 at r=512). Three regions: subcapacity (r=64, 128, null), transition (r=192, emerg￾ing), saturated (r=256, 512, positive). The headline is +5.64 ± 2.31 pp at r=256 (p = 0.0055, 5/5 seeds). Secondary x-axis: % trainable parameters. (robustness, GSM8K) carry the neural effect, consistent with a shared cross-domain mechanism … view at source ↗
Figure 10
Figure 10. Figure 10: Per-task decomposition of the r=256 effect (Llama-3.1-8B, n=5 seeds). Bars: pooled ∆ per task with bootstrap 95% CI; dots: per-seed values. The effect concentrates in robust￾ness (+10.3 pp) and procedural (+6.4 pp, n=28/seed). 4 System 2: Symbolic Consolidation (Sapience) 4.1 Method Sapience stores scientific findings as knowledge objects (KOs): discrete claims with subject, predi￾cate, object value, conf… view at source ↗
Figure 11
Figure 11. Figure 11: Cross-architecture × LoRA rank effect-size matrix. Cells: pooled ∆ pp (n seeds shown); hatched cells (“n.t.”) not tested; top row is the Llama-3.1-8B dose-response in heatmap form with the headline r=256 cell bolded. The r=128 null is substrate-general; the r=256 column at 70B/72B is inconclusive within the ±5 pp floor. 4.3 Positive result: cross-domain replay enables discovery Synthetic evaluation. Acros… view at source ↗
Figure 12
Figure 12. Figure 12: Component synergy. Each consolidation strategy alone (replay, edge cases, counterfac [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Three-layer memory architecture. KOs enter the hippocampal store (fast); ANML [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Consolidation value scales with domain distance: calibration set (n=32 unique domain [PITH_FULL_IMAGE:figures/full_fig_p029_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 1 canonical work pages

  1. [1]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Vinyals, A. A. Razavi, A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., “Overcoming catastrophic forgetting in neural networks,”Proceedings of the National Academy of Sciences, vol. 114, pp. 3521–3526, 2017

    2017

  2. [2]

    Efficient lifelong learning with A-GEM,

    A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny, “Efficient lifelong learning with A-GEM,” inInternational Conference on Learning Representations, 2019

    2019

  3. [3]

    Continual learning with deep generative replay,

    H. Shin, J. K. Lee, J. Kim, and J. Kim, “Continual learning with deep generative replay,” in Advances in Neural Information Processing Systems, 2017

    2017

  4. [4]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015

    Pith/arXiv arXiv 2015

  5. [5]

    Progressive neural networks,

    A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Sober, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,”arXiv preprint arXiv:1606.04671, 2016. 34

    Pith/arXiv arXiv 2016

  6. [6]

    Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory,

    J. L. McClelland, B. L. McNaughton, and R. C. O’Reilly, “Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory,”Psychological Review, vol. 102, pp. 419–457, 1995

    1995

  7. [7]

    What learning systems do intelligent agents need? Complementary learning systems theory updated,

    D. Kumaran, D. Hassabis, and J. L. McClelland, “What learning systems do intelligent agents need? Complementary learning systems theory updated,”Trends in Cognitive Sciences, vol. 20, pp. 512–534, 2016

    2016

  8. [8]

    The memory function of sleep,

    S. Diekelmann and J. Born, “The memory function of sleep,”Nature Reviews Neuroscience, vol. 11, pp. 114–126, 2010

    2010

  9. [9]

    Overlapping memory replay during sleep builds cognitive schemata,

    P. A. Lewis and S. J. Durrant, “Overlapping memory replay during sleep builds cognitive schemata,”Trends in Cognitive Sciences, vol. 15, pp. 343–351, 2011

    2011

  10. [10]

    Hippocampal replay is not a simple function of experience,

    A. S. Gupta, M. A. van der Meer, D. S. Touretzky, and A. D. Redish, “Hippocampal replay is not a simple function of experience,”Neuron, vol. 65, pp. 695–705, 2010

    2010

  11. [11]

    Task demands predict a dynamic switch in the content of awake hippocampal replay,

    H. F. Ólafsdóttir, F. Carpenter, and C. Barry, “Task demands predict a dynamic switch in the content of awake hippocampal replay,”Neuron, vol. 97, pp. 685–696, 2018

    2018

  12. [12]

    Sleep inspires insight,

    U. Wagner, S. Gais, H. Haider, R. Verleger, and J. Born, “Sleep inspires insight,”Nature, vol. 427, pp. 352–355, 2004

    2004

  13. [13]

    REM, not incubation, improves creativity by priming associative networks,

    D. J. Cai, S. A. Mednick, E. M. Harrison, J. C. Kanady, and S. C. Mednick, “REM, not incubation, improves creativity by priming associative networks,”Proceedings of the National Academy of Sciences, vol. 106, pp. 10130–10134, 2009

    2009

  14. [14]

    Encoding specificity and retrieval processes in episodic mem- ory,

    E. Tulving and D. M. Thomson, “Encoding specificity and retrieval processes in episodic mem- ory,”Psychological Review, vol. 80, pp. 352–373, 1973

    1973

  15. [15]

    Structure-mapping: A theoretical framework for analogy,

    D. Gentner, “Structure-mapping: A theoretical framework for analogy,”Cognitive Science, vol. 7, pp. 155–170, 1983

    1983

  16. [16]

    Continual learning through synaptic intelligence,

    F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,” in International Conference on Machine Learning, pp. 3987–3995, 2017

    2017

  17. [17]

    PackNet: Adding multiple tasks to a single network by iterative pruning,

    A. Mallya and S. Lazebnik, “PackNet: Adding multiple tasks to a single network by iterative pruning,” inIEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773, 2018

    2018

  18. [18]

    Retrieval-augmented generation for knowledge-intensive NLP tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” inAdvances in Neural Information Processing Systems, 2020

    2020

  19. [19]

    Self-RAG: Learning to retrieve, generate, and critique through self-reflection,

    A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, “Self-RAG: Learning to retrieve, generate, and critique through self-reflection,” inInternational Conference on Learning Representations, 2023

    2023

  20. [20]

    K. J. Holyoak and P. Thagard,Mental Leaps: Analogy in Creative Thought, MIT Press, 1995

    1995

  21. [21]

    The structure-mapping engine: Algorithm and examples,

    B. Falkenhainer, K. D. Forbus, and D. Gentner, “The structure-mapping engine: Algorithm and examples,”Artificial Intelligence, vol. 41, pp. 1–63, 1989. 35

    1989

  22. [22]

    Serendipity and information seeking: An empirical study,

    A. Foster and N. Ford, “Serendipity and information seeking: An empirical study,”Journal of Documentation, vol. 59, pp. 321–340, 2003

    2003

  23. [23]

    Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks,

    T. Tadros, A. K. Bhatt, and G. T. Bhatt, “Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks,”Nature Communications, vol. 13, p. 7742, 2022

    2022

  24. [24]

    Catastrophic interference in connectionist networks: The sequential learning problem,

    M. McCloskey and N. J. Cohen, “Catastrophic interference in connectionist networks: The sequential learning problem,” inPsychology of Learning and Motivation, vol. 24, pp. 109–165, Academic Press, 1989

    1989

  25. [25]

    Catastrophic forgetting in connectionist networks,

    R. M. French, “Catastrophic forgetting in connectionist networks,”Trends in Cognitive Sci- ences, vol. 3, pp. 128–135, 1999

    1999

  26. [26]

    About sleep’s role in memory,

    B. Rasch and J. Born, “About sleep’s role in memory,”Physiological Reviews, vol. 93, pp. 681– 766, 2013

    2013

  27. [27]

    Hippocampal and neocortical contributions to memory: Advances in the complementary learning systems framework,

    R. C. O’Reilly and K. A. Norman, “Hippocampal and neocortical contributions to memory: Advances in the complementary learning systems framework,”Trends in Cognitive Sciences, vol. 6, pp. 505–510, 2002

    2002

  28. [28]

    Awake replay of remote experiences in the hippocampus,

    M. P. Karlsson and L. M. Frank, “Awake replay of remote experiences in the hippocampus,” Nature Neuroscience, vol. 12, pp. 913–918, 2009

    2009

  29. [29]

    Reactivation of hippocampal ensemble memories during sleep,

    M. A. Wilson and B. L. McNaughton, “Reactivation of hippocampal ensemble memories during sleep,”Science, vol. 265, pp. 676–679, 1994

    1994

  30. [30]

    Two-stage model of memory trace formation: A role for ‘noisy’ brain states,

    G. Buzsáki, “Two-stage model of memory trace formation: A role for ‘noisy’ brain states,” Neuron, vol. 68, pp. 362–385, 2010

    2010

  31. [31]

    Human replay spontaneously reorganizes experience,

    Y. Liu, R. J. Dolan, Z. Kurth-Nelson, and T. E. J. Behrens, “Human replay spontaneously reorganizes experience,”Cell, vol. 178, pp. 640–652, 2019

    2019

  32. [32]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agar- wal, K. Slama, A. Ray, et al., “Training language models to follow instructions with human feedback,” inAdvances in Neural Information Processing Systems, 2022

    2022

  33. [33]

    Self- Instruct: Aligning language models with self-generated instructions,

    Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi, “Self- Instruct: Aligning language models with self-generated instructions,” inProceedings of the Annual Meeting of the Association for Computational Linguistics, 2023

    2023

  34. [34]

    Stanford Alpaca: An instruction-following LLaMA model,

    R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford Alpaca: An instruction-following LLaMA model,” Stanford Center for Research on Foundation Models, 2023

    2023

  35. [35]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” inInternational Conference on Learning Rep- resentations, 2022

    2022

  36. [36]

    BioMedLM: A 2.7B parameter language model trained on biomedical text,

    E. Bolton, A. Venigalla, M. Yasunaga, D. Hall, B. Xiong, T. Lee, R. Daneshjou, J. Frankle, P. Rajpurkar, J. Carlson, and C. D. Manning, “BioMedLM: A 2.7B parameter language model trained on biomedical text,”arXiv preprint arXiv:2403.18421, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    FinGPT: Open-source financial large language models,

    H. Yang, X.-Y. Liu, and C. D. Wang, “FinGPT: Open-source financial large language models,” arXiv preprint arXiv:2306.06031, 2023. 36

    arXiv 2023

  38. [38]

    The defensive activation theory: dreaming as a mechanism to prevent takeover of the visual cortex,

    D. M. Eagleman and D. A. Vaughn, “The defensive activation theory: dreaming as a mechanism to prevent takeover of the visual cortex,”Frontiers in Neuroscience, vol. 15, p. 632853, 2021

    2021

  39. [39]

    Unlocking LLM creativity in science through analogical reasoning,

    A. Shen, S. Druckmann, and J. Zou, “Unlocking LLM creativity in science through analogical reasoning,”arXiv preprint arXiv:2605.11258, 2026

    Pith/arXiv arXiv 2026

  40. [40]

    Tradition and innovation in scientists’ research strategies,

    J. G. Foster, A. Rzhetsky, and J. A. Evans, “Tradition and innovation in scientists’ research strategies,”American Sociological Review, vol. 80, no. 5, pp. 875–908, 2015

    2015

  41. [41]

    Surprisingcombinationsofresearchcontentsandcontextsarerelatedto impact and emerge with scientific outsiders from distant disciplines,

    F.ShiandJ.A.Evans, “Surprisingcombinationsofresearchcontentsandcontextsarerelatedto impact and emerge with scientific outsiders from distant disciplines,”Nature Communications, vol. 10, p. 7385, 2019

    2019

  42. [42]

    Accelerating science with human-aware artificial intelligence,

    J. Sourati and J. A. Evans, “Accelerating science with human-aware artificial intelligence,” Nature Human Behaviour, vol. 7, pp. 1682–1696, 2023

    2023

  43. [43]

    Choosing experiments to acceler- ate collective discovery,

    A. Rzhetsky, J. G. Foster, I. T. Foster, and J. A. Evans, “Choosing experiments to acceler- ate collective discovery,”Proceedings of the National Academy of Sciences, vol. 112, no. 47, pp. 14569–14574, 2015

    2015

  44. [44]

    Atypicalcombinationsandscientificimpact,

    B.Uzzi, S.Mukherjee, M.Stringer, andB.Jones, “Atypicalcombinationsandscientificimpact,” Science, vol. 342, no. 6157, pp. 468–472, 2013

    2013

  45. [45]

    New directions in science emerge from disconnection and discord,

    Y. Lin, J. A. Evans, and L. Wu, “New directions in science emerge from disconnection and discord,”Journal of Informetrics, vol. 16, no. 1, p. 101234, 2022

    2022

  46. [46]

    Surprise! Measuring Novelty as Expectation Violation,

    J. G. Foster, F. Shi, and J. A. Evans, “Surprise! Measuring Novelty as Expectation Violation,” working paper, SocArXiv 10.31219/osf.io/2t46f, 2021

    work page doi:10.31219/osf.io/2t46f 2021

  47. [47]

    The geometry of culture: Analyzing the meanings of class through word embeddings,

    A. C. Kozlowski, M. Taddy, and J. A. Evans, “The geometry of culture: Analyzing the meanings of class through word embeddings,”American Sociological Review, vol. 84, no. 5, pp. 905–949, 2019

    2019

  48. [48]

    Slowed canonical progress in large fields of science,

    J. S. G. Chu and J. A. Evans, “Slowed canonical progress in large fields of science,”Proceedings of the National Academy of Sciences, vol. 118, no. 41, p. e2021636118, 2021

    2021

  49. [49]

    Electronic publication and the narrowing of science and scholarship,

    J. A. Evans, “Electronic publication and the narrowing of science and scholarship,”Science, vol. 321, no. 5887, pp. 395–399, 2008

    2008

  50. [50]

    Large teams develop and small teams disrupt science and technology,

    L. Wu, D. Wang, and J. A. Evans, “Large teams develop and small teams disrupt science and technology,”Nature, vol. 566, no. 7744, pp. 378–382, 2019

    2019

  51. [51]

    Thediversity–innovationparadoxinscience,

    B. Hofstra, V. V. Kulkarni, S. Munoz-Najar Galvez, B. He, D. Jurafsky, and D. A. McFarland, “Thediversity–innovationparadoxinscience,”Proceedings of the National Academy of Sciences, vol. 117, no. 17, pp. 9284–9291, 2020

    2020

  52. [52]

    The associative basis of the creative process,

    S. A. Mednick, “The associative basis of the creative process,”Psychological Review, vol. 69, no. 3, pp. 220–232, 1962

    1962

  53. [53]

    Language Models Need Sleep: Learning to Self- Modify and Consolidate Memories,

    A. Behrouz, F. Hashemi, and V. Mirrokni, “Language Models Need Sleep: Learning to Self- Modify and Consolidate Memories,”arXiv preprint arXiv:2606.03979, 2026

    Pith/arXiv arXiv 2026

  54. [54]

    Self-Adapting Language Models,

    A. Zweiger, J. Pari, H. Guo, E. Akyürek, Y. Kim, and P. Agrawal, “Self-Adapting Language Models,”arXiv preprint arXiv:2506.10943, 2025. 37

    arXiv 2025

  55. [55]

    Sleep and the price of plasticity,

    G. Tononi and C. Cirelli, “Sleep and the price of plasticity,”Neuron, vol. 81, no. 1, pp. 12–34, 2014

    2014

  56. [56]

    The role of sleep in cognition and emotion,

    M. P. Walker, “The role of sleep in cognition and emotion,”Annals of the New York Academy of Sciences, vol. 1156, pp. 168–197, 2009. A Reproducibility: adapter and split hashes To enable full third-party verification that the rank-ablation results in §3.4 were produced by the claimed base models and ranks (and not by silent adapter mis-routing, as one pre...

    2009