Pith. sign in

REVIEW 4 major objections 4 minor 29 references

For coding agents, reliable memory is not a store the model consults; it is a harness property that delivers cue-anchored facts at the moment of need.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 08:50 UTC pith:CDERLN75

load-bearing objection Worth reading and worth arguing with: it introduces a genuinely useful measurement (forced-compaction survival curves) and a plausible design principle, but its headline numbers all come from single runs, so the precision is provisional. the 4 major comments →

arxiv 2607.20972 v1 pith:CDERLN75 submitted 2026-07-23 cs.AI cs.SE

Delivery, Not Storage: Cue-Anchored Working Memory as a Harness Property for Coding Agents

Swapnanil Saha This is my paper
classification cs.AI cs.SE
keywords agent memorycue-anchored triggerscontext compactiondelivery over storagecoding agentsdeterministic injectionsummary decaymemory offloading
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that for long-running coding agents, memory is a control-plane problem: the reliable channel is deterministic delivery of cue-anchored facts by the harness, not voluntary storage by the model. In controlled runs on a real coding feature, an agent with a pre-seeded store and connected memory tools made zero voluntary memory calls in 114 turns, while deterministic injection delivered seeded facts at every compact-resume in the decay probe (138 of 138) with zero false alarms. The pointed contrast: ten facts held only in conversation vanished at the first summarization (0/10) and stayed absent from 106 of 108 subsequent summaries, even though the final summary in the injected arm carried none of them. If the paper is right, agent memory should be architected as context construction by the harness, not as documents the model must remember to write and read.

Core claim

The central claim is a negative plus a positive: voluntary memory use by coding agents rounds to zero even when the store is pre-seeded with directly relevant facts, and the product that works is delivery — deterministic, harness-side injection of cue-anchored memories at the moments that structure the session. The paper's key demonstration is the repeated-compaction decay probe: ten facts surviving only in conversation disappear at the first summary and are absent from 106 of 108 summaries, while the same ten facts injected from a harness-owned store arrive intact at all 138 compact-resumes, and the final continuation summary carries zero of them. The conclusion the paper draws is that agen

What carries the argument

The key object is the cue-anchored memory model: each memory is a tuple (content, kind, triggers, scope, decay), with triggers as first-class standing conditions over a composable vocabulary — path, symbol, semantic, event, temporal — evaluated deterministically by the harness at lifecycle moments (session start, prompt submit, pre-tool-use, pre- and post-compaction). Kinds (directive, gotcha, finding, task, reference) provide default delivery when no trigger is explicit; a per-session fire ledger deduplicates and resets at compaction boundaries so anchored facts re-arm; injected content carries provenance framing and staleness checks against ground truth. The work this machinery does is to

Load-bearing premise

The headline numbers rest on a single gated run per arm after documented protocol repairs, with the audit ledger produced by the same author and harness that ran the experiment and with no variance estimate; if either run behaved anomalously or the ledger mis-recorded events, every central contrast — 0/10 at first summary, 106/108 empty, 138/138 delivered — changes.

What would settle it

Run the decay probe with an independent harness, independent audit logging, and a fresh corpus: if conversation-held facts survive the first summarization at a meaningful rate, or if any injected fact fails to appear in the session transcript at a compact-resume, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Memory-equipped coding agents should be judged on deterministic delivery at context boundaries, not on how often they call memory tools; voluntary use measured here rounds to zero even with a seeded store.
  • Harnesses should expose a first-class context-construction plane — session-start, cue-triggered, and post-compaction injection — rather than leaving memory to the model's initiative.
  • Conversation summaries are an unreliable carrier for operational facts: in this setup the summarizer dropped all ten facts at the first boundary in both arms, even when the facts were present in the summarized context.
  • Injected delivery carried all ten facts through 138 compact-resumes with zero false alarms across audited trigger evaluations, so the delivery failure mode is controllable when evaluation is deterministic.
  • A harness tuned for memoryless operation may actively fight a memory-carrying configuration (its thrash guard killed the injected arm under default read caps), so adding a memory tier requires adjusting harness assumptions about context refill pressure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If delivery-not-storage generalizes, the current 'memory tools' product category may be aimed at the wrong target: benchmarks that reward voluntary memory tool use would be measuring a behavior the paper finds near zero, and the design target becomes deterministic context construction by the harness.
  • The decay probe used one small fast model; an editor-level extension is to test whether the first-boundary 0/10 drop and the all-or-nothing summarizer selection replicate with larger models. If some summarizers preserve facts, the design conclusion narrows; the paper's own external-validity caveat flags this.
  • The paper leaves capture unevaluated — the ten facts were seeded by the harness, not discovered. A natural next experiment is automatic capture: detecting situationally-bound operational facts during work and authoring cue-anchored notes without any agent initiative, which the design theory requires.
  • Because trigger conditions are deterministic and audited, they are a predictable attack surface for planted memories (the paper cites a plant-persist-trigger attack class); an extension is to test how well staleness flags and provenance framing defend against a planted note that fires at a sensitive cue.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This manuscript argues that reliable long-horizon memory for coding agents must be a harness property: deterministic, cue-anchored injection of situationally relevant facts, rather than voluntary storage and retrieval by the agent. It develops a two-tier design theory grounded in cognitive psychology, specifies a cue-anchored memory model with composable trigger conditions, implements it in the Vectr daemon with two delivery channels, and evaluates it on a naturalistic coding task plus a repeated-compaction decay probe. The headline claims are that voluntary memory use is near zero even with a pre-seeded store (0 memory operations in 114 turns), that deterministic injection delivered seeded facts at 138 compact-resumes with zero false alarms, that conversation-held facts disappear at the first summary boundary (0/10 in both arms; 106/108 empty N summaries), and that the final summary carried none of the facts while the injected-store endpoint was clean. The paper is unusually candid: it discloses small samples, confounds, tool-ban violations, an unreconciled counter, a grader bug, shared authorship, and protocol repairs. However, the decisive decay-probe evidence comes from one gated run per arm and is self-instrumented through the author's own daemon and audit ledger.

Significance. If the central contrast holds, the paper makes a substantial contribution to agent-memory architecture: it reframes the problem as one of control-plane delivery rather than content-plane storage, provides a concrete mechanism for deterministic cue-anchored injection, and contributes a rare forced-compaction survival measurement for both summary-based and injected memory. The artifact package, audit-ledger discipline, and honest labeling of directional claims are real strengths, and the 138-delivery / 106-empty-summary contrast is a striking, falsifiable behavioral result. The main risk is evidentiary: the paper's own section 7 concedes 'one gated run per decay arm,' and every headline decay number rests on that single pair of runs and on a ledger produced by the author's own daemon. The architectural prescription is plausible and well-motivated, but the empirical support for the categorical 'arrive intact through all 138' claim is not yet at journal strength without independent replication or at least multiple runs with variance estimates.

major comments (4)
  1. [§5.7, §7 (Internal)] The central quantitative claims — 138/138 injected deliveries, 106/108 empty N summaries, 0/10 fact survival at the first boundary in both arms, and the clean M endpoint — all come from exactly one gated run per arm after documented protocol repairs. The paper itself concedes this in §7. No variance estimate exists for any headline number, and a single anomalous run or ledger miscount would change every central contrast. Because the abstract states categorical reliability ('arrive intact through all 138'), this is not a minor issue: the core architectural prescription is currently supported by n=1 per condition, self-instrumented. I would require at least a second independent run per arm, or a third-party audit of the ledger and transcripts, or a substantial scoping of the claims to a proof-of-concept observation rather than a reliability guarantee.
  2. [§5.7, §7 (Internal)] The read-cap asymmetry is a genuine confound: arm M ran at 6k tokens versus arm N at 8k, forced by the harness's thrash guard. This means the M-versus-N contrasts in audit completeness (0.70 vs 0.93), turns, and cost differ not only in the presence of the injected store but also in per-turn context budget. The paper discloses the confound and treats the guard behavior as a finding, but for the arm-level behavioral comparisons it remains an uncontrolled second variable. Since the central delivery claim is about the mechanism, the 138/138 delivery counts are less affected, but the paper's broader M-versus-N behavioral comparisons cannot be cleanly attributed to the memory tier without a symmetric-cap design or an additional control.
  3. [§5.3] The proxy-channel delivery count is not fully reconciled: the proxy status file records 5 injections, while the grading-time daemon audit records 6 PROACTIVE_INJECT events, with the discrepancy 'plausibly the pre-launch injection probe' but unreconcilable because the relevant daemon log was not retained. This is exactly the kind of missing support that matters for the paper's two-channel claim. The native-channel evidence and the decay-probe M arm are stronger, but the 'zero false alarms' and 'delivered in every injection-equipped seeded run' claims should either reconcile this counter or explicitly exclude the proxy channel from exact-count claims.
  4. [§7 (Construct), §6] The claimed novelty of the cue-anchored composition rests on five trigger vocabulary elements, but the graded runs exercise only path, event, and semantic triggers; symbol and temporal triggers are implemented but never fired by any graded run. The paper acknowledges this in §7, yet §6 and the abstract present the full composition as the contribution. As a design contribution this is defensible, but as a validated mechanism it is incomplete. I would like to see at least unit/integration tests demonstrating the symbol and temporal trigger paths, or the claims narrowed to the tested subset.
minor comments (4)
  1. [Abstract] Typo: 'alldeliberately authored, deliberately retrieved' should read 'all deliberately authored, deliberately retrieved'.
  2. [§5.7, Figure 1] The caption says 'two exceptions measure re-injection half-life' but the exceptions are summaries that do carry 10/10, so 'survival half-life' would be clearer; also state that the 10 remaining summaries in the M 4–66 block were 0/10 to make the all-or-nothing pattern immediately visible.
  3. [§5.2] The text says '0 memory calls in 114 turns' for the V arm; the table indicates n=1. Please state at first mention that this is the single seeded-voluntary run (V1), so the sample size is explicit.
  4. [§3] The memory tuple includes a 'decay' field, but the paper never defines a decay schedule for individual memories. If decay is a designed field, a one-sentence explanation would help; if it is aspirational, state that explicitly.

Circularity Check

0 steps flagged

No significant circularity: the central claims are measured behavioral contrasts, not derived quantities.

full rationale

The paper's central claim—'delivery, not storage'—is an empirical behavioral contrast, not a quantity derived from fitted inputs or from a self-referential definition. The M-arm injection success is a treatment check: the ten facts are deliberately seeded and injected by the harness by design, and the measured outcomes (the agent writes them in the final report; summaries drop them) are not forced—the N arm provides the counterfactual where the same facts are absent after compaction. No equation in the paper derives a headline number from a fitted parameter; no parameter is fit to a subset of data and then called a prediction. The repeated-compaction numbers (108 vs. 138 boundaries, 139 deliveries, 0/10 final summaries) are reported measurements with disclosed instrumentation, not implications of the design theory. The paper's acknowledged weakness—'The evaluated implementation and the benchmark harness share an author' (§7)—is a self-measurement validity concern, not a circularity: the audit ledger is the measuring instrument for the mechanism, and the claims are not reduced to the ledger as an input. The literature citations (extended mind, encoding specificity, prospective memory, etc.) are used generatively to motivate architectural choices, not as load-bearing self-citations, and there is no imported uniqueness theorem or ansatz smuggled in via citation. The N-arm 'transcript archaeology' endpoint and the M-arm clean endpoint are adversarially graded and cross-checked, strengthening the non-circularity of the comparison. Therefore, the derivation chain is self-contained and the central claim does not reduce to its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

The paper's empirical claims are measurement-based, so the ledger is dominated by experimental settings and domain assumptions rather than fitted constants. No numbers are fitted to data; the design choices that shape the results are the compaction window, the per-arm read caps, and the injection index budget. The domain assumptions carry the weight: the human-cognition analogy is asserted as generative (§2), a single vendor's agent/model/summarizer is treated as representative, the author's audit log is the ground truth for delivery, and verbatim token match is the survival metric. No new physical entities are introduced; the invented entity is the software design itself, with a falsifiable handle in the published audit excerpts.

free parameters (4)
  • Auto-compaction window (CLI floor) = 100k tokens
    Experimental constant defining how much context accumulates before each forced summary; the paper notes it halves per-boundary content pressure versus stock thresholds and argues this is conservative for summary survival, but it shapes every survival-curve number.
  • Per-arm read cap = 8k (N) / 6k (M)
    The memory arm ran under a tighter read cap forced by the harness thrash guard; disclosed as a confound on the audit-completeness comparison (0.70 vs 0.93).
  • Injection index budget = 3 of 4 seeded notes carried (one dropped)
    The two-tier index design dropped one seeded note in both native runs (a known ranking limitation); delivery completeness is sensitive to it, and the decay probe seeds one-line notes partly for this reason.
  • Semantic-trigger similarity floor = unspecified
    Semantic triggers fire above an unstated similarity floor; part of the trigger vocabulary is exercised in graded runs, but the floor value is not reported anywhere.
axioms (5)
  • domain assumption LLM agents are an appropriate analog for human episodic/prospective memory mechanisms (levels of processing, encoding specificity, involuntary retrieval, extended-mind criteria).
    §2 maps each cognitive mechanism directly to an architectural requirement ('each row forced an architectural decision'); if the analogy is decorative rather than generative, the design theory loses its grounding while the empirical contrast still stands.
  • domain assumption The measured agent product, model family, and summarizer are representative of coding agents generally.
    §5/§7: one agent product (Claude Code), one model family (Sonnet 5 for the matrix, Haiku 4.5 for decay), one corpus (Apache Camel); the all-or-nothing summarizer selection is treated as a stable property rather than a single-vendor artifact.
  • domain assumption Voluntary memory non-use is a property of agent behavior, not of tool discoverability, prompt design, or instructions.
    §5.2: 0 memory calls in 114 turns despite 32 guidance mentions; the conclusion 'voluntary memory does not happen' generalizes beyond the specific tools and instructions used.
  • domain assumption The author's daemon audit log is a faithful record of delivery and trigger-evaluation events.
    §5.3/§5.7: the 138/138 delivery claim and zero-false-alarm counts are certified by audit-logged events from the author's own implementation; excerpts are published, but the log is not independently verifiable without running the daemon.
  • domain assumption Token-level verbatim match is a fair measure of fact survival.
    §7 Construct: paraphrased survival would be missed; the paper argues the all-or-nothing block pattern indicates block-level summarizer behavior, so verbatim matching is not central to the survival claim.
invented entities (1)
  • Cue-anchored working memory (Vectr working-memory store with per-memory trigger conditions and deterministic harness-side injection) independent evidence
    purpose: The second-tier memory channel: store items carry {path, symbol, semantic, event, temporal} triggers, evaluated by the harness at session start, prompt submit, pre-tool-use, pre- and post-compaction, with budgeted, provenance-framed, staleness-checked delivery.
    A software design, not a physical entity. It has a falsifiable handle: the decay probe's 138/138 delivery claim and the zero-false-alarm counts are checkable against the published daemon audit excerpts, transcripts, and hook-attachment pairs.

pith-pipeline@v1.3.0-alltime-deepseek · 11041 in / 16858 out tokens · 164608 ms · 2026-08-01T08:50:03.692116+00:00 · methodology

0 comments
read the original abstract

Coding agents ship with one kind of memory: documents. Instruction files, plan artifacts, and auto-written memory directories are deliberately authored and deliberately retrieved: the agent must choose to write them and choose to read them back. Human expertise runs on a second tier that never gets written down: situationally-bound operational facts (gotchas, locations, local conventions) encoded as a side effect of the work and retrieved involuntarily when the situation cues them. We argue this second tier is the load-bearing one for long-running agents and must be a harness property, not an agent choice. We contribute: (1) a two-tier design theory grounded in the cognitive literature on memory offloading, incidental encoding, and event-based prospective memory, each mapped to an architectural requirement; (2) a cue-anchored memory model where memories carry first-class trigger conditions over a composable vocabulary (path, symbol, semantic, event, temporal), evaluated deterministically by the harness, a composition no surveyed academic or shipped system provides; (3) a controlled evaluation on a real coding task showing that voluntary memory use is near zero even with a pre-seeded store (0 memory operations in 114 turns), that deterministic injection delivered in every seeded run with zero false alarms, and that 39% of intra-session re-reads re-buy content paid for before a compaction boundary; (4) a repeated-compaction decay probe: ten facts held only in conversation vanish at the first summary and stay absent from 106 of 108 compactions, and the deprived agent greps the harness's own session files to rebuild them, while the same facts injected from a harness-owned store arrive intact through all 138 compact-resumes as the final summary carries none. Delivery, not storage, is the product: the reliable memory channel for agents is the one the agent never has to think about.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 12 linked inside Pith

  1. [1]

    arXiv:2606.23459, 2026

    TriggerBench: Investigating Prospective Memory for Large Language Models. arXiv:2606.23459, 2026

    Pith/arXiv arXiv 2026

  2. [2]

    arXiv:2510.01353, 2025

    MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments. arXiv:2510.01353, 2025

    arXiv 2025

  3. [3]

    Are We Ready For An Agent-Native Memory System? arXiv:2606.24775, 2026

    Pith/arXiv arXiv 2026

  4. [4]

    arXiv:2606.12329, 2026

    PROJECTMEM: A Local-First, Event-Sourced Memory and Judgment Layer for AI Coding Agents. arXiv:2606.12329, 2026

    Pith/arXiv arXiv 2026

  5. [5]

    arXiv:2606.16707, 2026

    User as Code: Executable Memory for Personalized Agents. arXiv:2606.16707, 2026

    arXiv 2026

  6. [6]

    arXiv:2606.07297, 2026

    SWE-Explore: Benchmarking How Coding Agents Explore Repositories. arXiv:2606.07297, 2026. 12

    Pith/arXiv arXiv 2026

  7. [7]

    BeyondStaticSummarization: ProactiveMemoryExtractionforLLMAgents.arXiv:2601.04463, 2026

    arXiv 2026

  8. [8]

    arXiv:2605.16045, 2026

    RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents. arXiv:2605.16045, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    arXiv:2603.11768, 2026

    Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Framework. arXiv:2603.11768, 2026

    Pith/arXiv arXiv 2026

  10. [10]

    arXiv:2308.15022, 2023

    Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models. arXiv:2308.15022, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    arXiv:2310.08560, 2023

    MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    arXiv:2504.19413, 2025

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    arXiv:2501.13956, 2025

    Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv:2501.13956, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    arXiv:2502.12110, 2025

    A-MEM: Agentic Memory for LLM Agents. arXiv:2502.12110, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    arXiv:2605.28201, 2026

    Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents. arXiv:2605.28201, 2026

    Pith/arXiv arXiv 2026

  16. [16]

    Persistent Memory Across Context Compactions (59 compactions, built our own)

    anthropics/claude-code issue #34556: “Persistent Memory Across Context Compactions (59 compactions, built our own).” Field report, 2026.https://github.com/anthropics/claude- code/issues/34556

    2026

  17. [17]

    F. I. M. Craik and R. S. Lockhart. Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11:671–684, 1972

    1972

  18. [18]

    T. S. Hyde and J. J. Jenkins. Recall for words as a function of semantic, graphic, and syntactic orienting tasks.Journal of Verbal Learning and Verbal Behavior, 12:471–480, 1973

    1973

  19. [19]

    Tulving and D

    E. Tulving and D. M. Thomson. Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80:352–373, 1973

    1973

  20. [20]

    E. Tulving. Episodic and semantic memory. InOrganization of Memory, Academic Press, 1972

    1972

  21. [21]

    L. R. Squire. Memory systems of the brain: A brief history and current perspective.Neurobiology of Learning and Memory, 82:171–177, 2004

    2004

  22. [22]

    Berntsen.Involuntary Autobiographical Memories: An Introduction to the Unbidden Past

    D. Berntsen.Involuntary Autobiographical Memories: An Introduction to the Unbidden Past. Cambridge University Press, 2009

    2009

  23. [23]

    M. A. McDaniel and G. O. Einstein. Strategic and automatic processes in prospective memory retrieval: A multiprocess framework.Applied Cognitive Psychology, 14:S127–S144, 2000

    2000

  24. [24]

    Clark and D

    A. Clark and D. Chalmers. The extended mind.Analysis, 58:7–19, 1998

    1998

  25. [25]

    D. M. Wegner. Transactive memory: A contemporary analysis of the group mind. InTheories of Group Behavior, Springer, 1987

    1987

  26. [26]

    E. F. Risko and S. J. Gilbert. Cognitive offloading.Trends in Cognitive Sciences, 20:676–688, 2016. 13

    2016

  27. [27]

    Sparrow, J

    B. Sparrow, J. Liu, and D. M. Wegner. Google effects on memory: Cognitive consequences of having information at our fingertips.Science, 333:776–778, 2011

    2011

  28. [28]

    B. C. Storm and S. M. Stone. Saving-enhanced memory: The benefits of saving on the learning and remembering of new information.Psychological Science, 26:182–188, 2015

    2015

  29. [29]

    N. J. Slamecka and P. Graf. The generation effect: Delineation of a phenomenon.Journal of Experimental Psychology: Human Learning and Memory, 4:592–604, 1978. 14

    1978