{"id":"ac1e7b9f-ecad-4cec-896d-fc1c9570df66","arxiv_id":"2608.11095","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Agentic coding prompts grow because the reasoning behind old instructions decays, and comments that preserve that reasoning halt the growth and recover instruction-following.","lead":"This paper studies why agent-coding instruction files like CLAUDE.md grow without bound, tracing it to lost memory of why each instruction was written. It proposes writing those reasons down as comments and shows in controlled tests that this can stop the growth and improve instruction-following.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The treatment arm's deletion-policy addendum, not the comments' content, may drive the reported 66pp reduction; no arm combines the addendum with uninformative comments.","rationale":"The paper makes a strong empirical contribution: the corpus study carefully distinguishes imperfect recall from staleness and fragility, and the inverse-IFEval testbed with known minimum covers is a genuinely useful evaluation idea. The reader's identified weakest assumption, that trial-deletion is ruled out by the O(2^|D|) audit cost, is real; a maintainer can delete an instruction, observe subsequent failures, and revert, so Equation 5 does not follow from imperfect recall alone. However, the most load-bearing concern is the experimental confound in Section 4.2: the informative-comments arm receives not only informative comments but also a detailed deletion-policy addendum that the control and placebo arms do not receive. This means the headline result, that comments encoding latent reasoning halt growth, is not identified. The protocol addendum itself prescribes when to delete, and a maintainer following it could prune effectively even if comments carried no useful information. The existing ablations cannot resolve this because they hold the addendum fixed within the informative-comment family. The reader's verdict of CONDITIONAL already requires an additional arm; my concern reinforces that condition and gives it a precise target. I am not changing the verdict because the requested arm is well-defined and the paper's other contributions stand. The test described would either validate the causal role of comment content or reduce the contribution to a deletion-policy effect, which would be a different and less novel claim.","tokens_in":26750,"tokens_out":7087,"duration_ms":65938,"concrete_test":"Run a new arm, PLACEBO+PROTOCOL, which receives the Figure 7(b) protocol addendum verbatim but with comments drawn from the Figure 7(d) placebo pool, on the same 184 worlds at T=51, at least one seed and preferably three. If PLACEBO+PROTOCOL's excess size is close to TREATMENT's (+1.4%) rather than PLACEBO's (+147.9%), the deletion policy alone explains the result and the causal role of comment content is unsupported; if it matches PLACEBO, the comments' informational content is necessary and the claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 4.2, the TREATMENT arm differs from CONTROL and PLACEBO in two ways at once: it carries informative comments, and it alone receives the protocol addendum of Figure 7(b), which tells the maintainer when to delete instructions ('Delete a directive when its comment's failure kept recurring anyway...' or when it duplicates another directive's target). The generic system prompt of Figure 7(a) gives no such pruning rule. Table 2's 66.2pp excess-size reduction is therefore unidentified: it may come from the addendum's explicit deletion policy, not from latent-reasoning information. The ablations in Table 10 keep the addendum constant within the informed-comment family; they never test the addendum with comment-shaped noise or with no comments. Without that arm, the paper's central claim that prompt comments encode latent reasoning and thereby halt growth is not established. The trial-deletion concern raised by the reader is secondary here: even if trial-deletion were feasible, this confound would remain, and it directly undermines the paper's headline causal claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the unbounded growth of agentic coding context files (CLAUDE.md, AGENTS.md, copilot-instructions.md). It proposes that growth is caused by imperfect recall: the latent reasoning behind an instruction decays over time, and because deleting an instruction without that reasoning risks an unverifiable correctness regression (claimed to cost O(2^|D|) counterfactual probes), rational maintainers only add and rarely delete. The paper formalizes this as \"catastrophic remembering,\" reports an observational study of 247,694 instruction lifetimes from 1,867 GitHub repositories showing that instruction counts grow by a median +226%, that deletion hazard falls with instruction age (log-hazard slope -0.032/commit), and that this decline survives gamma-frailty correction. It then presents a controlled experiment using an inverted IFEval testbed with a known minimum cover, in which prompt comments encoding latent reasoning reduce excess size from +60.4% to -5.8% (66.2pp) at parity constraint satisfaction, and a WildIFEval-based experiment in which comments improve instruction-following by up to 23.1%. The paper concludes that comments encoding latent reasoning halt the ratchet and buy back instruction-following.","tokens_in":26978,"tokens_out":4047,"duration_ms":37405,"significance":"If the central claims hold, the paper identifies a novel and practically important failure mode in agentic prompt maintenance and offers a simple, cheap intervention (comments) that could substantially improve LLM-agent behavior in real repositories. The work is unusually careful in several respects: the observational analysis uses pre-registered gates, explicit competing-risk censoring, conservative bias arguments, gamma-frailty models, and a multi-author interaction test that discriminates imperfect recall from content fragility. The experimental design makes the minimum cover observable, enabling a direct measurement of excess size, and includes placebo arms and ablations. The paper also reports reproducible machinery and explicit limitations. These strengths are substantial. However, the central causal claim of the experiment is threatened by a confound: the treatment arm differs from control in both the content of the comments and an explicit deletion-policy addendum in the system prompt, so the reported 66.2pp reduction is not uniquely attributable to latent-reasoning information.","major_comments":[{"comment":"The treatment arm is confounded with an explicit deletion policy. The informative-comments arm alone receives the protocol addendum of Figure 7(b), which instructs the maintainer when to delete ('Delete a directive when its comment's failure kept recurring anyway... or when it duplicates another directive's target'); control and placebo receive only the generic system prompt of Figure 7(a). The text in §4.2 states that \"Arms differ only in the handoff,\" but this is contradicted by Figure 7, where the system prompts also differ. The 66.2pp excess-size reduction in Table 2 may therefore be produced by the addendum's explicit pruning rule rather than by the latent-reasoning content of the comments. Table 10 holds the addendum constant within the informed-comment family and never tests the addendum with comment-shaped noise or with no comments. Without an arm that combines the addendum with uninformative or absent comments, the paper's central claim that comments encoding latent reasoning halt growth is not established. A factorial design (comments content × deletion-policy addendum) or an explicit control for the addendum is required before the causal reading is licensed.","section":"§4.2, Figure 7, Table 2"},{"comment":"The derived O(2^|D|) cost of safe deletion assumes that an instruction can be removed only after an exhaustive subset audit that estimates Δc(D|d) for every combination of instructions. This rules out the cheap trial-deletion strategy in which a maintainer deletes one instruction, observes subsequent failures or their absence, and re-adds it if needed. Real maintainers—and the agents modeled here, who can add and delete in each round—can implement this strategy, so the claim that deletion \"without risking a correctness regression\" costs O(2^|D|) is too strong. The divergence in Eq. (5) follows from the decay assumption ρ(a)→0 together with this infeasibility claim; if trial deletion is feasible, unbounded growth no longer follows from imperfect recall alone. The paper should address trial deletion explicitly, either by arguing why it is unavailable or prohibitively costly in the settings studied (e.g., one-shot task execution or high regression cost), or by extending the model to allow for probed deletions. As written, the theoretical foundation overstates the necessity of comments.","section":"§2, Eq. (4)–(5)"},{"comment":"The headline experimental effect is largely a two-instruction-cover result. In the |D*|=3 stratum at T=15, the uncommented arm's excess size CI is +15.2% [0.5, 31.8], which grossly overlaps cover, and the pre-registered criterion (i) of Table 11 does not hold in this slice. The paper acknowledges in Limitations that covers \"run to two or three instructions against a median file's 39,\" but this undercuts the generality of the claim \"comments halt growth\": the mechanism is demonstrated only at very small constraint counts, while the corpus observations concern far larger prompts. The pooled numbers in Table 2 combine strata that behave differently, so the 66.2pp reduction is not a stable estimate of the effect at realistic instruction counts. The paper should either present the small-cover scope as a direct limitation in the main text or provide evidence that the effect persists at larger covers, for example by increasing the number of verifiers per world.","section":"§4.2, Table 8, Appendix C.1"}],"minor_comments":[{"comment":"The title \"Why DoesCLAUDE.mdKeep Growing?\" has missing spaces; it should read \"Why Does CLAUDE.md Keep Growing?\". Also \"O(2|D|)\" in the abstract is missing the superscript caret and should appear as O(2^{|D|}).","section":"Title and Abstract"},{"comment":"The notation \"± is half a 95% bootstrap CI, asymmetric\" is confusing because the displayed intervals are not symmetric; it would be clearer to state that intervals are percentile bootstrap intervals and report the lower and upper bounds explicitly.","section":"Table 2"},{"comment":"The description of the inverse-IFEval transform says that the maintainer \"reconstructs D_j from o_j\" in T steps, but it does not say how the number of verifiers (|D*|) is hidden. Appendix B.1 states that covers run to two or three instructions, which is a small discretization; a brief note in the main text that the cover size distribution is {2,3} would make the scope of the experiment transparent earlier.","section":"§4.1, Appendix B.1"},{"comment":"Figure 6's caption says \"3 of 8,541 additions, one per constraint family\" without defining whether these numbers refer to the full experimental run or to a single seed. Adding a sentence that clarifies the counting would improve reproducibility.","section":"§4.2 and Figure 6"},{"comment":"The Limitations section is unusually candid, but it is placed at the end of the paper after the Discussion; in a journal format the key limitations (e.g., small-cover experiment, LLM-judged WildIFEval, single-annotation matcher) should be summarized in a shorter form near the introduction or the corresponding results sections.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The paper is ambitious and contains a great deal of careful empirical work, but the experimental confound in §4.2 is decisive for the central claim as currently framed. If the authors can add the missing factorial arms or otherwise decouple the deletion-policy addendum from the comment content, the paper could be suitable for publication. The theoretical overstatement about O(2^|D|) deletion costs also needs a substantive response. I would not recommend rejection, because the observational study and the experimental infrastructure are valuable and the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The observational half is the real contribution. Tracking 247,694 instruction lifetimes and estimating a deletion hazard that falls with age, surviving frailty adjustments and a multi-author interaction, is careful work. The pre-registered gates, the conservative bias arguments, and the honest limitations section all suggest the corpus result is trustworthy: prompts ratchet, and imperfect recall is the most plausible driver.\n\nThe experimental half has a genuine confound that the stress-test note nails. The treatment arm differs from control in two ways at once: it carries informative comments, and it alone receives the deletion-policy addendum in Figure 7(b), which explicitly tells the maintainer when to delete. Without an arm that combines the addendum with uninformative comments, the 66pp reduction in excess size cannot be attributed to the content of the comments. That is a fixable design flaw, but it is load-bearing for the paper's headline claim.\n\nThe theory section is the weakest part. Equation 5's divergence is essentially a restatement of the assumption that rho(a) tends to zero while additions continue; the O(2^|D|) deletion cost ignores the cheap trial-deletion-and-revert strategy that real maintainers use. So the formal model adds less than it claims. The inverted-IFEval benchmark is a nice idea, and the comment intervention does show a large effect within that testbed, but the regime was found by search, the covers are tiny (2-3 instructions), and the pooled effect is driven by the two-constraint strata.\n\nCredit where due: the ablations, the judge-robustness check, and the release of code and derived tables are all genuine. The paper is honest about its own limitations, sometimes to a fault.\n\nThis paper deserves a serious referee. The observational study is worth publishing on its own, and the experimental claim is potentially important. But the version I read should not be accepted as-is. The authors need to run the missing control arm, qualify the theory's scope, and report the three-constraint stratum separately rather than hiding it in the pooled number. I would send it to peer review and expect major revision.","headline":"A solid observational study of agentic prompt growth, with a real confound in the experimental half that needs fixing before the central causal claim holds.","tokens_in":27506,"tokens_out":1667,"would_cite":true,"duration_ms":17294,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Agentic prompts grow without bound because instructions outlive their rationale, and comments stop it.","keywords":["agentic coding","prompt maintenance","latent reasoning","instruction-following","prompt comments","catastrophic remembering","imperfect recall","context files"],"falsifier":"Run the same inverted-benchmark maintenance loop with a delete-and-revert protocol allowed in the uncommented arm; if that arm also settles near the minimum cover, the imperfect-recall mechanism is not what drives growth. Complement it with a repository search for commits where an instruction deletion was followed by a same-file or same-task revert, which would show trial deletion happening in the wild.","tokens_in":26519,"feed_emoji":"💬","tokens_out":6692,"duration_ms":54548,"temperature":0.7,"pith_summary":"This paper tries to establish that the unbounded growth of agentic coding prompts (files like CLAUDE.md) is caused by imperfect recall, not by changing requirements or fragile instructions, and that a one-line fix exists. The mechanism is a cost asymmetry: adding an instruction is cheap, but once a maintainer forgets why an instruction was added, deleting it safely would require checking an exponential number of alternative instruction sets, so instructions accumulate. The paper shows across 1,867 repositories and 247,694 instruction lifetimes that prompts more than triple, that deletion hazard falls with instruction age, and that wholesale rewrites only reset size before growth resumes. In controlled task worlds with known optimal prompts, comments encoding latent reasoning remove 99.3% of the excess instructions at equal correctness, and on real prompts they improve instruction-following by up to 23.1%. The paper's answer to its title question is that instructions outlive their rationale, and its proposed fix is the oldest in software engineering: write the why down next to the what.","feed_headline":"Agentic prompts triple because their rationale fades; comments fix it","feed_subtitle":"Across 1,867 repositories, instruction files triple; encoding their reason in comments cuts 99.3% of excess size.","key_machinery":"The central object is the instruction's latent reasoning r_d, the unrecorded rationale that made a maintainer add instruction d. Its recoverability ρ(d,a) decays with instruction age a, and flow balance on prompt size, E[Δ|D|] = A_t − Σ_d h(a_d), gives a divergent equilibrium |D_∞| = A/(ρ̄ s) as ρ̄→0. The paper's controllable testbed inverts a verifiable instruction-following benchmark: the benchmark's original instructions become the known minimum cover D⋆, its verifiers become hidden constraints, and a fresh maintainer must reconstruct D⋆ from a lossy objective under censored feedback. In that world the intervention is prompt comments—annotations stripped before execution that pass the why from one maintainer to the next—and they are what turns the unbounded ratchet into settlement near cover.","core_discovery":"Catastrophic remembering is the paper's name for the divergence in which a maintainer keeps instructions that should have been removed, the mirror image of catastrophic forgetting in continual learning. The formal claim is that the equilibrium prompt size diverges as the recoverability of latent reasoning ρ̄(a) decays to zero, even with a constant arrival rate of additions and a fixed constraint set, because the deletion hazard factorizes as h(a)≈ρ̄(a)s(a). The empirical discovery is that in real agentic prompts the deletion hazard falls with instruction age (log-hazard slope −0.032 per commit, steeper in multi-author files), which matches the imperfect-recall signature and contradicts staleness and content-fragility explanations. The experimental discovery is that comments carrying latent reasoning settle maintained prompts at the minimum cover: excess size drops from +211.3% to +1.4% over 51 steps at parity constraint satisfaction, and real-prompt instruction-following rises by 11.6 percentage points.","pith_inferences":["A natural next test is whether trial deletion—delete, observe failures, revert—is cheap enough in practice to bypass the exponential audit entirely; if it is, the divergence rate should depend on revert cost rather than recall alone.","The same decaying-reasoning mechanism may apply to system prompts and agent skill files, which the paper explicitly leaves outside its scope.","Comment fidelity will likely become the next weak point: stale or misleading rationale could reintroduce the code-comment divergence problem, so tooling that detects comment-instruction inconsistency would be a natural extension.","The repository analysis is observational, so a field experiment that gives real maintainers comment syntax and measures growth before and after would test whether the controlled result transfers."],"forward_implications":["If prompt comments encode latent reasoning, agentic prompt files can stabilize near their minimum cover instead of tripling in size.","Instruction-following improves when noisy instructions are pruned, so adding comments buys both smaller prompts and higher compliance.","Deletion policies should be tied to recoverable rationale, not to instruction age or file size, because age-based pruning targets the wrong mechanism.","Wholesale rewrites are a temporary reset, not a cure: growth resumes immediately and faster after a rewrite, so durable maintenance requires preserving reasoning across edits.","As coding agents get more capable at maintaining prompts, uncommented prompts ratchet harder, so the benefit of comments grows with maintainer capability."],"supporting_citations":[{"why":"Supplies the instruction-following benchmark whose inversion makes the minimum cover observable.","marker":"Zhou et al., 2023"},{"why":"Supplies real-world prompt/constraint pairs used to test comment-driven instruction-following gains.","marker":"Lior et al., 2025"},{"why":"Documents that constraint-following degrades as constraints accumulate, the motivation for minimizing prompt size.","marker":"Jiang et al., 2024"},{"why":"Provides the repository frame and file-level growth facts that the new instruction-lifetime analysis builds on.","marker":"Chatlatanagulchai et al., 2025"},{"why":"Shows large language models are distracted by irrelevant context, which underlies the instruction-following gain from prompt comments.","marker":"Shi et al., 2023"},{"why":"Supplies the organizational-rule dynamics used to factorize the deletion hazard and interpret instruction lifetimes.","marker":"March et al., 2000"},{"why":"Defines catastrophic forgetting, the dual concept the paper's 'catastrophic remembering' names itself against.","marker":"McCloskey and Cohen, 1989"}],"fun_headline_variants":["Catastrophic remembering: agentic prompts triple; comments keep them lean","Comments that encode why an instruction exists cut prompt bloat by 99.3%","Agentic prompt growth is catastrophic remembering; comments are the fix","Without comments, agentic instructions pile up; with them, prompts shrink 99.3%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on the assumption that maintainers cannot safely delete an instruction just by trying it and reverting if something breaks; if trial deletion is cheap, the exponential-audit premise and the whole divergence argument lose their force.","fun_headline_variants_meta":{"raw":{"variants":["Catastrophic remembering: agentic prompts triple; comments keep them lean","Comments that encode why an instruction exists cut prompt bloat by 99.3%","Agentic prompt growth is catastrophic remembering; comments are the fix","Without comments, agentic instructions pile up; with them, prompts shrink 99.3%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001105,"raw_usage":{"total_tokens":4636,"prompt_tokens":1001,"completion_tokens":3635,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":3550}},"tokens_in":617,"tokens_out":3635,"duration_ms":25651,"temperature":1.0,"reasoning_tokens":3550,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:22:28.056111+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same inverted-benchmark maintenance loop with a delete-and-revert protocol allowed in the uncommented arm; if that arm also settles near the minimum cover, the imperfect-recall mechanism is not what drives growth. Complement it with a repository search for commits where an instruction deletion was followed by a same-file or same-task revert, which would show trial deletion happening in the wild.","supporting_citations":[{"cited_title":"2024 , url =","cited_arxiv_id":null,"evidence_quote":"Documents that constraint-following degrades as constraints accumulate, the motivation for minimizing prompt size."}],"review_version":1}