{"id":"367410ab-21c3-438a-a9ac-847cecae6c86","arxiv_id":"2608.06216","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Continual learning is reframed as continual capability evolution along three axes: when learning happens, where capability is stored, and how updates are driven.","lead":"A new survey argues that continual learning is moving from updating a single model's parameters to adapting a whole system across the model's lifecycle, its memory, skills, and protocols. It proposes a When-Where-How taxonomy to organize this shift and maps recent methods onto it.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"How-axis over-relies on a single KL-minimization result and conflates data source with update mechanism.","rationale":"The paper is a survey and re-framing, not an empirical study, so its central claim is a proposed taxonomy. The When and Where axes are well supported by the surveyed methods and by the paper's own careful caveats in Section 3.5 and Section 5. The How axis is the least secure because it makes a structural claim about on-policy learning resting mainly on one cited result. The reader's weakest assumption captures the generalization risk correctly. I additionally flag an internal consistency issue: SDFT is classified as off-policy despite using self-generated data, so the axis conflates data provenance with update mechanism unless an extra reward-conditioning criterion is imposed. These concerns do not justify rejecting the paper; they justify the CONDITIONAL verdict already given. A controlled four-arm experiment would settle whether the structural claim survives, and whether the How axis needs to be re-grounded in explicit KL regularization rather than an intrinsic on-policy property.","tokens_in":32323,"tokens_out":7921,"duration_ms":71376,"concrete_test":"Re-run a continual post-training benchmark (e.g., TRACE) with four arms: (1) on-policy RL with the standard KL penalty; (2) on-policy RL with the KL penalty removed; (3) SFT on self-generated rollouts (i.e., on-policy data with a supervised loss); (4) SFT on fixed data with an explicit KL penalty to the reference. Measure backward transfer/forgetting and KL divergence to the reference policy after each task. If arm 2 forgets as much as arm 4, or arm 4 forgets as little as arm 1, then the \"implicit min-KL\" property is not intrinsic to on-policy updates, and Section 3.4.1's structural claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing point is the How axis. Section 3.4.1 asserts, on the basis of Shenfeld et al. [27], that on-policy RL is \"not merely an additional mechanism but one structurally aligned with\" forgetting prevention, because it implicitly picks the minimum-KL solution while off-policy SFT can drift arbitrarily. This single result is being asked to carry the on-policy/off-policy distinction as a fundamental dimension. Two problems make that insecure. First, the result is a property of a particular RL objective and reward/training setup; the paper extends it to all on-policy post-training and even to test-time RL (TTRL) without showing that the KL-minimization mechanism survives noisy rewards, weak verifiers, or absent explicit KL penalties. Second, the axis is internally inconsistent: SDFT [161] is placed under off-policy even though it trains on the model's own self-generated responses; the definition of on-policy is quietly narrowed to \"reward-conditioned self-generation.\" Thus the How axis is not really about update mechanism, or even about data source—it is a two-property grouping that depends on [27] to give it coherence. If the KL-minimization claim does not generalize, \"on-policy\" becomes just a label for reward-driven self-training, and the taxonomy loses one of its three pillars.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This survey argues that continual learning (CL) in the era of large language models and agentic AI is transitioning from parameter-centric sequential weight updates to system-level adaptation across the full model lifecycle. The authors propose a tri-axial taxonomy: When capability evolves (pre-training, post-training, inference time), Where capability is carried (parameters versus the external harness of memory, skills, and protocols), and How updates are driven (off-policy gradients, on-policy learning, and learning beyond gradients). Classical CL is positioned as one point in this space, and representative LLM-era methods are profiled along the axes, with a benchmark catalogue and cross-dimensional profiles. The paper also discusses structural ceilings of context, why harness engineering alone is insufficient, coordinated model-harness evolution, long-horizon agents as testbeds, and AGI priorities.","tokens_in":32657,"tokens_out":8737,"duration_ms":73863,"significance":"If the taxonomy is accepted, it provides a useful common language for organizing a fragmented literature and broadens CL's problem statement from forgetting suppression to lifecycle-level system design. The paper is honest about its scope: profiles are non-exclusive, figures are non-exhaustive, and limitations are explicitly stated. Its synthesis of memory, skills, protocols, inference-time updates, and gradient-free mechanisms in one framework is a genuine contribution, and the benchmark catalogue (Table B.1) and cross-dimensional profiles (Table C.1) are useful reference material. The main weakness is the How axis, where the on-policy/off-policy distinction is made to carry more weight than the cited evidence supports; this does not invalidate the taxonomy, but the structural claim needs revision before it can be accepted as stated.","major_comments":[{"comment":"The paragraph beginning 'The genuinely new mechanism that the LLM era brings to the How axis is on-policy learning' elevates a single empirical result, Shenfeld et al. [27], to a structural property: it asserts that on-policy RL implicitly prefers the minimum-KL solution and therefore forgets less, while off-policy supervised fine-tuning can drift arbitrarily far. The paper then extends this to all on-policy post-training and to TTRL [164] at test time without showing that the KL-minimization mechanism survives noisy rewards, weak verifiers, or settings without explicit KL penalties. This is load-bearing because the How axis is one of the paper's three pillars; if the claim is only an empirical regularity, the axis should be presented as such, with stated boundary conditions or additional independent evidence.","section":"Section 3.4.1"},{"comment":"The off-policy/on-policy distinction is operationalized as 'training data is generated by the very policy being updated, conditioned on reward signals,' which conflates data source with reward conditioning. This makes the placement of SDFT [161] as off-policy appear contradictory: its data are self-generated, but collected before training from a fixed reference policy, whereas on-policy self-distillation [29] uses the current policy. The reader is left to infer that reward conditioning, not the update mechanism, is the actual differentiator. Please give a single precise definition and apply it uniformly, explicitly distinguishing the two SDFT methods, or separate 'data source' and 'reward conditioning' into independent properties; the paper's own limitation statement about boundary crossings concerns cross-axis methods and does not resolve this definitional ambiguity.","section":"Section 3.4.1, Table C.1, and Section 5"}],"minor_comments":[{"comment":"The title contains an unintended space in 'T ransition,' and the first sentence of Section 5 reads 'his survey reframes' instead of 'This survey reframes.'","section":"Title and Section 5"},{"comment":"The sentence 'Figures 2 contains selected representative methods and do not result from an exhaustive literature-enumeration protocol' is ungrammatical; please revise to 'Figure 2 contains selected representative methods and does not result from an exhaustive enumeration protocol.'","section":"Section 3.5 and Figure 2 caption"},{"comment":"The distinction between SDFT [161] and the on-policy self-distillation method [29] is made only in Table A.2; readers of Section 3.4.1 will benefit from an explicit note in the main text that the two methods differ in whether the self-generated data come from a fixed reference policy or from the policy being updated.","section":"Section 3.4.1 and Table A.2"},{"comment":"The 'needle-in-a-haystack' phenomenon is invoked without a citation; please include the original or a representative reference.","section":"Section 4.1"},{"comment":"The phrase describing ZeroFlow as having an 'intrinsic stability property paralleling the small-KL behavior of on-policy RL' is stronger than the summarized evidence; if the on-policy claim is softened in Section 3.4.1, this parallel should be softened accordingly.","section":"Section 3.4.2"},{"comment":"References [32], [188], and [191] are blog posts or essays; they are labeled in the bibliography, but the in-text citations should also identify them as non-archival commentary at first mention.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"Several representative methods in Figures 2 and 3 come from the author groups (e.g., SRPO, ZeroFlow, SEEKR, ReCALL, MLLM-CTBench, and others). Given the paper's explicit non-exhaustive caveat, I do not regard this as disqualifying, but the editors may wish to ask the authors to state their inclusion criteria so that the density and empty-region observations are not read as a systematic census."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hou et al.'s survey is the most serious attempt I've seen to re-organize continual learning around something other than the classical replay/regularization/architecture/gradient taxonomy. The When-Where-How frame (lifecycle stage, capability carrier, update mechanism) genuinely brings test-time training, memory/skill/protocol harnesses, model merging, and RL-based post-training into one coordinate system. That is a real service. The paper is also careful about its own limits: figures are labeled non-exhaustive, profiles non-exclusive, and the limitations section is honest.\n\nThe soft spot is the How axis. Section 3.4.1 asks a single finding—Shenfeld et al.'s small-KL property of on-policy RL—to carry the claim that on-policy is structurally aligned with forgetting prevention. That is a lot of weight for one result, and the paper generalizes it to test-time RL without showing the KL-minimization mechanism survives noisy rewards or weak verifiers. More basically, the off-policy/on-policy distinction is really doing two jobs: data source and update mechanism. SDFT trains on the model's own generations but is classed off-policy because it lacks reward conditioning; that is defensible, but it means 'on-policy' is really 'reward-conditioned self-generation,' not a fundamental axis of update mechanisms. I'd want that reframed or hedged.\n\nAlso fair to flag: the representative-method selection leans heavily on the authors' own groups (SRPO, ZeroFlow, SEEKR, ReCALL, MLLM-CTBench all appear with co-authors here). The paper says the figures aren't exhaustive but never explains the selection procedure, and the self-bias reduces the utility of the density/emptiness claims. Minor: ref [40] is a federated scheduling paper, irrelevant where cited. Easily fixed.\n\nThe structural-ceiling discussion in Section 4.1 is more essay than proof, but it is clearly labeled as a position, so I don't hold it against the taxonomy. The core taxonomy is internally consistent and non-circular—it is an interpretation, not a derived result. I would send this to peer review. A good referee can get the authors to rework the How axis, add selection criteria, and fix the citation. The When/Where/How framing will be useful even if the specific on-policy claim does not generalize.","headline":"Genuinely useful When-Where-How reframing of continual learning, with a How axis that overextends one RL finding and a method selection tilted toward the authors' own work.","tokens_in":33113,"tokens_out":3610,"would_cite":true,"duration_ms":28885,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Continual learning is transitioning from parameter-centric learning to system-level adaptation.","keywords":["continual learning","large language models","agentic AI","catastrophic forgetting","test-time training","model merging","memory systems","skill libraries"],"falsifier":"Run the same sequence of diverse post-training tasks twice, once with off-policy supervised fine-tuning on static data and once with on-policy reinforcement learning over current-policy rollouts, holding final-task performance equal and measuring both KL divergence from the reference policy and retention on earlier tasks; if any off-policy method constrained to the same small KL forgets as little as on-policy learning, the claimed structural advantage of the How axis is refuted.","tokens_in":32100,"feed_emoji":"🔄","tokens_out":7412,"duration_ms":59062,"temperature":0.7,"pith_summary":"This survey argues that continual learning is outgrowing its classical definition as the problem of suppressing catastrophic forgetting in a single model during sequential training. It claims the field is instead becoming a question of how a whole system — model plus external memory, skills, and interaction protocols — keeps gaining capability over its full lifecycle. The paper organizes that transition along three axes: when capability evolves, where it is carried, and how updates are driven. Classical continual learning appears as one specific point on these axes. If the framing is right, forgetting is one narrow sub-problem inside a larger design question: deciding how a system should store, update, combine, and release capability across its components.","feed_headline":"Three axes now define continual learning","feed_subtitle":"A new survey maps the shift from weight updates to lifelong systems with memory, skills, and protocols.","key_machinery":"The central machinery is the When–Where–How profile. When names the lifecycle stage at which capability changes, spanning pre-training, post-training, and inference-time; Where names the carrier, either model parameters or the external harness of memory, skills, and protocols; How names the update mechanism, from off-policy gradient updates and on-policy gradient updates to learning beyond gradients, such as model merging, zeroth-order optimization, heuristic learning, and prompt evolution. This profile does the work of placing each surveyed method at a coordinate and making dense and empty regions visible, so the framework both organizes existing methods and points to underexplored combinations.","core_discovery":"The paper's central claim is that the field of continual learning is transitioning from parameter-centric learning to system-level adaptation. The evidence cited is the convergence of several recently developed method families: continued pre-training, multi-stage post-training, test-time training, reinforcement-learning-based alignment, model merging, zeroth-order optimization, and harness-level accumulation through memory, skills, and protocols. Each family relaxes one of three assumptions the classical setting held fixed: learning happens only in a dedicated training stage; capability is carried only in model parameters; and updates are driven only by gradients over externally supplied data. The paper therefore recasts continual learning as continual capability evolution, characterized by when, where, and how learning occurs, with classical continual learning as one coordinate point in this larger space.","pith_inferences":["If the tri-axial frame is adopted, single-number forgetting scores become less informative; a natural next step is to require each method to report a (when, where, how) profile and separate retention metrics per carrier.","The survey's open scheduling question — what to keep in context, write to memory, promote to skills, and consolidate into parameters — has a formal analog in memory-hierarchy management and could be made testable by measuring the cost and retention of different carrier assignments on long-horizon agent tasks.","The paper does not run this comparison, but its KL-based account implies that an off-policy update with an explicit small-KL constraint should close much of the forgetting gap with on-policy learning; if it does, the on-policy/off-policy axis is a proxy for update conservatism rather than a distinct mechanism.","The paper's position that harness engineering is not yet a learning mechanism implies a testable criterion: a harness-based system counts as continually learning only if it improves without human edits to its memory, skills, or protocols over a held-out horizon."],"forward_implications":["Forgetting becomes multi-faceted: not just parameter overwriting, but retrieval decay in memory, context-window overflow, and capability mismatch in skills and protocols.","Evaluation should move toward long-horizon agent trajectories and per-carrier metrics, since short static benchmarks measure shipped capability rather than capability gained during operation.","Hand-tuned harness engineering can look like continual learning but does not mechanistically constitute it, because the agent of revision is a person rather than the system's own closed loop.","A key open mechanism is coordinated evolution of model and harness: bidirectional transfer between parameters and external carriers, scheduling across carriers, and active forgetting as capability release.","The survey prioritizes continual learning for the frontier model over domain-specific continual learning on the path to AGI."],"supporting_citations":[{"why":"Supplies the classical continual-learning taxonomy and stability-plasticity framing that the paper identifies with the parameter-centric reference point.","marker":"[20]"},{"why":"Defines classical continual learning for large language models as the reference setting that the When–Where–How framework extends.","marker":"[21]"},{"why":"Supplies the key structural result for the How axis: on-policy reinforcement learning implicitly minimizes KL divergence to the reference policy and therefore forgets less than off-policy fine-tuning.","marker":"[27]"},{"why":"Defines the harness layer and its three components, memory, skills, and protocols, which anchor the Where axis.","marker":"[36]"},{"why":"Demonstrates inference-time parameter updates through test-time training, making the inference-time position on the When axis concrete.","marker":"[35]"},{"why":"Shows capability accumulation in an external skill library, supporting the harness carrier on the Where axis.","marker":"[13]"},{"why":"Carries the on-policy principle to inference-time learning, linking the When and How axes.","marker":"[164]"},{"why":"Demonstrates parameter and harness updates driven by on-policy reinforcement learning, one of the survey's clearest multi-axis examples.","marker":"[17]"},{"why":"Supplies the model-merging lineage that anchors the beyond-gradients branch of the How axis.","marker":"[30]"}],"fun_headline_variants":["Continual learning goes beyond parameters","From weights to systems: CL's new axes","When, How, Where: the new CL triad","System-level adaptation is the new CL","Continual learning: from params to systems"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The How axis rests on a single cited result — on-policy reinforcement learning forgets less than off-policy fine-tuning because it implicitly makes updates with minimal KL divergence from the reference policy — and if that result does not hold in the broader post-training and inference-time settings surveyed here, the off-policy/on-policy distinction is not the fundamental axis the paper claims.","fun_headline_variants_meta":{"raw":{"variants":["Continual learning goes beyond parameters","From weights to systems: CL's new axes","When, How, Where: the new CL triad","System-level adaptation is the new CL","Continual learning: from params to systems"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000733,"raw_usage":{"total_tokens":3251,"prompt_tokens":889,"completion_tokens":2362,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":2296}},"tokens_in":505,"tokens_out":2362,"duration_ms":15108,"temperature":1.0,"reasoning_tokens":2296,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:33:47.501876+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same sequence of diverse post-training tasks twice, once with off-policy supervised fine-tuning on static data and once with on-policy reinforcement learning over current-policy rollouts, holding final-task performance equal and measuring both KL divergence from the reference policy and retention on earlier tasks; if any off-policy method constrained to the same small KL forgets as little as on-policy learning, the claimed structural advantage of the How axis is refuted.","supporting_citations":[],"review_version":2}