{"id":"0b836883-1c06-4fca-a7fd-ad2a88a049b3","arxiv_id":"2411.17471","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A recursive least squares framework lets Concept Bottleneck Models add new concepts and classes without gradient updates, reportedly preventing catastrophic forgetting.","lead":"This paper proposes a continual learning method for Concept Bottleneck Models, interpretable AI systems that explain predictions through human-understandable concepts. The method updates the concept and decision layers with recursive linear algebra instead of gradient descent, and the authors claim it retains all past knowledge while adding new concepts and classes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'absolute knowledge memory' proof for the classifier is invalid because the classifier's input features C* are predicted-concept features that change whenever W_c is updated; Eq. 9 assumes fixed features over all phases, so Eq. 13 cannot be equivalent to batch ridge regression.","rationale":"The reader's verdict is REJECT with high confidence, and the weakest assumption identified is that recursive updates inherit a memory guarantee only when features are fixed across phases, which fails for the classifier whose input is the evolving predicted-concept vector. My independent reading reaches the same conclusion: the classifier's feature set C* depends on W_c, which changes every phase, so Eq. 13 cannot equal the batch ridge solution of Eq. 9 over a consistent representation. The paper's own reported class forgetting rates are nonzero, which is direct evidence against the 'absolute knowledge memory' claim. I also agree with the secondary issue that base weights are obtained by backpropagation and are not shown to match the analytic ridge solution assumed by the recursion, further weakening the formal guarantee. No separate objection is needed; these are load-bearing and sufficient to reject the central claim as stated. The recommended verdict remains REJECT, so verdict_should_be is UNCHANGED relative to the reader's assessment.","tokens_in":16974,"tokens_out":4253,"duration_ms":46687,"concrete_test":"Store phase-1 data and, after the final phase, recompute phase-1 classifier features with the final concept weights: C*_1^final = sigma(Z*_1 W_c^final W'_fe). Then fit a batch ridge classifier on all phases using those final-concept features, and evaluate old-task class accuracy. If CONCIL achieved absolute memory, its W_y^final would coincide with this batch ridge classifier and old-task accuracy would match. Any gap quantifies the forgetting caused by the changing W_c. As a companion check, reinitialize the base layers with analytic ridge solutions instead of BP-trained weights and rerun the same comparison to isolate the base-initialization mismatch.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 5.8 is that the recursive updates are equivalent to batch ridge regression over all data, yielding 'absolute knowledge memory.' For the concept layer this equivalence requires that the features Z* are fixed across phases and that the base weights Wc0 are already the ridge solution of Eq. 4. The paper trains Wc0 and Wy0 by backpropagation (Section 5.2) and never shows that they satisfy the analytic normal equations, so even the concept-layer recursion is not guaranteed to reproduce the cumulative ridge solution. The classifier layer fails more structurally. Eq. 9 defines W_y as the batch ridge solution using features C*_i = sigma(C_hat_i W'_fe), where C_hat_i = Z*_i W_c^(i) depends on the concept weights at phase i. These features are not invariant: whenever W_c is updated in Eq. 12, old tasks' classifier features should be recomputed, but old data is unavailable. Eq. 13 instead accumulates Ry and cross-moment summaries from stale C*_i. Hence W_y^(t) is not the ridge solution for any single common feature representation, and the proof of absolute memory does not apply to the classifier. The nonzero class forgetting reported in Table 1 (CONCIL: 0.0919 CUB, 0.1029 AwA) is direct empirical evidence of this mismatch and contradicts the claim that catastrophic forgetting is eliminated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper defines a new continual learning setting for Concept Bottleneck Models (CBMs), called Concept-Incremental and Class-Incremental Continual Learning (CICIL), in which both the concept vocabulary and the class set grow over time. The authors propose CONCIL, a framework that first trains a standard CBM by backpropagation on a base task and then updates the concept layer and classifier through recursive ridge-regression updates, using only current-task data and summary matrices. The central claim is that these recursive updates achieve 'absolute knowledge memory': the model behaves as though it had been trained on all data seen so far, thereby eliminating catastrophic forgetting. The paper reports experiments on CUB and AwA showing higher concept and class accuracy and lower forgetting than a naive fine-tuned CBM baseline. The claimed theoretical guarantee is the main contribution, but it is not established by the presented derivation.","tokens_in":17269,"tokens_out":5824,"duration_ms":69983,"significance":"If the 'absolute knowledge memory' guarantee were correct, the paper would make a useful contribution by providing a computationally efficient, privacy-preserving continual learning method for CBMs, with a new task formulation that reflects realistic dynamic multimodal settings. The recursive matrix updates are an appealing idea, and the paper correctly identifies a gap in the CBM literature. However, the central theoretical claim is invalid as stated: the classifier's input features depend on the continually updated concept layer, so the recursive updates are not equivalent to batch ridge regression over a fixed feature representation. The nonzero class forgetting reported in Table 1 also directly contradicts the 'eliminating catastrophic forgetting' claim. The task definition and empirical comparison to a naive baseline remain of interest, but the paper's headline contribution is not supported.","major_comments":[{"comment":"The objective in Eq. (4) is described as minimizing loss 'over the current task's data,' but Eq. (5) is the closed-form solution of a cumulative loss summed over all tasks i = 0..t. These two statements are inconsistent. The recursive updates in Sections 5.6 and 5.7 are derived from the cumulative objective, so the paper never actually solves the stated per-phase regression problem. The derivation of the claimed memory property therefore lacks a well-defined starting point and needs to be re-stated as an explicit cumulative-loss objective.","section":"§5.3, Eq. (5)"},{"comment":"The update for W_y(t) assumes that the classifier input features C*_i are fixed across phases. However, Eq. (6) defines C*_t = σ(C_hat_t W'_fe) with C_hat_t = Z*_t W_c(t), and W_c(t) is updated at every phase via Eq. (12). For i < t, the features used to build R_y(t-1) are computed with W_c(i), not with the current W_c(t). At test time, the model applies the final W_c(t) to all inputs, so the deployed feature representation differs from the one used to accumulate the summaries. Consequently, Eq. (13) is not equivalent to batch ridge regression over any common feature matrix, and the 'absolute knowledge memory' assertion in Section 5.8(i) is not supported for the classifier.","section":"§5.7, Eq. (13)"},{"comment":"The recursive update formulas are exact only if the base weights W_c(0) and W_y(0) already minimize the corresponding ridge objectives (Eq. (4) and Eq. (8)) on the base data. The paper obtains these weights by backpropagation in Section 5.2 and provides no argument that they satisfy the normal equations. Without this condition, even the concept-layer recursion does not recover the cumulative ridge solution, so the claimed equivalence to centralized training fails at the first update.","section":"§5.2, §5.6"},{"comment":"The CONCIL class forget rates are 0.0919 on CUB and 0.1029 on AwA, which are non-negligible and directly contradict the statement in Section 5.8(i) that catastrophic forgetting is eliminated. The sentence in Section 6.5 that these results 'empirically validate our theoretical claim of absolute knowledge memory' is therefore an overinterpretation; at best the results show low forgetting relative to the naive baseline, not absolute retention.","section":"Table 1, §6.5"}],"minor_comments":[{"comment":"Sections 7 and 10 are verbatim duplicates of the same 'Limitations and Future Work' text; one should be removed.","section":"§7, §10"},{"comment":"The phrase 'if the updated concepts set is an empty set' is unclear; the condition appears to concern the current concept-set dimension being smaller than L_max and should be rewritten.","section":"Eq. (7)"},{"comment":"The notation L_max is used in Eq. (7) before it is defined; please define it at first use.","section":"§5.4"},{"comment":"The experimental section states that p ranges from 2 to 9, but Table 1 reports phases 2 through 9; it should be clarified whether all reported results correspond to p = 9 and how the phase-1 base training is incorporated into the metrics.","section":"§6.2"},{"comment":"The empirical comparison includes only a naive fine-tuned baseline; comparing against an exemplar-free continual learning method adapted to CBMs (e.g., EWC or LwF) would substantially strengthen the claim that CONCIL is a superior continual learning approach.","section":"§6.5"}],"recommendation":"reject","confidential_remarks":"The paper's central theoretical contribution is a claimed proof of 'absolute knowledge memory' via recursive ridge regression. As detailed in the major comments, the proof is invalid because the classifier's input features are not fixed across phases and because the base weights are not shown to satisfy the ridge normal equations. These are load-bearing issues: they invalidate the main claim and cannot be fixed by local edits. The empirical results even show nonzero class forgetting, contradicting the headline claim. I recommend rejection, though the task formulation and the general idea of recursive analytic updates for CBM layers may be worth revisiting in future work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's real contribution is the CICIL task definition — joint concept- and class-incremental learning for CBMs — which I haven't seen elsewhere. Applying recursive least squares to an expanding concept vocabulary is a legitimate extension of analytic learning, and the authors deserve credit for formulating the problem cleanly and showing that a simple recursive update beats a fine-tuned CBM baseline on CUB and AwA. The near-flat concept accuracy across phases is genuinely striking.\n\nThat said, the central claim is not supported. The 'absolute knowledge memory' proof in Section 5.8 assumes the features feeding each layer are fixed across phases. For the concept layer, Z* is fixed because the backbone is frozen and the feature expansion is random, so the Rc recursion would be fine if the base weights were the ridge solution of Eq. 4. But the paper trains the base model with backpropagation and never shows those weights satisfy the analytic normal equations, so even the concept-layer recursion has a gap.\n\nThe classifier fails more structurally. Its input is the predicted concept vector C*, which depends on Wc. Whenever Wc is updated in Eq. 12, old tasks' classifier features should be recomputed, but old data is unavailable. Eq. 13 accumulates Ry and cross-moments from stale features, so Wy is not the batch ridge solution for any single common feature representation. The nonzero class forgetting rates in Table 1 (0.0919 on CUB, 0.1029 on AwA) are direct empirical evidence of this mismatch and contradict the claim that catastrophic forgetting is eliminated.\n\nThe evaluation also lacks error bars and standard continual learning baselines — only a fine-tuned CBM baseline is compared, and the paper's own 'absolute memory' claim should make CONCIL match the joint-training upper bound, which is never reported. There is also a duplicated Limitations and Future Work section (Section 7 and Section 10 are identical), which suggests the manuscript is not fully polished.\n\nNone of this kills the task definition. A corrected paper that either proves the memory guarantee under the actual feature drift or honestly frames the method as a fast, low-forgetting analytic baseline would be worth publishing. As it stands, the headline claim and the theory behind it need major revision. The paper deserves a serious referee — the task is new enough and the experimental setup is clear enough that a careful review could turn it into something useful — but it should not be accepted in its current form.","headline":"A useful new task definition wrapped around an overclaimed memory guarantee: the recursive least-squares update can't deliver 'absolute knowledge memory' because the classifier's input features change across phases.","tokens_in":17813,"tokens_out":1737,"would_cite":false,"duration_ms":18611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Concept Bottleneck Model whose concept and classifier layers are updated by recursive ridge regression can learn new concepts and classes without forgetting old ones, because each update provably matches the solution trained on all data…","keywords":["continual learning","concept bottleneck models","catastrophic forgetting","analytic learning","recursive matrix updates","class-incremental learning","concept-incremental learning","multimodal interpretability"],"falsifier":"Run CONCIL on a small CBM and, after the final phase, solve the exact ridge-regression weights on the full accumulated dataset in one batch; compare the classifier weights and old-task predictions. Any mismatch, or any drop in old-task class accuracy when the concept layer is updated, shows the claimed equivalence to centralized training does not hold exactly.","tokens_in":16768,"feed_emoji":"🧠","tokens_out":8499,"duration_ms":228415,"temperature":0.7,"pith_summary":"This paper defines a new continual-learning setting for Concept Bottleneck Models (CBMs), in which each incoming task brings new classes and new human-understandable concepts while all previously learned concepts and classes must remain available. It proposes CONCIL, which turns the concept layer and classifier updates into recursive regularized least-squares problems, so incremental phases require no gradient-based optimization and no storage of old data. The paper claims this design achieves \"absolute knowledge memory\": the model behaves as if it had been trained once on the union of all past and current data, and therefore does not suffer catastrophic forgetting. A reader should care because it offers a mechanism by which interpretable, concept-based models could be updated continuously in real-world multimodal streams, such as medical or surveillance systems where new classes and attributes appear over time.","feed_headline":"Recursive matrix updates let concept models learn without forgetting","feed_subtitle":"Gradient-free ridge updates keep concept and class accuracy nearly flat over nine incremental phases.","key_machinery":"The carrying object is the recursive regularized least-squares update. For the concept layer, with expanded features $Z_t^*$ and cumulative concepts $C_{\\le t}$, the paper maintains $R_c^{(t)} = (\\sum_{i=0}^t Z_i^{*\\top} Z_i^* + \\lambda_1 I)^{-1}$ and updates it by $R_c^{(t)} = R_c^{(t-1)} - R_c^{(t-1)} Z_t^{*\\top}(I + Z_t^* R_c^{(t-1)} Z_t^{*\\top})^{-1} Z_t^* R_c^{(t-1)}$; the weight update $W_c^{(t)}$ then concatenates a correction for old concepts with a newly fitted block for new concepts. The classifier layer runs the identical mechanism on the expanded predicted concepts $\\hat{C}_t^*$ with its own matrix $R_y^{(t)}$. Two random, fixed feature-expansion maps $\\sigma(z W_{fe})$ and $\\sigma(\\hat{c} W'_{fe})$, 25,000 dimensions in the experiments, give the linear layers enough capacity. This machinery matters because it replaces gradient-based fine-tuning with matrix multiplications and small inversions whose cost depends on feature dimension rather than accumulated data size.","core_discovery":"The central claim is that the hard part of continual learning for CBMs can be moved out of optimization and into linear algebra. After the backbone is frozen, the concept layer is a ridge regression from expanded visual features $z^*$ to an ever-growing concept set, and the classifier is a ridge regression from expanded predicted concepts $\\hat{c}^*$ to an ever-growing class set. Both have closed-form solutions whose sufficient statistics are the regularized inverse correlation matrices $R_c^{(t)}$ and $R_y^{(t)}$; the Sherman-Morrison-Woodbury formula turns the cumulative solutions into recursive updates that use only the current phase's data and the previous matrices. On its own terms, the paper proves that these recursive updates reproduce the batch solution, and its experiments show near-flat concept accuracy and class accuracy across up to nine phases on CUB and AwA, while the fine-tuned CBM baseline's class accuracy collapses from roughly 0.61 to 0.15 on CUB.","pith_inferences":["The paper's proof of absolute memory treats each analytic layer as if its input features were fixed. But updating $W_c^{(t)}$ recomputes the predicted concepts that feed the classifier, so old tasks' classifier features change after every concept update; a strict reading of the guarantee covers the concept layer fully and the classifier only to the extent the concept layer stops moving.","A direct way to test that boundary is to freeze the concept layer after base training, update only the classifier, and compare old-task class accuracy; if forgetting remains nonzero, the remaining source is concept-layer drift rather than classifier update error.","The random 25,000-dimensional feature expansions carry much of the representational load, so their width, activation, and initialization are likely decisive; the paper does not report how sensitive the results are to these choices.","The authors' own limitation section notes that linear layers may miss nonlinear concept-class relations and that inverse-matrix overhead grows with the concept and class vocabulary, so the approach's ceiling is an empirical question for larger, more granular concept sets."],"forward_implications":["Each incremental phase becomes a single-pass matrix update, so adding a task costs no gradient steps and no replay buffer; the computation scales with feature dimensionality rather than total data size.","Because only the inverse correlation matrices are carried forward and raw samples are discarded, the scheme is naturally compatible with privacy and storage constraints in continual multimodal deployments.","If the no-forgetting guarantee holds, a CBM could be updated indefinitely on streams of new classes and attributes while its interpretability bottleneck remains intact, since the concept space itself keeps growing.","On the paper's experiments the gap in class accuracy between CONCIL and the baseline widens with each phase, so the method is most valuable exactly when the number of incremental tasks is large.","The same recursive scheme could be applied to any CBM variant with a concept bottleneck, including post-hoc or label-free CBMs, whenever concept annotations are available per phase."],"supporting_citations":[{"why":"Defines the CBM architecture and base-training recipe that CONCIL extends and compares against.","marker":"[14]"},{"why":"Supplies the analytic-learning paradigm that motivates replacing gradient-based training with closed-form solutions.","marker":"[13]"},{"why":"Shows how to compute network weights directly as a pseudoinverse, the analytic-learning basis for the closed-form updates.","marker":"[25]"},{"why":"Provides the pseudoinverse learning algorithm for feedforward networks, another root of the analytic updates.","marker":"[26]"},{"why":"Provides multilayer analytic learning without iteration, the template for CONCIL's recursive layer-by-layer regression.","marker":"[28]"},{"why":"Defines the incremental-learning task types that the paper extends toward simultaneous concept and class increments.","marker":"[10]"},{"why":"Supplies the CUB birds dataset and attribute labels used to benchmark concept and class retention.","marker":"[29]"},{"why":"Supplies the AwA dataset and attribute labels, the second benchmark for the empirical claims.","marker":"[30]"}],"fun_headline_variants":["Ridge regression updates stop catastrophic forgetting in CBMs","Recursive matrix solutions make concept models remember","Gradient-free concept model updates enable continual learning","Concept models learn continually with pure linear algebra","New method keeps concept models learning without forgetting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The no-forgetting guarantee rests on the assumption that each analytic layer's input features stay fixed across all phases and that base weights already solve the same least-squares problem; the classifier input actually changes whenever the concept layer is updated, and base training is gradient-based rather than closed-form.","fun_headline_variants_meta":{"raw":{"variants":["Ridge regression updates stop catastrophic forgetting in CBMs","Recursive matrix solutions make concept models remember","Gradient-free concept model updates enable continual learning","Concept models learn continually with pure linear algebra","New method keeps concept models learning without forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000352,"raw_usage":{"total_tokens":1935,"prompt_tokens":977,"completion_tokens":958,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":889}},"tokens_in":593,"tokens_out":958,"duration_ms":25262,"temperature":1.0,"reasoning_tokens":889,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:19:24.622873+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CONCIL on a small CBM and, after the final phase, solve the exact ridge-regression weights on the full accumulated dataset in one batch; compare the classifier weights and old-task predictions. Any mismatch, or any drop in old-task class accuracy when the concept layer is updated, shows the claimed equivalence to centralized training does not hold exactly.","supporting_citations":[{"cited_title":"Analytic Learning, pages 237–241","cited_arxiv_id":null,"evidence_quote":"Supplies the analytic-learning paradigm that motivates replacing gradient-based training with closed-form solutions."},{"cited_title":"Learning the pseudoinverse solution to network weights.Neural Networks, 45:94–100, 2013","cited_arxiv_id":null,"evidence_quote":"Shows how to compute network weights directly as a pseudoinverse, the analytic-learning basis for the closed-form updates."},{"cited_title":"Pseudoinverse learning algorithm for feedforward neural networks","cited_arxiv_id":null,"evidence_quote":"Provides the pseudoinverse learning algorithm for feedforward networks, another root of the analytic updates."},{"cited_title":"Noniterative deep learning: Incorporating restricted boltzmann machine into multilayer random weight neural networks","cited_arxiv_id":null,"evidence_quote":"Provides multilayer analytic learning without iteration, the template for CONCIL's recursive layer-by-layer regression."},{"cited_title":"van de Ven, Tinne Tuytelaars, and Andreas S","cited_arxiv_id":null,"evidence_quote":"Defines the incremental-learning task types that the paper extends toward simultaneous concept and class increments."},{"cited_title":"Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly","cited_arxiv_id":null,"evidence_quote":"Supplies the AwA dataset and attribute labels, the second benchmark for the empirical claims."}],"review_version":1}