{"id":"9c1ff137-761b-4a7d-8a7f-b49dcd158f35","arxiv_id":"2505.12245","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":1,"one_line_summary":"AFCL solves federated continual learning without gradients by aggregating local Gram matrices and label statistics, proving exact spatio-temporal invariance: the global model equals centralized joint learning for any data partition.","lead":"AFCL replaces gradient-based training in federated continual learning with closed-form least-squares updates on frozen pre-trained features, so the server-side model is provably identical to centralized training on the pooled data. It offers a fast and communication-light way to learn a classifier over an evolving set of classes without catastrophic forgetting.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's order-invariance proof ignores the dynamic column permutation of one-hot labels; exact numerical identity of W under client reordering is not established.","rationale":"The reader's weakest assumption is the quality of the frozen backbone, which is a disclosed practical limitation rather than a flaw in the mathematical claim. By contrast, the soft spot I find is internal to Theorem 2: the proof of order invariance only permutes rows of the pooled data while keeping the one-hot label columns fixed, but the algorithm's known-unknown splitting makes the label column order depend on client registration order. This does not destroy the substantive property, since the per-class coefficient vectors are the same in any order, but it means the theorem's 'identical global model independent of client registration order' is only true up to a permutation of output columns, a nuance the paper does not state. The algebra of Lemma 2 and Theorem 1 appears sound; the gap is in the mapping from the algorithm to the permutation argument. Because the central invariance property can be repaired by restating the theorem in terms of output-column permutation, the conditional verdict remains appropriate rather than being moved to rejection.","tokens_in":24808,"tokens_out":14166,"duration_ms":156830,"concrete_test":"Run Algorithm 1 with three clients whose class sets are, say, {a}, {b}, {c}, and also with one overlapping class such as {a}, {b,c}, {a,c}, using fixed synthetic features. Execute two registration orders and compare the final W computed by (10) against the closed-form (14) for each order, then compare the two orders after aligning columns by class identity. If the W matrices match only after column alignment, Theorem 2's exact numerical identity claim should be weakened to invariance up to output-column permutation; if they do not match even after alignment, the recursive aggregation itself has a bug.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix D proves order invariance by treating client reordering as a row permutation of the pooled matrices: F* = P F, Y* = P Y, with the same label column order. But under the known-unknown class splitting of Section 3.2, the server constructs the one-hot mapping incrementally from registration order: Onehot_{k+1} appends newly introduced classes as new columns in first-appearance order. Swapping two clients can change which classes are 'known' versus 'unknown' at each round, thereby changing the column indices of all classes in Y and hence in W. The permutation proof never models this column permutation; it only establishes invariance under row permutations with a fixed global class ordering. As a literal numerical statement, Theorem 2 is therefore not proved, and in fact W matrices produced under two different registration orders differ by the class-column permutation induced by first-appearance order. What does hold, and follows from Lemma 2 and Theorem 1, is invariance of the predictive function up to a permutation of output columns. Because the abstract and Theorem 2 advertise the stronger 'identical and invariant' claim, this proof gap is load-bearing for the central claim as stated.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"AFCL is a federated continual learning method that avoids gradient updates entirely. Each client extracts features with a frozen pretrained backbone, solves a ridge-regression classifier for 'known' and 'unknown' classes, and uploads local weights plus regularized Gram matrices. The server recursively updates a global knowledge matrix and then converts it to the global model. The paper proves (Lemma 2, Theorem 1) that the recursive update equals the closed-form centralized ridge solution over all data seen so far, and claims (Theorem 2) that the final model is invariant to data partitioning and client order. Experiments on CIFAR-100, Tiny-ImageNet, and ImageNet-R compare AFCL against seven baselines and show large accuracy and runtime gains, with AFCL's accuracy exactly constant across heterogeneity settings for each dataset.","tokens_in":25023,"tokens_out":9093,"duration_ms":94431,"significance":"Theorem 1 is a genuinely useful result: it gives a simple, single-pass aggregation that exactly reproduces centralized ridge regression over non-IID federated data, and the induction proof in Appendix C is self-contained and correct. The known-unknown class splitting extends analytic learning to class-continual FCL without storing raw data. The experiments support the partition-invariance corollary: the AFCL numbers in Table 1 are identical across all alpha and T settings, exactly as the theory predicts. At the same time, the order-invariance claim is overstated: Appendix D only analyzes row permutations with a fixed label-column ordering, whereas the actual algorithm constructs one-hot columns incrementally by registration order. Thus the advertised 'identical and invariant' statement needs to be weakened to invariance up to a class-column permutation, or the proof must model column permutations. The acknowledged dependence on a well-pretrained backbone (Section 5) is a real scope limitation but not a mathematical flaw.","major_comments":[{"comment":"The proof of Theorem 2 models spatio-temporal heterogeneity and client reordering as a row permutation P of the pooled matrices F and Y (Eq. 45). However, in the actual algorithm the one-hot columns are created in first-appearance order by Eq. (2), so changing which client or task first introduces a class also permutes the columns of Y_{1:K} and hence the columns of W. As a literal numerical statement, 'the final global model ... is independent of spatio-temporal data heterogeneity and client registration order' is false; what holds is that the predictive function is unchanged up to the induced permutation of output classes. Please restate Theorem 2 accordingly and update the proof, or prove the stronger statement by explicitly tracking the column permutation. This is load-bearing because the abstract and Section 3.5 advertise exact invariance.","section":"Theorem 2 / Appendix D / Eqs. (2), (38), (45)"},{"comment":"The paper states in Section 5 that AFCL relies on a well-pretrained frozen backbone. Since Theorem 1 only equates AFCL with the centralized ridge solution on the fixed features, a poor backbone makes the invariance property vacuous. The experiments use a single pretrained ResNet-18, so the empirical claim that AFCL is 'consistently superior' is not tested across backbone quality. I recommend adding ablations with weaker or differently pretrained backbones, or at least a quantitative statement of the dependence, to delineate the scope of the claimed advantage.","section":"Section 5 / experimental scope"},{"comment":"All AFCL rows in Table 1 are exactly constant across settings, which is strong evidence for the theoretical invariance. However, no standard deviations or seed counts are reported for the baselines, some of which are stochastic (e.g., FedCBC's generative replay, FedMGP's prompts). The margins over the second-best baseline would be more convincing with at least three seeds and mean +/- std; please also state whether the AFCL entries are single deterministic runs or exact values. This affects the strength of the empirical 'consistent superiority' claim but not the core derivations.","section":"Section 4.2 / Table 1"}],"minor_comments":[{"comment":"The sentence 'the final global model W_K obtained by our AFCL is equivalent to empirical risk minimization in (1) with the full datasets D_{1:k} from all K clients' should read D_{1:K}, since the context is the final model over all K clients.","section":"Section 3.4"},{"comment":"The labels 'Wights for the Known' and 'Wights for the Unknown' contain a typo and should read 'Weights for the Known' and 'Weights for the Unknown'.","section":"Figure 2"},{"comment":"The word 'addtion' should be 'addition' in the sentence describing temporal heterogeneity.","section":"Appendix A"},{"comment":"Algorithm 1 line 2 and Section 3.1 refer to Tanh(..., Theta_T), but no Tanh mapping is defined or used in the method description; remove it or introduce it explicitly.","section":"Algorithm 1 / Section 3.1"},{"comment":"The notation 'W in R^{l_e in (d_k-d_{k-1})}' is a typo; it should be 'check W_k in R^{l_e times (d_k-d_{k-1})}'.","section":"Appendix E"},{"comment":"The 'Improve' column should be defined in the caption; the text reports an average margin of 93.38%, which appears to be the mean of the relative improvements in the table, but the calculation is not stated.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The order-invariance gap is the main obstacle; it is fixable by restating Theorem 2 in terms of permutation equivariance of the output columns. I do not see grounds for rejection, but I would not accept without that revision. The paper's relation to the authors' prior analytic learning work (ACIL and AFL) should be made more explicit in the final version; the current text cites those papers but does not clearly state what is new beyond re-applying the recursive Moore-Penrose machinery to FCL."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the central algebra (Theorem 1, Lemma 2) holds — I re-derived the induction and the closed-form equivalence, and it checks out. The real contribution is showing that block-wise recursive Moore-Penrose aggregation over clients exactly reproduces the pooled regularized least-squares linear head on frozen features, including when new classes appear. That is a useful theorem for the FCL subfield, and the server-side recursive update is cheap. The experiments are consistent with the claim: AFCL gets identical accuracy across all alpha/T settings, and the runtime gains are large. Credit where due: the authors state the frozen-backbone limitation plainly in Section 5, and they do not hide that the analytic machinery comes from their own prior ACIL/AFL work.\n\nWhere it gets soft: Theorem 2's proof treats client reordering only as a row permutation of the pooled data, with fixed column order of the one-hot labels. But under the known-unknown splitting, registration order determines which classes are 'known' vs 'unknown' at each step, and therefore determines the column order of Y1:K. Swap two clients and you change the column indices of every class in Y, hence in W. The proof never models this. So the literal statement that the global model is 'identical' under different registration orders is not proved and is not exactly true as a matrix equality. What is true, and follows from Theorem 1, is that the predictive function is invariant up to a permutation of output columns. That is still strong and useful, but the abstract and Theorem 2 advertise the stronger claim, and that gap is load-bearing for the wording.\n\nOther soft spots are minor: no error bars or seeds in the main tables, no direct centralized-linear-probe baseline in Table 1, and the gamma used in Table 1 is not specified (Table 2 sweeps it, but the default is unclear). The 'root cause is gradients' narrative is an argument, not a theorem, so I would treat it as motivation. Also, the class-continual label structure in Eq. (38) assumes each new client's unknown classes are disjoint from all previous ones; the known/unknown splitting enforces this by construction, but it is still an assumption worth stating.\n\nWho this is for: people working on federated continual learning and on analytic/pseudoinverse learning. It deserves a serious referee. The fix for Theorem 2 is a restatement as invariance up to label-column permutation, and the experiments would benefit from seeds and a linear-probe baseline. I would send it out.","headline":"The recursive analytic aggregation is a genuine, useful result, but Theorem 2 as stated overclaims order invariance by ignoring label-column permutation.","tokens_in":25586,"tokens_out":2495,"would_cite":true,"duration_ms":26915,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves that a federated continual learning system can be built entirely without gradients, and that its global model is exactly the centralized joint-learning solution regardless of how data are split across clients or tasks.","keywords":["analytic learning","federated continual learning","catastrophic forgetting","spatio-temporal invariance","non-IID data","class-incremental learning","closed-form least squares","frozen pre-trained backbone"],"falsifier":"On a small synthetic dataset, implement the recursive aggregation of equations (8)-(10) and compare the result with direct computation of equation (14); any nonzero difference beyond floating-point tolerance, or any change in the final $W_K$ when the clients are processed in a different order, would refute Theorems 1 and 2. Equivalently, with a deliberately weak frozen backbone, AFCL's accuracy should be predictable from linear-probe accuracy on those frozen features; if AFCL systematically exceeds that ceiling, the claim that the classifier is a pure least-squares fit would be wrong.","tokens_in":1663,"feed_emoji":"🧮","tokens_out":1811,"duration_ms":71310,"temperature":0.7,"pith_summary":"The paper tries to establish that catastrophic forgetting in federated continual learning is caused by gradient updates on non-IID data, and that a fully gradient-free design removes the problem rather than patching it. It proposes AFCL, in which clients freeze a pre-trained backbone, fit a linear classifier by closed-form least squares, and upload only Gram-matrix and weight statistics. The server recursively aggregates these statistics into a Global Knowledge Matrix; the paper proves the resulting global model equals the model that centralized joint learning would produce on all data from all clients. The claimed consequence is spatio-temporal invariance: the final model does not depend on how heterogeneous the data are across clients or tasks, nor on client order. A sympathetic reader would care because it turns the hardest FCL failure mode into an algebraic identity, with single-epoch training and single-round aggregation as side benefits.","feed_headline":"Gradient-free federated learning matches joint training exactly","feed_subtitle":"A recursive closed-form update makes the global model invariant to non-IID data splits and client order.","key_machinery":"The load-bearing object is the Global Knowledge Matrix $G_k$, updated by $G_k = [A_k G_{k-1} + B_k \\hat{W}_k,\\; B_k \\check{W}_k]$ with $A_k$ and $B_k$ defined through cumulative regularized Gram matrices $\\tilde{R}_k = \\sum_{i=1}^k R_i$, where $R_i = F_i^\\top F_i + \\gamma I$. The proof uses the Woodbury matrix identity to show by induction that $G_k = (\\tilde{R}_k)^{-1} F_{1:k}^\\top Y_{1:k}$; then the server's reconstruction $W_k = [\\tilde{R}_k - (k-1)\\gamma I]^{-1} \\tilde{R}_k G_k$ collapses algebraically to the ridge-regression solution over all data seen so far. The known-unknown class splitting, implemented through dynamic one-hot encoding functions, keeps the block label matrix non-overlapping so the same recursion supports class-continual growth.","core_discovery":"The paper claims that spatio-temporal catastrophic forgetting in federated continual learning can be eliminated, not mitigated, by removing gradients from training. On the client side, each virtual client forward-passes its data through a frozen pre-trained backbone, encodes labels with known-unknown one-hot splitting, and solves two ridge regressions in closed form, yielding local weights for previously known and newly introduced classes plus a regularized Gram matrix. On the server side, a recursively updated Global Knowledge Matrix absorbs each client's local statistics, and the final global classifier $W_k$ computed from it is exactly $(F_{1:k}^\\top F_{1:k} + \\gamma I)^{-1} F_{1:k}^\\top Y_{1:k}$, the centralized joint-learning solution over all data seen so far. Because the objective is invariant to row permutation of the data, the global model is independent of how samples are split across clients and tasks, and independent of client registration order.","pith_inferences":["The same recursive aggregation should transfer to any ridge-regression head on frozen features, including regression, multi-label classification, or kernelized features, because the proof uses only the block structure of the label matrix and the Woodbury identity.","A testable prediction is that AFCL's accuracy ceiling equals the linear-probe accuracy of the frozen features; pairing the recursion with random projections or kernels could raise that ceiling while preserving the invariance result.","The privacy claim deserves closer scrutiny: the uploaded regularized Gram matrix and local weight matrices are sufficient statistics of local data, and in low-sample or low-dimensional regimes they may reveal more than the statement that the uploads 'cannot be used to infer' local data.","The asynchronous virtual-client treatment means one server update handles both new data from existing clients and newly joined clients; a natural extension is to compare this exact aggregation with federated averaging when clients can share only a subset of their statistics."],"forward_implications":["Average accuracy should stay constant as spatial heterogeneity intensifies, such as smaller Dirichlet $\\alpha$, because the model is the same function of the pooled data; the experiments show AFCL's accuracy staying flat while baselines collapse.","Increasing the number of temporal tasks should not degrade the final model, since the pooled-data objective is unchanged; the results show no accuracy drop when the task count rises from 5 to 10.","Client registration order cannot affect the final weights; permuting the clients yields the same $W_K$ up to floating-point roundoff.","Each client needs only one forward pass and one upload per round, and the server only one aggregation, so communication and computation scale without multi-round synchronization or gradient backpropagation.","New classes can appear in any round through the known-unknown split without retraining old clients or storing raw samples, because the recursion consumes only Gram matrices and local closed-form weights."],"supporting_citations":[{"why":"Supplies the block-wise recursive Moore-Penrose inverse that the server's recursive aggregation builds on.","marker":"[39]"},{"why":"Supplies analytic class-incremental learning with absolute memorization, the base closed-form technique adapted to the federated setting.","marker":"[30]"},{"why":"Supplies a single-round analytic federated learning method with pre-trained models, the direct precursor for the server-side aggregation.","marker":"[40]"},{"why":"Supplies the MSE-based analytic learning framework for few-shot class-incremental tasks, supporting the use of MSE with pre-trained features.","marker":"[20]"},{"why":"Supplies evidence that frozen pre-trained features plus a lightweight head handle continual learning, motivating the frozen backbone choice.","marker":"[25]"},{"why":"Supplies the Si-Blurry protocol used to partition datasets into temporally heterogeneous tasks.","marker":"[52]"},{"why":"Supplies the Dirichlet distribution procedure used to create spatially non-IID client splits.","marker":"[53]"},{"why":"Supplies the spatio-temporal catastrophic forgetting baseline and problem statement that AFCL is benchmarked against.","marker":"[18]"}],"fun_headline_variants":["Gradient-free FL matches joint training exactly","AFCL: closed-form FL eliminates spatio-temporal forgetting","Non-IID splits? AFCL equals centralized joint learning","Analytic federated learning: exact invariance, no gradients"],"cache_read_input_tokens":27776,"weakest_assumption_plain":"The load-bearing premise is that the frozen pre-trained backbone already extracts features good enough for every task in the stream; the proof of exact equivalence is algebraic, but if the backbone is not well pre-trained for the task distribution, the exact solution is uniformly weak and spatio-temporal invariance becomes vacuous.","fun_headline_variants_meta":{"raw":{"variants":["Gradient-free FL matches joint training exactly","AFCL: closed-form FL eliminates spatio-temporal forgetting","Non-IID splits? AFCL equals centralized joint learning","Analytic federated learning: exact invariance, no gradients"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1475,"prompt_tokens":975,"completion_tokens":500,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":436}},"tokens_in":591,"tokens_out":500,"duration_ms":5559,"temperature":1.0,"reasoning_tokens":436,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:39:17.479021+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small synthetic dataset, implement the recursive aggregation of equations (8)-(10) and compare the result with direct computation of equation (14); any nonzero difference beyond floating-point tolerance, or any change in the final $W_K$ when the clients are processed in a different order, would refute Theorems 1 and 2. Equivalently, with a deliberately weak frozen backbone, AFCL's accuracy should be predictable from linear-probe accuracy on those frozen features; if AFCL systematically exceeds that ceiling, the claim that the classifier is a pure least-squares fit would be wrong.","supporting_citations":[{"cited_title":"AFL: A single-round analytic approach for federated learing with pre-trained models","cited_arxiv_id":null,"evidence_quote":"Supplies a single-round analytic federated learning method with pre-trained models, the direct precursor for the server-side aggregation."},{"cited_title":"GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task","cited_arxiv_id":null,"evidence_quote":"Supplies the MSE-based analytic learning framework for few-shot class-incremental tasks, supporting the use of MSE with pre-trained features."},{"cited_title":"Ranpac: Random projections and pre-trained models for continual learning","cited_arxiv_id":null,"evidence_quote":"Supplies evidence that frozen pre-trained features plus a lightweight head handle continual learning, motivating the frozen backbone choice."},{"cited_title":"Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning","cited_arxiv_id":null,"evidence_quote":"Supplies the Si-Blurry protocol used to partition datasets into temporally heterogeneous tasks."},{"cited_title":"Ensemble distillation for robust model fusion in federated learning","cited_arxiv_id":null,"evidence_quote":"Supplies the Dirichlet distribution procedure used to create spatially non-IID client splits."},{"cited_title":"Overcoming spatial-temporal catastrophic forgetting for federated class-incremental learning","cited_arxiv_id":null,"evidence_quote":"Supplies the spatio-temporal catastrophic forgetting baseline and problem statement that AFCL is benchmarked against."}],"review_version":1}