{"id":"0d25dae0-5046-4970-91e8-6cf3fc1f751b","arxiv_id":"2506.21054","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FedDAA dynamically clusters clients and distinguishes real drift from virtual and label drift, retaining historical data when decision boundaries are unchanged.","lead":"FedDAA is a federated learning method that detects whether each client's data has changed in a way that should make it forget old data or keep it. The authors report accuracy gains of about 8 percent over strong baselines on three image benchmarks, plus a theoretical convergence analysis.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Eq. (4) real-drift criterion compares old and new prototypes to the same current cluster centers, so pure input-distribution (virtual) drift can flip a client's cluster assignment and be mislabeled as real drift; the experiments never isolate this case.","rationale":"The reader's weakest assumption is exactly the same load-bearing point: the paper treats the cluster-assignment change in Eq. (4) as equivalent to real drift without proving that virtual or label drift cannot produce the same change. My analysis of Definition 2 and Eq. (4) confirms that P_t^k is a class-conditional mean of model outputs, so it is sensitive to P(x|y) and to the model checkpoint; the assignment of both old and new prototypes to the same current cluster centers makes a pure input shift capable of flipping the assignment. The isolated pure-virtual-drift experiment I propose would settle whether this failure actually occurs in the paper's own data-generation protocol. Because this is the same concern the reader already identified, the CONDITIONAL verdict remains appropriate: if the proposed test shows a high false-positive rate, the paper's central contribution would need to be rejected or substantially reframed; if the false-positive rate is negligible, the current verdict is supported. No adjustment to the reader's verdict is warranted before running that check.","tokens_in":22326,"tokens_out":5220,"duration_ms":63531,"concrete_test":"Construct a pure-virtual-drift run using the paper's CIFAR-10 setup (Appendix C.1): keep all client labels and label permutations fixed across time steps (no y -> R-1-y or y+1 mod R and no Dirichlet resampling), keep the number of decision boundaries fixed, and at time step 2 rotate every client's images by 120 degrees. Run Algorithms 1 and 2 exactly as specified and compute the fraction of clients for which Eq. (4) reports c^{t-1}_k != c^t_k. Repeat for 240 degrees. If this fraction is materially above zero (e.g., >5%), Eq. (4) does not separate virtual from real drift and the central contribution is unsupported. To control for the unspecified checkpoint in Eq. (2), compute both P^{t-1}_k and P^t_k with the same model checkpoint h(w^t, .) as used in Algorithm 2, and report which checkpoint is used.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that FedDAA distinguishes real drift from virtual/label drift via Eq. (4): a client is classified as having real drift iff the cluster assignment of its current prototype differs from that of its historical prototype. This equivalence is load-bearing because the DAA module discards historical data precisely when Eq. (4) fires. The equivalence is not justified and is fragile under the paper's own prototype definition. In Eq. (2), P_t^k is a matrix of class-conditional means of model outputs h(w^t,x). Thus P_t^k depends on P(x|y) through the inputs x and on the current model checkpoint w^t, not only on the conditional P(y|x). If inputs are rotated (the paper's virtual-drift generator), the model outputs for those inputs change; both P^{t-1}_k and P^t_k are then assigned against the same current centers P^t_c (Eq. 4), so a pure P(x) shift can move P^{t-1}_k and P^t_k into different clusters even though every client's decision boundary P(y|x) is unchanged. The DAA module would then drop historical data, which is exactly the wrong adaptation for virtual drift. This is not merely hypothetical: the experimental protocol in Appendix C.1 always combines label-permutation real drift with 120/240-degree rotation and Dirichlet label shifts, so a false positive on the virtual component is invisible in the reported averages. The claim that FedDAA distinguishes real drift from virtual/label drift therefore rests on an unproved invariance of Eq. (4) to marginal shifts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedDAA, a clustered federated learning framework for multi-source concept drift. It introduces three modules: a Number of Clusters Determination (NCD) module based on clustering client data prototypes, a Real Drift Local Detection (RDLD) module that flags real drift when a client's historical and current prototypes are assigned to different prototype cluster centers (Eq. (4)), and a Distribution-Aware Adaptation (DAA) module that discards historical data for real-drifted clients but retains it for virtual/label-drifted clients. The authors provide a convergence analysis (Theorem 1) and report average accuracy gains of 7.84%–8.52% over FedDrift on Fashion-MNIST, CIFAR-10, and CIFAR-100, with an ablation study and comparisons to an Oracle.","tokens_in":22672,"tokens_out":3496,"duration_ms":39940,"significance":"The problem addressed is relevant: distinguishing real drift from virtual and label drift in federated learning would enable principled selective retention of historical data. The paper's strengths include a clearly documented experimental protocol, ablations that isolate the contributions of the three modules, comparisons against several baselines and an Oracle, and detailed pseudocode. If the drift-detection criterion in Eq. (4) were formally justified and the convergence theorem were valid, the work would be a solid contribution to clustered FL under non-stationary distributions. However, the central detection equivalence is not established, and the convergence analysis relies on an assumption that effectively assumes the main algorithmic claim; these issues are load-bearing and prevent acceptance in the current form.","major_comments":[{"comment":"The real-drift detection criterion c^{t-1}_k != c^t_k is not justified. The prototype P^t_k in Eq. (2) is a matrix of class-conditional means of model outputs h(w^t, x), so it depends on the current model checkpoint w^t and on the input distribution through x. A pure virtual drift, such as rotating inputs, changes h(w^t, x) even if P(y|x) is fixed; both P^{t-1}_k and P^t_k are then assigned to the nearest centers among the same current cluster centers P^t_c, so a marginal shift in P(x) can move the historical and current prototypes into different clusters. The DAA module would then discard historical data, which is exactly the wrong adaptation for virtual drift. The paper does not prove that the nearest-center assignment is invariant to virtual drift or label drift, nor does it bound the probability of such a false positive.","section":"§4.3, Eq. (4)"},{"comment":"The experimental protocol never isolates the false-positive scenario described above. At every time step, label-permutation real drift is applied simultaneously with image rotations (120 or 240 degrees) and Dirichlet label shifts, so a client whose prototype assignment flips because of rotation alone is indistinguishable from a client that truly changed P(y|x). The reported averages in Table 1 and the forgetting-mitigation results in Table 2 therefore cannot validate the claim that FedDAA distinguishes real drift from virtual/label drift. An experiment with pure virtual drift (e.g., rotations with no label permutation) is needed to test whether Eq. (4) keeps such clients in K_clean and whether historical data are retained.","section":"Appendix C.1"},{"comment":"The convergence analysis is not established. Assumption 5 only states E[C] = C*, but the proof uses the much stronger equality E_C[L(w^0, \\hat{\\alpha}^0, C)] = L(w^0, \\hat{\\alpha}^0, C*) (Appendix B.3, Eq. (35); see also Lemma 2 in Appendix B.2). This equality does not follow from E[C] = C*; it requires L(w^0, \\hat{\\alpha}^0, ·) to be linear or some additional distributional assumption on C. Moreover, the NCD module selects C by silhouette scores, and no argument is given that the selected cluster count is unbiased for an optimal C*, so Assumption 5 effectively assumes the correctness of the module that the theorem is meant to validate. Finally, Theorem 1 bounds a gap between objective values at two different cluster counts; the right-hand side contains a negative term -E C* Δ^2_ξ / (4η) plus positive terms, so it is not a standard convergence bound even if the preceding steps were valid.","section":"§5, Assumption 5 and Appendix B.3"}],"minor_comments":[{"comment":"The heading 'Approximated objective function for piratical implementation' contains a typo; 'piratical' should be 'practical'.","section":"Appendix B.1 heading"},{"comment":"The word 'numebr' should be 'number' in the sentence 'for a specific cluster numebr C'.","section":"Appendix B.2"},{"comment":"In the FedDrift column, the entries for the first two rows appear as '36.72 ± 0.4165.61 ± 0.2969.20 ± 0.66' and similar, with missing separators between values; the table formatting needs correction.","section":"Table 2"},{"comment":"The algorithm title 'FL Distribution-Awared Adaptation' contains a typo; 'Awared' should be 'Aware'.","section":"Algorithm 3 title"},{"comment":"The description of the RDLD module refers to the 'the the prototype cluster centers' in the first sentence; the duplicated article should be removed.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the empirical framework is well documented, but the central drift-detection claim and the convergence theorem both need substantial work. In particular, Eq. (4) requires either a formal invariance argument or an experimental setting that isolates virtual drift, and Assumption 5 needs to be replaced by a condition that is actually used in the proof. These are load-bearing issues rather than presentation problems, so I recommend major revision rather than rejection, since the core idea is defensible if the detection mechanism can be validated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid engineering paper with a real gap it addresses. Most FL drift work treats everything as real drift and discards history; FedDAA tries to distinguish real drift from virtual/label drift and retain old data when the decision boundary hasn't changed. The empirical gains over FedDrift (7.84–8.52% across three benchmarks) are consistent with that story. The paper is honest about its own limits: the conclusion admits the rehearsal strategy is simple, and the footnote on the fourth drift source is a thoughtful touch.\n\nWhat's new: dynamic cluster-count selection via silhouette on class-conditional prototypes, plus a drift-source-dependent adaptation rule. The ablation study isolates each module and does real work; the NCD module is the biggest contributor, which makes sense. The empirical claim itself is not circular — accuracy is measured on held-out test data against external baselines. Table 2 (forgetting mitigation) is a nice extra; it shows retention behavior directly.\n\nWhere it's soft, in order:\n\n1. The Eq. (4) detection rule is the load-bearing piece and it is not justified. The prototype P^t_k is a matrix of mean model outputs h(w^t, x) conditioned on label. That depends on P(x|y) through the input distribution and on the current weights. A pure input rotation can shift both P^{t-1}_k and P^t_k relative to the same current cluster centers, and nothing prevents a virtual-drift-only client from flipping its cluster assignment. The stress-test note is right: because the protocol always applies rotation together with label permutation, a false positive on the virtual component would be hidden in the average. The fix is straightforward: add a virtual-drift-only condition (rotation without label change) and report detection precision/recall per drift type.\n\n2. The convergence theorem is weak in a specific way. Assumption 5 assumes E[C] = C*, i.e., the algorithm's cluster count is unbiased for the true count. That makes the theorem a consistency result conditional on the cluster-number module being right, not an end-to-end guarantee. The authors should either drop the pretense of a rate and frame it as a bound under that assumption, or analyze the NCD module. As written, it's not a fatal flaw, but a careful reviewer will flag it.\n\n3. Missing: code and a rehearsal-matched baseline. FedDrift discards history; the comparison would be more persuasive with a variant of FedDrift that keeps old data under the same memory budget.\n\nBottom line: the central idea is plausible, the experiments are documented, and the limitations are acknowledged. This deserves a serious referee, and the authors can address the main concern with one additional experiment. I'd take it to reading group.","headline":"FedDAA is a genuinely useful combination — dynamic cluster counting plus drift-source-aware retention — but the real-drift detector in Eq. (4) rests on an invariance that the paper neither proves nor isolates in its experiments.","tokens_in":23150,"tokens_out":1926,"would_cite":true,"duration_ms":23058,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedDAA distinguishes real drift from virtual/label drift in federated learning and selectively keeps historical data, reporting 7.84–8.52 percentage point accuracy gains over FedDrift.","keywords":["federated learning","concept drift","real drift detection","virtual drift","label drift","clustered federated learning","data prototype","catastrophic forgetting"],"falsifier":"Take a two-cluster task with fixed labels and fixed $P(y|x)$, then rotate the input images in small steps (0, 5, 10, ..., 30 degrees) across time steps; if any rotation causes the client's prototype $P^t_k$ to change its nearest cluster center in Eq. (4), FedDAA will flag real drift and discard historical data, contradicting the claimed separation of virtual drift from real drift.","tokens_in":22114,"feed_emoji":"📈","tokens_out":8626,"duration_ms":79407,"temperature":0.7,"pith_summary":"The paper claims that a federated learning system can adapt to concept drift better by first identifying which of three drift sources is occurring: real drift (a shift in $P(y|x)$), virtual drift (a shift in $P(x)$), or label drift (a shift in $P(y)$). It proposes FedDAA, which clusters clients by data prototypes and declares real drift exactly when a client's historical and current prototypes fall into different clusters. Clients flagged with real drift retrain on current data only, while clients without real drift train on both historical and current data, so useful old knowledge is not thrown away. The reported result is 7.84% to 8.52% higher average accuracy than the strongest baseline on Fashion-MNIST, CIFAR-10, and CIFAR-100, with a convergence guarantee under a dynamically changing number of clusters.","feed_headline":"Clustered FL gains 8.5% by telling real drift from input shifts","feed_subtitle":"When only input or label distributions shift, it keeps old data instead of forgetting it.","key_machinery":"The load-bearing object is the data prototype $P^t_k \\in \\mathbb{R}^{R\\times R}$ (Definition 2), whose $r$-th column is the mean model output $h(w^t,x)$ over samples of class $r$. It compresses a client's conditional distribution $P(y|x)$ into a matrix that can be clustered. The mechanism that carries the argument is the cluster-assignment comparison of Eq. (4): previous and current prototypes are each mapped to the nearest center among the current prototype cluster centers $P^t_c$, and a change from $c^{t-1}_k$ to $c^t_k$ is the sole signal of real drift. That single test determines whether historical data are retained or discarded, so the adaptation strategy and the convergence analysis both rest on the prototype tracking conditional-distribution changes without being displaced by marginal shifts.","core_discovery":"The central discovery is that the drift source, not merely the fact of drift, should drive the adaptation strategy. FedDAA encodes each client's conditional distribution at time $t$ in a class-conditioned prototype matrix $P^t_k$ of averaged model outputs, clusters all clients' prototypes on the server via K-means with the cluster count chosen by silhouette scores, and labels client $k$ as having real drift if the nearest prototype-cluster center changes between $t-1$ and $t$, that is, $c^{t-1}_k \\neq c^t_k$ in Eq. (4). Real-drifted clients update on current data only, whereas everyone else keeps using historical data as well, which the authors argue prevents catastrophic forgetting under virtual and label drift. The paper proves a convergence bound (Theorem 1) for the federated objective under smoothness, bounded-gradient, and bounded-dissimilarity assumptions, and the small gap to an oracle in experiments (roughly one point) is presented as evidence that the drift-source detection captures most of the benefit of full knowledge.","pith_inferences":["Editorial inference: switching the detection from a single previous time step to a short prototype history could make real-drift alarms less sensitive to transient fluctuations in the model output.","Editorial inference: a pure virtual-drift benchmark with rotations but no label remapping would isolate whether Eq. (4) truly avoids false real-drift alarms, since the paper's setup mixes rotation with changing label mappings.","Editorial inference: the Euclidean distance from a prototype to its assigned center could be used as a drift-magnitude measure, turning the binary keep-or-discard decision into a graded memory budget.","Editorial inference: because the prototype only needs per-class mean model outputs, the same detection rule should transfer to non-image data such as time series or text, which the paper does not test."],"forward_implications":["In deployments where clients experience virtual drift such as image rotation or label drift such as shifting class frequencies, FedDAA retains historical data and should avoid the catastrophic forgetting that single-model baselines exhibit.","Because the cluster count is re-estimated at every time step, the system can move from two to four decision boundaries without server-side reconfiguration of the number of clusters.","The accuracy gains over FedDrift (8.52% on Fashion-MNIST, 7.84% on CIFAR-10, 8.31% on CIFAR-100) imply that distinguishing real drift from virtual/label drift, rather than clustering alone, is what drives most of the improvement.","The sampling-rate table shows that dropping from 100% to 20% client participation costs at most about two points of accuracy, so communication cost can be cut substantially in practice.","The convergence bound in Theorem 1 makes the choice of communication rounds and learning rate principled: the gap to the optimal objective is bounded by a constant that shrinks with the expected number of clusters."],"supporting_citations":[{"why":"Supplies the three-source taxonomy of real, virtual, and label drift that FedDAA's detection module is designed to distinguish.","marker":"[1]"},{"why":"Provides the clustered-FL objective and the optimization steps (Eqs. 14-16) on which FedDAA's adaptation and convergence analysis build.","marker":"[4]"},{"why":"Contributes the mixture-of-distributions federated surrogate framework used in the proof of Theorem 1.","marker":"[10]"},{"why":"Is the strongest baseline (FedDrift) whose average accuracy FedDAA claims to beat by 7.84 to 8.52 points.","marker":"[14]"},{"why":"Is the adaptive single-model baseline A-FedAvg used in the comparison tables.","marker":"[21]"},{"why":"Is the single-model concept-drift baseline Flash used in the comparison tables.","marker":"[22]"},{"why":"Is the Fashion-MNIST dataset on which the main accuracy comparison is run.","marker":"[28]"},{"why":"Is the CIFAR-10 dataset used for the main comparison and the per-task forgetting-mitigation table.","marker":"[29]"},{"why":"Is the ResNet-18 architecture used as the model for CIFAR-10 in the experiments.","marker":"[30]"},{"why":"Is the MobileNet architecture used as the model for CIFAR-100 in the experiments.","marker":"[31]"}],"fun_headline_variants":["Real drift? Forget it. Virtual? Keep it. FL +8.5%","Telling drift sources apart boosts FL accuracy by 8.5%","FedDAA clusters clients to pick what to forget in FL","Separating real from virtual drift: 8.5% FL gain","Drift type matters: FL clusters to retain useful history"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that virtual drift and label drift never move a client's data prototype across a prototype-cluster boundary, so a change in cluster assignment can be taken as proof of real drift.","fun_headline_variants_meta":{"raw":{"variants":["Real drift? Forget it. Virtual? Keep it. FL +8.5%","Telling drift sources apart boosts FL accuracy by 8.5%","FedDAA clusters clients to pick what to forget in FL","Separating real from virtual drift: 8.5% FL gain","Drift type matters: FL clusters to retain useful history"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0011,"raw_usage":{"total_tokens":4641,"prompt_tokens":1048,"completion_tokens":3593,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":664,"completion_tokens_details":{"reasoning_tokens":3499}},"tokens_in":664,"tokens_out":3593,"duration_ms":26962,"temperature":1.0,"reasoning_tokens":3499,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:35:19.164051+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a two-cluster task with fixed labels and fixed $P(y|x)$, then rotate the input images in small steps (0, 5, 10, ..., 30 degrees) across time steps; if any rotation causes the client's prototype $P^t_k$ to change its nearest cluster center in Eq. (4), FedDAA will flag real drift and discard historical data, contradicting the claimed separation of virtual drift from real drift.","supporting_citations":[{"cited_title":"Learning under concept drift: A review.IEEE Trans","cited_arxiv_id":null,"evidence_quote":"Supplies the three-source taxonomy of real, virtual, and label drift that FedDAA's detection module is designed to distinguish."},{"cited_title":"Fedrc: Tackling diverse distribution shifts challenge in federated learning by robust clustering","cited_arxiv_id":null,"evidence_quote":"Provides the clustered-FL objective and the optimization steps (Eqs. 14-16) on which FedDAA's adaptation and convergence analysis build."},{"cited_title":"Federated multi-task learning under a mixture of distributions","cited_arxiv_id":null,"evidence_quote":"Contributes the mixture-of-distributions federated surrogate framework used in the proof of Theorem 1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the strongest baseline (FedDrift) whose average accuracy FedDAA claims to beat by 7.84 to 8.52 points."},{"cited_title":"Adaptive federated learning in presence of concept drift","cited_arxiv_id":null,"evidence_quote":"Is the adaptive single-model baseline A-FedAvg used in the comparison tables."},{"cited_title":"Flash: Concept drift adaptation in federated learning","cited_arxiv_id":null,"evidence_quote":"Is the single-model concept-drift baseline Flash used in the comparison tables."},{"cited_title":"Deep residual learning for im- age recognition","cited_arxiv_id":null,"evidence_quote":"Is the ResNet-18 architecture used as the model for CIFAR-10 in the experiments."}],"review_version":1}