{"id":"bed784fa-a9b4-4ad7-afe3-a9444a06bb2e","arxiv_id":"2412.13466","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Oversampling the skewed class before recovery training restores federated unlearning model accuracy better than four baselines.","lead":"This paper proposes a recovery method for federated learning models after a client with heavily skewed label data leaves, by generating extra examples of the rare class on remaining clients, cleaning them, and retraining. It reports accuracy gains on the rare class across MNIST, FMNIST, and USPS under three skew levels.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Method requires every remaining client to hold multiple real skewed-class samples; the paper never states or tests this, so the recovery claim does not cover the common case where the leaving client is the sole owner of a class.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the method assumes each remaining client has at least one real skewed-class sample. I agree with that reading and sharpen it: because SMOTE interpolates between a sample and its same-class neighbor, the actual precondition is at least two samples per remaining client, not merely one. This is a necessary condition for Eq. 4 to be defined and for the autoencoder to learn a useful class-specific latent space. The paper's Definition 1 does not explicitly exclude ni_C = 0, and the real-world motivating scenario (a dominant client leaves, others have very little of the skewed class) includes the zero-sample case. The experimental allocations guarantee each remaining client has a substantial number of skewed-class samples, so the tested regime never challenges this boundary. The central claim therefore holds only under an unstated and untested data-availability assumption. Secondary concerns noted by the reader, such as tuning k on the test set and the absence of released code, are valid but do not change my assessment; they reinforce the need for a scoped claim. The verdict should remain CONDITIONAL, since the concern does not invalidate the reported results for the tested regime but requires the paper to explicitly state and evaluate the minimum-sample precondition before the general recovery claim can be accepted.","tokens_in":15243,"tokens_out":15424,"duration_ms":145245,"concrete_test":"Run the full Imba-ULRc pipeline on the paper's MNIST setup with alpha = 0.9, but modify the allocation so that one remaining client has exactly 0 samples of the skewed class and another has exactly 1 sample, while keeping all other data and parameters unchanged. Observe whether SMOTE in Eq. 4 can find a same-class neighbor, whether the autoencoder can train on the skewed class, and whether the pipeline completes or requires an ad hoc fallback. If the pipeline fails or produces no valid synthetic samples, the paper must explicitly restrict its claim to remaining clients with at least two (or k+1) skewed-class samples and state the minimum ni_C assumption; absent that, the general recovery claim is not established for zero-sample clients.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that after unlearning a client with a skewed class, the remaining clients can recover the unlearning model by generating synthetic skewed-class data via an autoencoder plus SMOTE. For this to work, every remaining client must have enough real skewed-class samples to seed the SMOTE interpolation in Eq. 4 and to train a usable latent manifold for that class. At minimum, at least two same-class samples are needed for SMOTE to interpolate, and practically more are needed for a stable autoencoder. The paper never states this precondition. Definition 1 writes nu_C >> ni_J >> ni_C without any lower bound on ni_C, and the motivating example in Section 1 (a large bank leaves while small banks have much less high-net-worth data) naturally includes clients with zero samples of the skewed class. No fallback is provided for zero-seed clients, so the oversampling step is undefined there. In the experiments, however, data allocation gives each remaining client (1-alpha)*N_C/4 samples, which for the tested alpha values is between roughly 75 and 300 samples; the boundary ni_C = 0 or ni_C = 1 is never exercised. The paper's general claim, stated in the abstract and introduction, that remaining clients can 'supplement the skewed class data' is therefore unsupported for the realistic zero-sample case, and the method's scope is narrower than the title and problem statement suggest.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses federated unlearning when the leaving client holds most of the data for one class (the skewed class). After unlearning, the remaining clients are assumed to have too little skewed-class data to recover the model. The proposed method, Imba-ULRc, has each remaining client train a local autoencoder, oversample the skewed class in the latent space with SMOTE (Eq. 4), denoise the generated samples using a density-factor threshold based on k-nearest neighbors (Section 3.3), and then perform recovery training on the unlearning model. The experimental section compares Imba-ULRc with four baselines on MNIST, FMNIST, and USPS under three skew levels alpha = 0.8, 0.85, 0.9. Table 1 reports that Imba-ULRc achieves the best skewed-class accuracy in all nine configurations and the best or near-best global accuracy in seven of nine, with ablations in Table 2 and a sensitivity analysis for k in Figure 5.","tokens_in":15552,"tokens_out":4615,"duration_ms":40861,"significance":"If the method holds in the regime it claims, it addresses a real gap in federated unlearning: recovering model quality after a skewed-label client leaves, without accessing that client's data. The empirical comparison uses standard public benchmarks, multiple baselines, and ablation studies, and the method is conceptually simple. However, the paper currently omits a key precondition (each remaining client must have enough real skewed-class samples), and the denoising parameter k is selected on the test sets used for final reporting. These issues narrow the validity of the central claim and weaken the evidence. The contribution is still potentially useful for the regime where every remaining client has at least a moderate number of skewed-class samples, but the manuscript must be revised to state and test this boundary. The paper does not provide code or machine-checked proofs; its evidence is purely empirical with three runs per setting.","major_comments":[{"comment":"The method requires each remaining client to have at least one real sample of the skewed class to seed SMOTE interpolation in Eq. (4) and to train an autoencoder that can represent that class's manifold, but the paper never states this precondition. Definition 1 only states nu_C >> ni_J >> ni_C with no lower bound on ni_C, and the motivating example in Section 1 (small banks with much less high-net-worth data) naturally includes clients with zero such samples. The experimental protocol in Section 4.2 gives each remaining client (1-alpha)*N_C/4 samples, which is large for the tested alpha values, so the zero-sample case is never exercised. The abstract and introduction claim a general recovery method, but the method is undefined when ni_C = 0. Please state the required condition explicitly and either provide a fallback for zero-seed clients or scope the claims to the regime where every remaining client has at least a few skewed-class samples.","section":"Section 3.2, Definition 1 and Eq. (4)"},{"comment":"The denoising parameter k is selected by varying k and measuring the skewed-class accuracy after recovery, apparently on the same test sets used to report the final results in Table 1. This is a form of test-set tuning, and it can selectively favor the proposed method when comparing against baselines that do not have access to the test set for parameter selection. Use a held-out validation split to select k and then report test accuracy for the chosen k, or report results across a range of k values and show that the main conclusions are insensitive to the choice.","section":"Section 5.3, Figure 5 and Table 1"},{"comment":"The caption of Table 1 says 'Balanced Accuracy of the Global Model', but the text repeatedly refers to this column as 'accuracy of the global model' or 'global model accuracy'. The reported values, such as 95.82 for MNIST at alpha=0.8, look like standard overall accuracy rather than balanced accuracy (macro-average recall). Since the test set may not be class-balanced, the term 'balanced accuracy' has a specific meaning that should be defined and used consistently. Clarify which metric is reported and ensure it is computed identically for all methods, or the comparison across methods may be misleading.","section":"Table 1 and Section 5.1"},{"comment":"The density formulas have notation inconsistencies that prevent reproduction. Eq. (5) uses 'K + 1' in the denominator although the neighbor count is denoted by lowercase k elsewhere, and K is already used for the set of all classes in Definition 1. Eq. (6) has an inner sum over q' whose upper limit is written as 'Pk' in a way that is ambiguous. Since the denoising step is a central contribution, please rewrite these equations with clearly defined variables and consistent notation.","section":"Section 3.3, Eqs. (5)-(6)"},{"comment":"All comparisons are based on three runs with no statistical significance testing. The skewed-class accuracy gaps are large, but the global-accuracy differences are often small (e.g., MNIST alpha=0.8: Imba-ULRc 95.82 +/- 0.02 vs MOON 95.62 +/- 0.05; USPS alpha=0.8: Imba-ULRc 85.37 +/- 0.06 vs FLRS 85.50 +/- 0.09). With n=3, such differences may not be meaningful. Report individual run results or apply paired significance tests across runs and configurations to support the claim of consistent improvement.","section":"Section 5.1, Table 1"}],"minor_comments":[{"comment":"The reverse-order reconstruction loss is hard to follow because the notation mixes yp_i,c, y1_i, and dp_i,c without clear indexing. Explain the purpose of the reordering and define all indices explicitly.","section":"Section 3.2, Eq. (3)"},{"comment":"The text says 'five federated learning clients' and that Client 1 is unlearned, which leaves four remaining clients. State the number of remaining clients explicitly and report how many independent repetitions are used for each configuration.","section":"Section 4.2"},{"comment":"The line 'where N dn_i denote the total amount of noised dataset Di_dn' uses 'noised' where 'denoised' is meant. This is a typo that could confuse readers.","section":"Algorithm 1"},{"comment":"The distribution T is used in Eq. (1) without a definition. Clarify what T represents, e.g., the output distribution of the model.","section":"Section 2.1, Eq. (1)"},{"comment":"The x-axis and y-axis labels are not described in the text or figure; state explicitly that the x-axis is the value of k and the y-axis is the skewed-class test accuracy.","section":"Figure 5"},{"comment":"Reference [11] is cited for SMOTE, but [11] is the DeepSMOTE paper; provide a reference to the original SMOTE paper by Chawla et al. and keep [11] only if DeepSMOTE is specifically used.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the journal and the core idea is worth publishing if the scope issue is resolved. In my view, the missing precondition about positive skewed-class samples per remaining client is the most important technical gap because it makes the title and abstract claim broader than the method actually delivers. The test-set tuning of k is a methodology flaw that should be fixed without too much difficulty. I recommend major revision rather than rejection because the central empirical result on the tested configurations is plausible and the fixes are local, not a change of the entire approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a credible, modest empirical paper. Imba-ULRc beats the four baselines on skewed-class accuracy in all nine dataset/skew configurations, and global accuracy is best or near-best in most cases. On the tests actually run, the central claim holds.\n\nWhat is new: applying autoencoder-plus-SMOTE oversampling to federated unlearning recovery, then adding a LOF-style density factor to filter generated noise. The components are known, but the combination in this recovery setting is not present in the cited baselines. The ablation is genuinely useful: denoising helps in 16 of 18 cases, and performance stays stable across alpha values.\n\nThe soft spots, in order of importance. First, scope: the method silently assumes every remaining client has at least one, practically several, real samples of the skewed class, because SMOTE needs seeds and the autoencoder needs enough of that class to learn a usable latent space. Definition 1 only states nu_C >> ni_J >> ni_C, with no lower bound on ni_C. The experiments never exercise the zero-seed or one-seed case—with alpha in {0.8, 0.85, 0.9} and five clients, each remaining client gets roughly 75 to 300 skewed-class samples. The abstract's claim that remaining clients can supplement the skewed class is therefore only supported when they already hold a nontrivial share. The motivating bank example naturally includes clients with zero samples. This should be stated as a limitation or handled with a fallback. Second, k for denoising is selected as 5 after inspecting test accuracy in Figure 5; without a held-out split or principled selection rule, the reported numbers are mildly optimistic. Third, no code is released, the benchmark is small (three datasets, five clients, ten global rounds), and on USPS the global accuracy trails FLRS slightly in two of three skew levels. The global-model advantage is real but not universal.\n\nThese issues do not break the paper. The comparison is fair—all methods train on the same unlearning model with the same rounds—and the skewed-class gains are consistent and practically meaningful. The writing is honest; the conclusion even mentions computational overhead and incentive design.\n\nThis paper is for people working on federated unlearning recovery or data-generation approaches to label skew in FL. I would probably cite it as related work if I were writing in that area, and I would want to see it refereed with the scope issue addressed and code released. It deserves a serious referee.","headline":"A credible modest empirical result for federated unlearning recovery under label skew, but the method assumes every remaining client already has skewed-class seeds and the zero-sample case is untested.","tokens_in":16042,"tokens_out":3865,"would_cite":true,"duration_ms":32216,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"After a skewed-class client is unlearned, the remaining clients can synthesize and denoise that class's data to restore model accuracy without touching the leaving client's data.","keywords":["federated unlearning","label distribution skew","model recovery","SMOTE","autoencoder","density-based denoising","imbalanced federated learning"],"falsifier":"Run the recovery pipeline on a split where one remaining client has zero skewed-class samples: the SMOTE step for that client is undefined and the code would have no seed to interpolate from. A recovery result on such a split would require a fallback mechanism the paper does not specify; conversely, confirming that accuracy collapses only in that zero-sample case would localize the method's boundary.","tokens_in":15042,"feed_emoji":"🔄","tokens_out":5103,"duration_ms":44476,"temperature":0.7,"pith_summary":"The paper tackles a blind spot in federated unlearning: when a client whose data is dominated by one class leaves and is unlearned, the global model's accuracy on that class collapses, and the remaining clients have too few of that class's samples to retrain the bias away. It proposes a recovery method, Imba-ULRc, in which each remaining client trains a local autoencoder, oversamples the skewed class in the latent space with SMOTE, removes low-density synthetic outliers, and then uses the enhanced local datasets for federated recovery training. The central claim is that this restores the unlearning model's performance on the skewed class and overall without ever accessing the leaving client's data. If true, it gives federated systems a way to honor unlearning requests without accepting a permanently degraded model whenever label distributions are skewed.","feed_headline":"Oversampling and denoising rebuild unlearned models on skewed classes","feed_subtitle":"A local autoencoder plus SMOTE replaces the leaving client's data; density filtering keeps the synthetic samples clean.","key_machinery":"The load-bearing machinery is a local autoencoder whose training objective combines a standard reconstruction loss with a reverse-order reconstruction loss: the decoder is fed encoded features in the reverse of the order they were produced, so the model learns to map latent variations back to the original data space. SMOTE then interpolates new latent points between a skewed-class sample and a neighboring encoded sample, and the decoder turns them into synthetic data. To keep that data clean, each generated point receives a density score from the distances to its k nearest same-class neighbors, and its density factor is the ratio of the neighbors' average density to its own; generated points whose factor exceeds the median are discarded. The surviving data are used for local gradient-descent recovery training, and the server aggregates the local models with data-size weights.","core_discovery":"On the paper's own terms, the discovery is that the damage done by unlearning a skewed-class client is recoverable from the remaining clients' local data alone. By augmenting each remaining client's skewed-class samples in a learned low-dimensional feature space, then filtering the synthetic samples by a density factor computed against their local neighborhood, Imba-ULRc produces balanced, cleaner local datasets. Recovery training on these datasets drives the unlearning model back to high skewed-class accuracy: in the reported results it reaches 92.41 to 93.38 percent on MNIST, 89.07 to 89.77 percent on FMNIST, and 85.26 to 87.53 percent on USPS across skew levels alpha = 0.8, 0.85, and 0.9, outperforming the four baselines in most comparisons. The ablation shows the denoising step improves over SMOTE-only recovery in 16 of 18 cases.","pith_inferences":["Editorial inference: the method implicitly assumes every remaining client has at least one real skewed-class sample; if some client has none, SMOTE's interpolation has no seed and the encoder has never seen that class. A practical extension would let clients borrow latent seeds from peers or re-weight aggregation for zero-shot clients.","Editorial inference: the median density-factor threshold is a heuristic with a testable alternative; replacing it with a quantile or a per-client calibrated threshold may trade recall of genuine boundary samples against noise removal, which the paper does not explore.","Editorial inference: since the synthetic data are generated locally and never leave each client, the approach may also dampen privacy leakage from raw data sharing, but the paper does not quantify how much information the synthetic samples or the trained autoencoder reveal."],"forward_implications":["Federated unlearning no longer has to mean accepting a permanently biased model: after the leaving client's contribution is erased, the remaining clients can rebuild skewed-class accuracy through their own synthesized data.","The recovery works without access to the unlearning client's data or model, preserving the privacy boundary that motivates federated unlearning.","Because skewed-class accuracy stays roughly flat as skew alpha rises from 0.8 to 0.9, the method is claimed to be robust to how dominant the leaving client was in the skewed class.","The density-factor denoising step is a separable component: adding it to SMOTE-based recovery improved results in 16 of 18 experiments, so it can likely be attached to other oversampling-based recovery pipelines."],"supporting_citations":[{"why":"Supplies the oversampling engine: latent-space SMOTE interpolation used to create new skewed-class samples.","marker":"[11]"},{"why":"Baseline recovery method (FUDP retraining) that Imba-ULRc is compared against on skewed-class and global accuracy.","marker":"[23]"},{"why":"Baseline recovery method (QUICKDROP dataset distillation) that the paper must beat.","marker":"[10]"},{"why":"Baseline federated-learning method for label skew (restricted softmax) adapted as a recovery baseline.","marker":"[9]"},{"why":"Baseline federated-learning method (MOON contrastive loss) adapted as a recovery baseline.","marker":"[29]"},{"why":"Provides the unlearning algorithm (UPGA) used to produce the unlearning model that recovery training starts from.","marker":"[15]"},{"why":"Supplies the density-based local outlier idea used to justify the density-factor noise filter.","marker":"[37]"}],"fun_headline_variants":["Recover skewed-class unlearning with oversampling and denoising","Unlearned models on skewed data: synthetic oversampling repairs","Federated unlearning fix: SMOTE plus density filtering restores accuracy","Skewed labels no problem: oversampling and denoising rebuild models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Each remaining client must hold at least one real sample of the skewed class, because the oversampling step interpolates from existing samples and the autoencoder needs that class during training.","fun_headline_variants_meta":{"raw":{"variants":["Recover skewed-class unlearning with oversampling and denoising","Unlearned models on skewed data: synthetic oversampling repairs","Federated unlearning fix: SMOTE plus density filtering restores accuracy","Skewed labels no problem: oversampling and denoising rebuild models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1248,"prompt_tokens":944,"completion_tokens":304,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":226}},"tokens_in":560,"tokens_out":304,"duration_ms":3776,"temperature":1.0,"reasoning_tokens":226,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:05:54.239591+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the recovery pipeline on a split where one remaining client has zero skewed-class samples: the SMOTE step for that client is undefined and the code would have no seed to interpolate from. A recovery result on such a split would require a fallback mechanism the paper does not specify; conversely, confirming that accuracy collapses only in that zero-sample case would localize the method's boundary.","supporting_citations":[{"cited_title":"Deepsmote: Fusing deep learning and smote for imbalanced data","cited_arxiv_id":null,"evidence_quote":"Supplies the oversampling engine: latent-space SMOTE interpolation used to create new skewed-class samples."},{"cited_title":"QuickDrop: Efficient Federated Unlearning by Integrated Dataset Distillation","cited_arxiv_id":"2311.15603","evidence_quote":"Baseline recovery method (QUICKDROP dataset distillation) that the paper must beat."},{"cited_title":"Fedrs: Federated learning with restricted softmax for label distribution non-iid data","cited_arxiv_id":null,"evidence_quote":"Baseline federated-learning method for label skew (restricted softmax) adapted as a recovery baseline."},{"cited_title":"Lof: identifying density-based local outliers","cited_arxiv_id":null,"evidence_quote":"Supplies the density-based local outlier idea used to justify the density-factor noise filter."}],"review_version":1}