{"id":"fb10feaa-650a-4330-9438-717f8a438ef5","arxiv_id":"2502.05832","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Class imbalance degrades few-sample model compression, and an OOD-data-based framework (OE-FSMC) with complementary labels improves accuracy across several compression methods.","lead":"This paper studies what happens when a pre-trained model is compressed using very few samples from an imbalanced dataset, and proposes adding out-of-distribution images with invented labels to rebalance the training data. The method is aimed at privacy-sensitive deployments, such as small hospitals that can only access a handful of patient cases per disease.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The theoretical guarantee underpinning OE-FSMC is invalid: Eq. 4's complementary label distribution violates Theorem 1's uniform-label condition and, in a two-class example, flips the Bayes argmax, so the claimed 'harmless' OOD rebalancing is unsupported.","rationale":"After reading the full manuscript, the single most load-bearing point is not an experimental artifact but the theoretical guarantee that is supposed to make OOD data safe. The paper's abstract and Section 1 promise that OE-FSMC 'effectively mitigates' imbalance degradation; Section 4.2 claims a rigorous foundation from Wei et al. and states that Eq. 4 'ensures the stability of the Bayesian classifier's prediction.' If that guarantee fails, the framework's core assumption that noisy OOD labels are non-harmful is unsupported, and the reported gains could be due to hyperparameter choices rather than the proposed mechanism. The reader's weakest_assumption pointed to the unproved extension of Theorem 1 to Eq. 4 and to the compression pipeline; my analysis goes further and shows the extension is not merely missing but algebraically false for the natural disjoint-support interpretation of out-of-distribution data. I therefore agree with the reader's rejection: the manuscript's central claim cannot be accepted as written. This is not an ad hominem or a consensus disagreement; it is an internal contradiction between Eq. 4 and Theorem 1. The empirical direction may still have value, and a revised version that proves or replaces the theoretical claim, defines the undefined components (γ_y, p_j), and releases reproducible code with error bars could be reconsidered.","tokens_in":11948,"tokens_out":7465,"duration_ms":73599,"concrete_test":"Analytically check the two-class claim: set K=2, β=(0.9,0.1), and use Eq. 4 to compute Γ=(0.1,0.9). For any x in the in-distribution support and mixing proportion π=0.6, compare argmax_y P_s(x|y)P_s(y) with argmax_y P_mix(x|y)P_mix(y) under the disjoint-support assumption P_out(x)=0. If the argmax differs (it does by the algebra above), Theorem 1 does not extend to Eq. 4. A supplementary simulation with two Gaussian classes and a disjoint uniform OOD cloud would confirm whether the decision boundary shifts and by how much.","verdict_should_be":"REJECT","load_bearing_attack":"The central safety claim, used to justify adding OOD data, is the assertion in Section 4.2 that 'incorporating OOD data into the training set can be harmless' (Theorem 1) and that the complementary sampling strategy of Eq. 4 'ensures the stability of the Bayesian classifier's prediction.' But Theorem 1 is stated only for labels uniformly sampled from the label space, whereas Eq. 4 samples labels according to Γ_j = (α - β_j)/(Kα - 1), which is non-uniform whenever the original class priors β are imbalanced. The paper gives no proof that Eq. 4 inherits Theorem 1. The extension is not merely unproved; it is false under the natural reading. Take K=2 with β=(0.9,0.1). Then α=1, so Γ=(0.1,0.9), i.e., the complementary distribution is the reverse of the original prior. If the OOD input support is disjoint from the in-distribution support (which is what 'out-of-distribution' normally means), then for any x in the in-distribution support, P_mix(x|y) ∝ P_s(x|y) while P_mix(y) = (1-π)P_s(y)+πΓ(y). For any π>0.5, argmax_y P_mix(x|y)P_mix(y) becomes the minority class for the very points where the original Bayes classifier chose the majority class. Thus Eq. 4 actively changes the decision boundary rather than preserving it. Because the framework's benefit in the compression and fine-tuning stages is framed as 'non-harmful' OOD exploitation, this invalidates the theoretical basis of the central claim. The empirical gains reported in Tables 1-2 could still be real, but the paper's stated guarantee that OOD data does not hurt is contradicted by its own equation, and no experiment isolates whether the complementary labels preserve the teacher-student alignment.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses class imbalance in few-sample model compression, a practically important setting where only a small imbalanced training set is available. The authors first demonstrate empirically that existing few-sample compression methods (CD, FSKD, MiR, PRACTISE) lose accuracy under long-tailed imbalance. They then propose OE-FSMC, a framework that adds out-of-distribution (OOD) data with complementary labels during both the compression and fine-tuning stages, together with a joint distillation loss and a class-dependent regularization term. The experiments on long-tailed CIFAR-10/100 and ILSVRC-2012 show consistent accuracy improvements when OE-FSMC is combined with each of the four base methods. The paper claims a theoretical guarantee, based on Theorem 1 of Wei et al. (2022), that the complementary sampling strategy is harmless to the Bayesian classifier's prediction.","tokens_in":12319,"tokens_out":5765,"duration_ms":58073,"significance":"If the empirical claims hold, the paper addresses a real and understudied problem: few-sample model compression under class imbalance. The framework is simple, compatible with several existing compression pipelines, and shows consistent gains, especially at very small sample sizes. The experimental coverage is reasonably broad (three datasets, four base methods, ablations, hyperparameter sensitivity). The main value is empirical; however, the theoretical justification as written is invalid, and one component of the method (Eq. 6) contradicts its stated purpose. These issues must be fixed before the paper can be considered sound, but the core empirical idea may be salvageable after a substantial revision of the method description and theoretical claims.","major_comments":[{"comment":"The complementary label distribution Γ_j = (α - β_j)/(K·α - 1) is non-uniform whenever the class priors β are imbalanced, yet Theorem 1 is stated only for labels uniformly sampled from the label space. The paper provides no proof that Eq. (4) inherits the \"harmless\" property. In fact, the claim is false for a natural counterexample: with K=2, β=(0.9, 0.1), α=1, so Γ=(0.1, 0.9), and if the OOD support is disjoint from the in-distribution support (as the paper's OOD purity step implies), then for any in-distribution x the mixed posterior satisfies P_mix(x|y) ∝ P_s(x|y) while P_mix(y) = (1-π)β_y + πΓ_y. For any π > 0.5, argmax_y P_mix(x|y)P_mix(y) flips to the minority class for points where the original Bayes classifier chose the majority class. This directly contradicts the assertion that Eq. (4) \"ensures the stability of the Bayesian classifier's prediction\" and invalidates the theoretical basis for the framework's central safety claim. The authors must either prove a correct invariance condition for their sampling scheme, adopt a label distribution for which the theorem actually applies (e.g., uniform) and find another way to rebalance, or substantially reframe the method as a purely empirical heuristic without the \"non-harmful\" guarantee.","section":"§4.2, Eq. (4) and Theorem 1"},{"comment":"The class-aware weight w_j = p_j / Σ_k p_k, with p_j the relative frequency of class j, assigns smaller weights to minority classes, directly contradicting the prose that \"Minority classes are assigned higher weights through this formula.\" As written, Eqs. (5)-(6) would further downweight minority classes in the channel importance score, exacerbating the very imbalance problem the paper aims to solve. This is a load-bearing inconsistency in the method definition; it must be corrected (for example, by using an inverse-frequency weighting such as w_j = (1/p_j) / Σ_k (1/p_k)) and the surrounding text must be made consistent with the formula actually used in the experiments.","section":"§4.3, Eq. (6)"},{"comment":"Even if the Bayesian classifier invariance held, the statement \"we can infer that the feature space of the student model remains aligned with that of the teacher model on Dmix\" is a non-sequitur: Theorem 1 concerns the Bayes-optimal decision over the input space, not the intermediate feature alignment produced by distillation or pruning. The paper should either provide an argument connecting the two, or explicitly state this as an assumption that is not proven. As it stands, the theoretical pipeline from Theorem 1 to the method's design is incomplete.","section":"§4.2, inference from Bayes classifier to student feature alignment"},{"comment":"The analysis of pruning bias assumes that the importance score of a filter is a linear function of class frequencies, s_k^{(l)} = Σ_j (m_j/M) s_{k,j}^{(l)}. This is an illustrative assumption, but it is not generally true for the pruning criteria used by the evaluated methods (e.g., PRACTISE uses block dropping with a recoverability metric, not a class-frequency-weighted filter score). The theoretical motivation for why class imbalance harms compression should be reconciled with the actual mechanisms of the evaluated methods, or the analysis should be explicitly labeled as a motivating heuristic rather than a general derivation.","section":"§4.1, Eq. (1)"}],"minor_comments":[{"comment":"The symbol γ is used both for the complementary sampling rate in Eq. (4) and for the class-dependent regularization weight in Eqs. (8)-(9); using distinct symbols (e.g., Γ_j for the sampling rate and γ_y for the weight) would prevent confusion.","section":"§4.2-§4.3"},{"comment":"The text switches to first-person singular in several places (\"I will analyze\", \"I propose\") despite multiple authors; the manuscript should use \"we\" consistently.","section":"§4.1"},{"comment":"In the version provided, the text in Figure 3 is corrupted (sequences like \"/uni00000026/uni00000027/...\" appear instead of readable labels). The authors must ensure the final PDF renders all figure text correctly; as it stands, the figure is unreadable and the ablation discussion in §5.4 cannot be fully verified.","section":"Figure 3"},{"comment":"The ablation study is reported only through the unreadable Figure 3(a) and qualitative statements; no numerical results, error bars, or per-component accuracy values are given, making it hard to assess the magnitude of each component's contribution.","section":"§5.4"},{"comment":"The \"Num\" row mixes different total sample sizes across datasets (10/20/50/100 for CIFAR-10, 200/500/1000 for CIFAR-100, 1000/2000/3000 for ILSVRC-2012). The caption should clarify which numbers correspond to which dataset and that the ILSVRC-2012 settings correspond to 1/2/3 samples per class.","section":"Table 2"},{"comment":"The explanation that PRACTISE is less affected by imbalance \"likely because the block-dropping strategy is less sensitive to imbalance\" is speculative; a per-class accuracy breakdown or a latency-accuracy comparison under imbalance would substantiate this claim.","section":"§5.2"},{"comment":"Several reference entries have formatting issues (e.g., \"32rd\" in Zhou et al. 2020) and some entries are missing page numbers; a careful proofread of the bibliography is needed.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The empirical results are promising, but the theoretical section is currently invalid: the claimed extension of Theorem 1 to the complementary label distribution is unsupported and in fact false in a simple two-class case, and Eq. (6) contradicts the stated goal of increasing minority-class weights. These are load-bearing issues but are fixable by either proving a correct invariance result, changing the sampling scheme, or reframing the paper as an empirical study and removing the theoretical guarantee. I would encourage the authors to also clarify what is novel relative to Wei et al.'s Open Sampling, since the core OOD-rebalancing idea is imported from that work; the novelty in this manuscript appears to be the application to few-sample compression and the joint distillation/regularization components. If the authors cannot repair the theoretical claims, the paper should be repositioned honestly as an empirical heuristic, which would still be of interest to the compression community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the paper identifies a real and under-studied problem—class imbalance in few-sample model compression—and shows consistently that imbalance hurts existing methods. That part is solid and worth knowing. The proposed fix, OE-FSMC, is a simple plug-in using OOD data with complementary labels, and the empirical results across CD, FSKD, MiR, PRACTISE on CIFAR and ILSVRC mostly show gains. The ablation and sensitivity analysis are honest and useful.\n\nThe new contribution is genuinely the combination, not the mechanism: complementary label assignment comes from Open Sampling (Wei et al. 2022), and the paper says so. That is fine, but it means the novelty is at the level of application and empirical demonstration, not theory.\n\nNow the soft spots, in order of severity. Section 4.2 claims Theorem 1 of Wei et al. extends to the complementary label distribution of Eq. 4, but Theorem 1 is stated only for uniform labels. The paper gives no proof for the non-uniform case, and the claim is actually false. With two classes, beta=(0.9,0.1), Eq. 4 gives Gamma=(0.1,0.9); for OOD support disjoint from in-distribution, mixing at pi>0.5 inverts the class prior on in-distribution points and flips the Bayes argmax. So the stated guarantee is wrong under the paper's own equations. This is load-bearing because the framework's safety is framed as 'harmless' OOD rebalancing.\n\nSecond, Eq. 6 contradicts the prose. The text says minority classes get higher weights through this formula, but w_j = p_j / sum p_k with p_j the relative frequency does the opposite. Either the formula or the description is wrong, and since the pruning adjustment is one of the two main components, this matters.\n\nThird, the class-dependent weight gamma_y in Eqs. 8-9 is never defined, so the regularization term is underspecified. Fourth, there are no error bars, no code, and the hyperparameters (lambda, eta) are tuned on the test set. That limits how much the empirical claims can be trusted, though the gains are large enough that the direction is probably right.\n\nCitations are fine: the main borrowed idea is attributed, and the one self-citation (He et al. 2024) is not load-bearing.\n\nWho is this for? Someone working in few-sample compression or edge deployment with imbalanced medical/financial data. It deserves a serious referee because the problem is real and the method may work after fixing the internal contradictions and adding rigor. As it stands, the paper cannot support the method as written; I would send it back for major revision, with the theoretical claim either removed or properly proved and Eq. 6 corrected.","headline":"Real problem, plausible empirical gains, but the safety theorem as stated is false and Eq. 6 contradicts its own prose; major revision before it can be trusted.","tokens_in":12907,"tokens_out":2491,"would_cite":false,"duration_ms":23782,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Out-of-distribution images with complementary labels restore accuracy in few-sample model compression, lifting four existing methods on long-tailed CIFAR-10/100 and ILSVRC-2012.","keywords":["few-shot learning","network compression","class imbalance","knowledge distillation","out-of-distribution data","model pruning","long-tailed distribution","image classification"],"falsifier":"On the same long-tailed CIFAR-10 splits used in the paper, compare OE-FSMC against a variant that labels the OOD images uniformly at random rather than by Eq. 4; if uniform labeling matches OE-FSMC's accuracy, the complementary-label rule is not the active mechanism, and if OE-FSMC fails to beat the no-OOD baseline on minority-class accuracy, the claimed non-harmfulness extension is falsified. A reader could also measure teacher--student penultimate-layer similarity on minority-class validation images after compression to test the asserted feature-alignment mechanism directly.","tokens_in":11689,"feed_emoji":"⚖️","tokens_out":13470,"duration_ms":112687,"temperature":0.7,"pith_summary":"Few-sample model compression—turning a large pretrained network into a small deployable one using only a handful of images—is attractive where privacy rules out sharing a full dataset. The paper argues that existing methods were validated on artificially balanced splits (every class equally sampled) and therefore missed a practical failure mode: when the few available samples are long-tailed, minority-class filters get pruned, distillation drifts toward majority classes, and fine-tuning cannot bring the lost features back. It introduces OE-FSMC, a plug-in framework that adds readily available out-of-distribution (OOD) images, labels them with a complementary distribution that rebalances the class prior, and guards against OOD overfitting with a joint distillation loss and a class-dependent regularizer. Wrapped around four existing methods (CD, FSKD, MiR, and PRACTISE) on long-tailed CIFAR-10/100 and ILSVRC-2012, OE-FSMC improves top-1 accuracy, with the largest gains at the smallest sample sizes. If the paper is right, privacy-sensitive deployments can fix imbalance without collecting more in-distribution samples.","feed_headline":"Out-of-distribution data rescues compression from class imbalance","feed_subtitle":"A plug-in framework lifts accuracy across four few-shot compression methods on long-tailed CIFAR and ILSVRC.","key_machinery":"The load-bearing object is the complementary label distribution, $\\Gamma_j = (\\alpha - \\beta_j)/(K\\alpha - 1)$, which converts the imbalance in the few-sample set into a rebalanced prior for the mixed dataset. Around it the framework adds three mechanisms: class-aware pruning weights $w_j = p_j/\\sum_k p_k$, which rescale filter importance so minority-class channels are not preferentially removed; the joint distillation loss $L = \\lambda L_{\\mathrm{KD,Few}} + (1-\\lambda)L_{\\mathrm{KD,OOD}}$, which pools knowledge from real and OOD data while letting $\\lambda$ control the balance; and the fine-tuning regularizer $L_{\\mathrm{total}} = \\mathbb{E}[\\ell(f(x;\\theta),y)] + \\eta \\mathbb{E}_{\\tilde x \\sim P_{\\mathrm{aux}}}[\\gamma_{\\tilde y}\\ell(f(\\tilde x;\\theta),\\tilde y)]$, whose class-dependent weight $\\gamma_{\\tilde y}$ protects minority classes from OOD overfitting. The theoretical anchor is the cited theorem that OOD augmentation with uniformly sampled labels leaves the Bayesian prediction unchanged, which the paper extends, by assertion rather than proof, to complementary labels and to the compression pipeline.","core_discovery":"The paper's central claim is that class imbalance is a first-order cause of accuracy loss in few-sample model compression, not a nuisance that fine-tuning can absorb, and that the loss can be repaired by rebalancing the training distribution with out-of-distribution (OOD) data. Each OOD image receives a label drawn from the complementary sampling rule $\\Gamma_j = (\\alpha - \\beta_j)/(K\\alpha - 1)$, where $\\beta_j = m_j/M$ is the original class frequency and $\\alpha = \\max_j \\beta_j + \\min_j \\beta_j$, so the mixed class prior approximates the uniform test prior and minority classes receive more auxiliary samples. During pruning, the channel importance score is reweighted by class-aware weights $w_j = p_j / \\sum_k p_k$ to protect minority-class filters; during distillation and fine-tuning, a joint loss $L = \\lambda L_{\\mathrm{KD,Few}} + (1-\\lambda)L_{\\mathrm{KD,OOD}}$ and a class-dependent regularization term keep the OOD signal from dominating. Experiments wrapping CD, FSKD, MiR, and PRACTISE in this framework on long-tailed CIFAR-10/100 and ILSVRC-2012 report consistent top-1 accuracy gains, most pronounced at 10--20 training samples. The paper grounds the safety of OOD augmentation in a cited theorem stating that uniformly labeled OOD data does not change a Bayesian classifier's prediction, and asserts that this alignment carries over to complementary labels and to the compression and fine-tuning pipeline.","pith_inferences":["Because OE-FSMC acts on the data distribution rather than on the loss landscape, it should compose with algorithmic imbalance remedies such as logit adjustment or class-balanced reweighting; stacking them might close more of the gap to an oracle with real minority-class samples.","A testable boundary condition is the distance between the OOD pool and the in-distribution domain: the complementary-label argument predicts graceful degradation only while OOD images carry no information about true classes, so measuring accuracy against OOD-pool similarity would map the safety envelope.","The framework's success implies a practical protocol for privacy-preserving deployment: local institutions can simply use whatever images they have on hand as a balancing resource, rather than being asked to collect rare-class examples they cannot obtain."],"forward_implications":["The four existing compression methods can adopt OE-FSMC without changing their network architecture or reported hyperparameters, making the fix additive rather than a redesign.","The accuracy gain is largest when the training set has only 10--20 samples, so the method is most useful exactly where imbalance is hardest to correct manually.","An OOD pool of about 500 pre-sampled images, with the auxiliary set roughly the size of the few-sample set, suffices to capture the benefit, easing storage pressure on edge devices.","Both integration points contribute to the gain: OOD data during compression preserves minority-class filters that fine-tuning cannot later restore, whereas OOD data during fine-tuning rebalances the classifier; dropping either component lowers accuracy.","The authors position the framework as structure-agnostic and point to model quantization as the natural next compression family to try."],"supporting_citations":[{"why":"Supplies the Open Sampling rule and Theorem 1, the theoretical basis for claiming complementary-labeled OOD data is non-harmful.","marker":"Wei et al. (2022)"},{"why":"CD, the cross-distillation compression method that serves as one of the four base methods OE-FSMC wraps.","marker":"Bai et al. (2020)"},{"why":"FSKD, the few-shot knowledge distillation method used as another base method and improved by OE-FSMC.","marker":"Li et al. (2020)"},{"why":"MiR, the mimic-then-replace compression method used to test the framework's generality.","marker":"Wang et al. (2022)"},{"why":"PRACTISE, the block-dropping acceleration method whose pruning strategy shows the smallest imbalance drop but still improves under OE-FSMC.","marker":"Wang and Wu (2023)"},{"why":"Surveys few-shot learning with class imbalance and frames the CIFSL setting that motivates an OOD-based rebalancing approach.","marker":"Ochal et al. (2023)"},{"why":"Supplies CIFAR-10/100 and their long-tail imbalance protocol for constructing the imbalanced few-sample training sets.","marker":"Krizhevsky and Hinton (2009)"}],"fun_headline_variants":["OOD data rebalances few-shot compression, lifts accuracy","Class imbalance in few-shot compression? Add OOD data","Few-shot compression fix: OOD data rebalances classes","Plug-in OOD method boosts few-shot compression accuracy","OOD-Enhanced compression: rebalance classes, improve accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's safety rests on the claim that a theorem proven for OOD data with uniformly sampled labels also holds when labels are drawn from the complementary distribution of Eq. 4 and when the data flows through pruning, distillation, and fine-tuning; the paper asserts this extension without proof, and if the OOD data instead pulls the student model away from the true in-distribution classes, the accuracy gains would reverse.","fun_headline_variants_meta":{"raw":{"variants":["OOD data rebalances few-shot compression, lifts accuracy","Class imbalance in few-shot compression? Add OOD data","Few-shot compression fix: OOD data rebalances classes","Plug-in OOD method boosts few-shot compression accuracy","OOD-Enhanced compression: rebalance classes, improve accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000208,"raw_usage":{"total_tokens":1462,"prompt_tokens":1059,"completion_tokens":403,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":675,"completion_tokens_details":{"reasoning_tokens":322}},"tokens_in":675,"tokens_out":403,"duration_ms":4169,"temperature":1.0,"reasoning_tokens":322,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:47:59.143829+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the same long-tailed CIFAR-10 splits used in the paper, compare OE-FSMC against a variant that labels the OOD images uniformly at random rather than by Eq. 4; if uniform labeling matches OE-FSMC's accuracy, the complementary-label rule is not the active mechanism, and if OE-FSMC fails to beat the no-OOD baseline on minority-class accuracy, the claimed non-harmfulness extension is falsified. A reader could also measure teacher--student penultimate-layer similarity on minority-class validation images after compression to test the asserted feature-alignment mechanism directly.","supporting_citations":[],"review_version":1}