{"id":"7a427a41-198f-4972-bfb6-5dd1abd95bd2","arxiv_id":"2502.06878","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"AutoSMOTE uses Gumbel-Softmax to learn discrete oversampling rules per minority sample, reporting modest average-rank gains over SMOTE-family baselines.","lead":"This paper proposes AutoSMOTE, an end-to-end oversampling framework that learns discrete decisions (which samples to use, how many neighbors, which aggregation) for imbalanced classification. The authors claim better accuracy and generalization than existing oversamplers, but the experiments select hyperparameters on the test set and the theoretical proof is largely assumed.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported superiority is not supported because hyperparameters, including the extra cohort group count, are selected on the test split.","rationale":"The reader's weakest assumption is Theorem 2, and that theorem is indeed not a proof: it assumes the VC-dimension ordering follows from hypothesis-class nesting and assumes without justification that generalization error is proportional to VC dimension. I agree that the theoretical claim is unsupported. However, the most load-bearing flaw for the central 'superiority' claim is the evaluation protocol. Section 5.4 explicitly tunes hyperparameters on the only held-out split and reports the best metric, which makes Table 1, Table 2, and Figures 2 and 3 unreliable as evidence of generalization. This concern is concrete and testable. My emphasis differs from the reader's stated weakest assumption, though the reader's rationale also mentions the test-set issue, so my agreement is partial. The verdict remains REJECT: the underlying idea of learnable discrete oversampling might be salvageable with proper validation, but the current empirical and theoretical claims are not supported.","tokens_in":14893,"tokens_out":6002,"duration_ms":58871,"concrete_test":"Re-run the Section 6 comparison under a nested protocol: hold out the 20% test set, choose k in {2,...,6} (and group count in {1,...,7} for AutoSMOTEcohort) on a separate validation split or via cross-validation, fix the selected hyperparameters, and evaluate each method once on the held-out test set. If AutoSMOTEcohort is no longer best in average rank, or the differences fall within the reported standard deviations, the claimed superiority is an artifact of test-set selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The experimental evidence for the central claim is invalidated by the model-selection protocol. Section 5.4 says the data is split once into 80% training and 20% testing, then \"we run each model across 2 to 6 nearest neighbors and record the best performance metric, enabling fair comparison,\" and for AutoSMOTEcohort the number of groups is searched over 1 to 7 and \"record the best result.\" No validation set is described. Consequently, every entry in Table 1 and the average ranks in Table 2 are the result of selecting hyperparameters on the test split rather than an unbiased estimate of generalization. AutoSMOTEcohort has the extra tunable group count, so it has more chances to win by test-set overfitting. Section 6.1's claim of lower test error than MLP is subject to the same bias. Unless the comparison is redone with validation-based tuning and a single final test evaluation, the reported superiority is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AutoSMOTE, a deep-learning-based oversampling framework for imbalanced classification. The method formulates oversampling as a composition of discrete decision criteria (oversampling participation, k-nearest-neighbor count, and aggregation function), learned end-to-end with MLPs and a Gumbel-Softmax relaxation. Two variants are introduced, AutoSMOTEself (per-instance decisions) and AutoSMOTEcohort (decisions applied to learnable groups of minority instances). The manuscript claims a theoretical generalization bound via VC dimension that ranks the two variants below a plain MLP in generalization error, and it reports experiments on eight tabular datasets claiming that AutoSMOTEcohort achieves the best overall average rank and lower test error than MLP-based oversampling.","tokens_in":15145,"tokens_out":2905,"duration_ms":29560,"significance":"The core idea of treating oversampling as a set of learnable discrete decisions that are jointly optimized with the classifier is genuinely interesting and could be a useful contribution to imbalanced classification. The paper also provides an ablation study separating the effect of each decision criterion and reports a runtime comparison, which are valuable elements. However, the central empirical claim of superiority is invalidated by the model-selection protocol, and the theoretical result is not actually proven. If the method were evaluated with a proper validation-based hyperparameter selection and the theoretical claim were either rigorously established or removed, the underlying framework could be of interest to the community, but in its current form the paper's headline conclusions are not supported.","major_comments":[{"comment":"The hyperparameter selection protocol leaks test information. Section 5.4 states that 'we run each model across 2 to 6 nearest neighbors and record the best performance metric' and for AutoSMOTEcohort 'we perform a search across 1 to 7 groups and record the best result,' but no validation set is described; the data is split once into 80% training and 20% testing. Therefore the numbers in Tables 1, 2, and 3 are selected on the test split rather than being unbiased estimates of generalization. AutoSMOTEcohort has an additional tunable parameter (the number of groups), so it has more opportunity to benefit from this test-set overfitting, directly undermining the claim in Table 2 that it achieves the best overall rank and the Section 6.1 claim that it has lower test error than the MLP oversampler. This is a load-bearing flaw because the main contribution is empirical superiority.","section":"5.4"},{"comment":"The proof of Theorem 2 is not a derivation. It assumes without proof that HAutoSMOTEcohort ⊂ HAutoSMOTEself ⊂ HMLP, then asserts 'it is reasonable to expect that their VC dimensions would follow the same order,' and finally uses an asserted proportionality ϵgen(H) ∝ VCdim(H). None of these steps is established: hypothesis class inclusion is not formally shown from the architectural description; VC dimension does not in general follow from class inclusion; and Definition 4 provides an upper bound involving sqrt(VCdim log N/N), not a proportionality of the generalization error itself. The theorem's conclusion therefore rests on the very ordering it purports to prove. Section 6.1 claims to empirically validate the theorem, but the empirical comparison is already invalidated by the test-set selection issue in Section 5.4, and training error curves (Figure 2) do not directly demonstrate a VC-dimension ordering.","section":"4, Theorem 2"}],"minor_comments":[{"comment":"The variant names are inconsistently and incorrectly typeset as 'AutoSM OT Eself' and 'AutoSM OT Ecohort' in Section 3.5.2, which should be cleaned up throughout.","section":"3.5.2"},{"comment":"There are grammatical issues such as 'It has of 5473 data instances' (Page-blocks), 'It contains 2417 data instances with 103 attributes' (Yeast) where 'attributes' is likely 'features,' and 'datasets related for diverse fields' in the opening sentence of Section 5.1.","section":"5.1"},{"comment":"The phrase 'MPLs' appears in the Related Work section ('MPLs have higher interpretability'), apparently a typo for 'MLPs.'","section":"2"},{"comment":"For the Wisconsin dataset, the reported Precision, Recall, and F1-score for AutoSMOTEself and AutoSMOTEcohort are identical to three decimal places, which is suspicious and likely a copy-paste error; the authors should confirm which values are correct.","section":"Table 1, Wisconsin"},{"comment":"Figures 2 and 3 are referenced but not visible in the manuscript text provided for review; the authors should ensure the figures are included and clearly labeled, and ideally also report the numerical values of training and test error so the claimed trends can be assessed.","section":"6.1"}],"recommendation":"reject","confidential_remarks":"The two major issues are load-bearing and together undermine the paper's central claims. The test-set hyperparameter selection is a fundamental methodological error that invalidates all reported performance comparisons, and the theoretical result is not proven but merely asserted through an unverified proportionality and class-nesting assumption. Even though the underlying framework idea is promising, these flaws cannot be repaired by local edits; they require redoing the experimental evaluation with a proper validation protocol and either proving or substantially rewriting the theoretical claim. Given the current state, I cannot recommend publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is real: a learnable, end-to-end oversampler where discrete choices—which minority instances to use, how many neighbors, which aggregation function—are made by Gumbel-Softmax MLPs jointly trained with the classifier. That is a legitimate twist on OC-SMOTE-NN and worth exploring. The two variants are coherent and the ablation study asks the right questions. But the paper's two load-bearing pillars do not hold up.\n\nFirst, the experimental protocol is invalid. Section 5.4 says the data are split once into 80% training and 20% testing, and then each model is run across 2 to 6 nearest neighbors “and record the best performance metric.” For AutoSMOTEcohort, the number of groups is searched over 1 to 7 and the best recorded. There is no validation set. Every entry in Tables 1–2 and Figure 3 is therefore the product of selecting hyperparameters on the test split, and the cohort variant has extra knobs to overfit with. The reported superiority is not supported. Second, Theorem 2 is not a proof. It assumes the hypothesis class nesting, asserts without derivation that the VC dimensions follow the same order, and then asserts ϵgen ∝ VCdim to conclude the generalization ordering. Section 6.1's “validation” is a qualitative look at training/test curves and does not rescue it.\n\nI want to give credit where it is due: the framework is clearly explained, the related work is covered reasonably well, and the authors are not sloppy writers. The problems are deeper than presentation. Some smaller issues also bother me: missing ADASYN results for several datasets, identical standard deviations across different methods in places, and no statistical significance testing anywhere. The claim that MLPs have higher interpretability than GANs is at least debatable and could use support.\n\nBottom line: this is a salvageable paper, not a publishable one as-is. The mechanism is novel and the flaws are fixable. A proper revision would use a validation-based tuning protocol, either produce a real generalization bound or drop the theory, and add proper statistical comparisons. If this crossed my desk, I would send it to reviewers with that explicit framing rather than desk-reject it, because the core idea deserves referee time. But I would not cite it in its current form, and I would not hold it up as a model of experimental practice.","headline":"A genuinely novel oversampling mechanism, but the test-set hyperparameter tuning and the circular VC-dimension 'proof' sink the paper's central claims.","tokens_in":15594,"tokens_out":2686,"would_cite":false,"duration_ms":27002,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that oversampling for imbalanced classification can be reformulated as a composition of learnable discrete decision criteria, packaged as an end-to-end deep framework (AutoSMOTE) whose cohort variant achieves the best…","keywords":["class imbalance","oversampling","deep learning","multi-layer perceptron","representation learning","Gumbel-Softmax","VC dimension","synthetic minority oversampling"],"falsifier":"Train AutoSMOTEcohort, AutoSMOTEself, and an MLP oversampler to matched training error on an imbalanced dataset and compare held-out test error; if the MLP has lower test error, or if the ordering is not cohort < self < MLP, the claimed generalization hierarchy is contradicted.","tokens_in":14718,"feed_emoji":"⚖️","tokens_out":7615,"duration_ms":64074,"temperature":0.7,"pith_summary":"The paper argues that oversampling done as a separate preprocessing step is mismatched with the downstream classifier, because synthetic samples are generated without regard to the loss that the model will be optimized for. It introduces AutoSMOTE, a deep learning framework that folds oversampling into training: for each minority instance, the model learns whether to oversample it, how many nearest neighbors to use, and which aggregation function to apply. These discrete choices are made differentiable with Gumbel-Softmax, so the sampling policy and the classifier are trained jointly by backpropagation. The paper reports that this decision-based formulation improves generalization relative to plain MLP oversamplers, and that AutoSMOTEcohort ranks first overall on eight imbalanced datasets.","feed_headline":"AutoSMOTE beats classic and deep oversamplers","feed_subtitle":"It learns participation, neighbor count, and aggregation per instance, ranking first across eight datasets.","key_machinery":"The load-bearing mechanism is a decision criteria mapping ψj = μj ∘ Φj, where Φj maps a minority instance to an integer and μj maps that integer to one of finitely many predefined decisions. For each of the three criteria—oversampling participation, k-nearest-neighbor count, and aggregation function—an MLP produces class probabilities, and Gumbel-Softmax converts them into differentiable one-hot choices. The chosen decisions are composed to generate a synthetic sample: if participation is selected, the chosen k neighbors are combined by the chosen function, drawn from linear interpolation, minimum, maximum, sum, average, and weighted average. This composition defines the hypothesis classes whose nesting (cohort ⊂ self ⊂ MLP) drives the VC-dimension generalization argument.","core_discovery":"The central discovery is that an oversampling function is better modeled as a composition of discrete decision criteria than as a free-form continuous function. AutoSMOTEself makes the three decisions per minority instance independently, while AutoSMOTEcohort first learns a grouping of minority instances and applies the decision criteria to each group collectively. Both variants are trained end to end with an MLP classifier under categorical cross-entropy, with Gumbel-Softmax supplying gradients through the discrete selections. The paper's theoretical claim is that the resulting hypothesis classes are nested—AutoSMOTEcohort inside AutoSMOTEself inside the MLP oversampler—so their VC dimensions (a standard capacity measure) and hence their generalization errors are ordered accordingly, which the authors then validate empirically.","pith_inferences":["Editorial inference: the same decision-criteria composition could be applied to undersampling or cost-sensitive weighting, since the mechanism only requires differentiable selection over discrete choices.","Editorial inference: because participation, k, and aggregation are interpretable, the learned policy can be inspected to see which synthetic strategy the model favors in different minority regions, giving a post-hoc explanation of when a dataset benefits from interpolation versus extremal aggregation.","Editorial inference: a natural extension is to test whether the relative importance of the three criteria shifts with imbalance ratio; the paper's ablation covers only three datasets, so the pattern across a wider range of skews remains an open question."],"forward_implications":["If the generalization ordering holds, AutoSMOTEcohort should consistently beat AutoSMOTEself and MLP oversamplers on held-out test sets, not only on the eight datasets reported.","Because the oversampling policy is trained jointly with the classifier, the framework can be attached to any differentiable classifier without redesigning the sampling step.","The restricted hypothesis class lowers the risk of overfitting in small minority classes, which matters for high-dimensional or sparse tabular datasets.","The learned aggregation functions allow the model to switch between SMOTE-style interpolation and nonlinear summaries per instance, making the synthesis policy more flexible than a fixed SMOTE."],"supporting_citations":[{"why":"Defines SMOTE and linear interpolation, the core baseline and one of the aggregation functions the framework can select.","marker":"[8]"},{"why":"Supplies the instance-level oversampling formalization that AutoSMOTE extends with learnable decision criteria.","marker":"[30]"},{"why":"Gives the universal approximation theorem for MLPs, which motivates the expressive-baseline comparison.","marker":"[31]"},{"why":"Provides Gumbel-Softmax, the reparameterization trick that makes the discrete decision criteria trainable.","marker":"[32]"},{"why":"States approximation capabilities of multilayer feedforward networks, used alongside the universal approximation theorem.","marker":"[34]"},{"why":"Defines VC dimension, the capacity measure on which the generalization ordering in Theorem 2 is built.","marker":"[35]"},{"why":"DeepSMOTE is a deep-learning oversampling baseline the experiments must beat.","marker":"[14]"},{"why":"OC-SMOTE-NN is a learnable oversampling baseline that the paper compares against.","marker":"[15]"}],"fun_headline_variants":["Discrete oversampling outperforms continuous and deep baselines","AutoSMOTE learns per-instance decisions for imbalanced data","Modeling oversampling as discrete criteria improves generalization","Nested hypothesis classes explain oversampling performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the nested hypothesis classes have VC dimensions in the same order and that generalization error scales with VC dimension; the paper asserts both steps as reasonable expectations rather than proving them.","fun_headline_variants_meta":{"raw":{"variants":["Discrete oversampling outperforms continuous and deep baselines","AutoSMOTE learns per-instance decisions for imbalanced data","Modeling oversampling as discrete criteria improves generalization","Nested hypothesis classes explain oversampling performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000678,"raw_usage":{"total_tokens":3004,"prompt_tokens":791,"completion_tokens":2213,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":407,"completion_tokens_details":{"reasoning_tokens":2150}},"tokens_in":407,"tokens_out":2213,"duration_ms":18687,"temperature":1.0,"reasoning_tokens":2150,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:48:21.737167+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train AutoSMOTEcohort, AutoSMOTEself, and an MLP oversampler to matched training error on an imbalanced dataset and compare held-out test error; if the MLP has lower test error, or if the ordering is not cohort < self < MLP, the claimed generalization hierarchy is contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines SMOTE and linear interpolation, the core baseline and one of the aggregation functions the framework can select."},{"cited_title":"Velayuthan, N","cited_arxiv_id":null,"evidence_quote":"Supplies the instance-level oversampling formalization that AutoSMOTE extends with learnable decision criteria."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Gumbel-Softmax, the reparameterization trick that makes the discrete decision criteria trainable."},{"cited_title":"Hornik, Approximation capabilities of multilayer feedforward net- works, Neural networks 4 (2) (1991) 251–257","cited_arxiv_id":null,"evidence_quote":"States approximation capabilities of multilayer feedforward networks, used alongside the universal approximation theorem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines VC dimension, the capacity measure on which the generalization ordering in Theorem 2 is built."},{"cited_title":"Dablain, B","cited_arxiv_id":null,"evidence_quote":"DeepSMOTE is a deep-learning oversampling baseline the experiments must beat."},{"cited_title":"Karunasingha, B","cited_arxiv_id":null,"evidence_quote":"OC-SMOTE-NN is a learnable oversampling baseline that the paper compares against."}],"review_version":1}